Website outage report

Hi all,

Following up from the website outage on Friday, we have conducted a Post-Mortem to identify measures we can take to mitigate a similar incident occurring in the future. In this review we also look at our general incident response process as a whole to continually improve how we work.

For anyone that missed it, our main marketing site https://mycroft.ai went down due to a HackerNews “Hug of Death”. This did not impact https://home.mycroft.ai or any device related services.

Some data from the Store was temporarily reverted in bringing the site back online however this will be restored over the coming week. If your purchase was included in this we will reach out directly to confirm those details.

We like to be open about any incidents that occur and as such publish live incident reporting as it’s happening as well as an overview of our response measures. You can find that incident report here.

Big thanks to those who promptly reported the issue to us :slight_smile:

1 Like

it also appears the Picroft_Buster-Keaton_2020-01-10.zip file is corrupted .

Hey Kent, what are you seeing with the Picroft image?

Its most likely a corrupted download as this is hosted independently.

Are you getting the same SHA-256 hash: 9cd9ec16388d72dfc6069bc7ea62d1b73e944df103571f48c88a49ec09f5f6d4

You can check that using a browser based checker like https://md5file.com/calculator if you don’t have one available on your machine. Don’t worry this doesn’t upload the entire image to the server, it just calculates the hash locally in your browser.

@gez-mycroft I read the Incident Report with interest, as both a Mycroft fan and as a WordPress specialist experienced in scaling it to perform at scale.

I’ve mostly been lurking here, but I’m passionate about the project and learning Python is on my todo list because of Mycroft… If there’s a faster way for me to make some contributions, this may be it!

Quoting the Incident Report, one of the mitigation strategies is “Investigate further scaling optimisations”. If there is still a need, I’d love to offer some suggestions, and I’m available for hands-on help if desired.

Hi Matthew, welcome to the forums and thanks for the offer. There’s definitely more ways to help Mycroft than Python, I’ll PM you :slight_smile:

1 Like