Originally published at: http://mycroft.ai/blog/mark-ii-update-march-2020/
As a small startup focused on getting a product into people’s hands as soon as possible, we have generally focused on pushing features. You write a few tests, everything seems to work, so you ship it and move on. At the time everything works well enough, and you have enormous goals, so you let a few things slide and jump into adding the next feature.
However every time you let something small slide, you leave behind a small bug. None of these are big issues so you fix some, brush others aside, and soldier on. Over time bugs start to get harder to diagnose. A bug report comes in, you investigate, think you’ve spotted it, but actually it was caused by another bug elsewhere in the software stack.
At this point it’s tempting to throw a patch onto it and get back to the “real work” shipping features. But the longer you leave it the harder it gets to maintain, and you actually spend more time chasing more complex bugs than if you addressed the underlying issues.
We have started to travel down this path. When writing complex software It’s almost impossible not to. However, the Mark II hardware delay has given us a fresh perspective. We need to solidify our existing stack by doing the boring hard work that will make it more robust, be more adaptable to new hardware platforms, and provide a reliable and seamless experience for the Mark II.
The key focus for March has been improving our test frameworks. Our overarching goal for these tests is to build an automated way to ensure a user’s interactions with the voice assistant software are always meeting expectations.
Mycroft-core has always utilized unit tests to ensure specific functions operate as expected. Skills have also had integration tests to verify that for a given utterance the right Skill is invoked and that it responds appropriately. These tests have generally been written to check on the primary functions and “happy paths” for success. They test known and correct input returns the correct output. They do not always test what happens when something unexpected happens.
To this end we’ve been expanding our use of Behave. This is a Behavior Driven Development (BDD) framework that we have been using in our Selene Backend. For those interested, we’ll be posting a full introduction to our implementation of this framework, which we’ve labeled “Voight-Kampff”, on our blog shortly.
The better we can test the software, the quicker we can move in development, and the more reliable the Mark II will be for you.
The next milestone the dev team will focus on is improving the reliability and consistency of packaging and deployment through the use of container technologies.
Currently to install Mycroft you must clone a Github repository and run an installation script. After answering some questions our scripts will setup your system, install dependencies from your operating system, the Python Packaging Index, and some directly from Github. This all works most of the time. The challenge being that differences in the system underneath can cause unexpected issues during installation or use.
Some of these mechanisms are also less robust than others. Whilst it is rare, if an error occurs during some git processes, it can cause corruptions in the git tree that require manual interventions to fix. Asking a Mark II customer who purchased a consumer ready device to “prune their orphaned git objects” is just not going to cut it.
Enter Snapcraft by Canonical. Snaps are containerized applications for Linux. Packaging up all of the dependencies for your application within the Snap itself makes it self-contained. This removes the possibility of dependency conflicts, ensures consistent and repeatable builds, and makes the application platform agnostic. Using Snaps means we are also able to harness Snapcraft’s release processes for updating devices and verifying the installs.
As the Snapcraft ecosystem is new to us, we’ve started by Snapping some component technologies first. This will allow us to properly understand the technology and package up useful Mycroft components that can be used in the final Mycroft-core Snap. Mimic1, our on-device text-to-speech engine, has been Snapped and is currently in testing. Precise, our wake word spotter, is up next. Then both of these will be published in the Snapcraft Store for use in the official Mycroft AI Snap.
If you are interested in the Snap packaging system and want to help us out, please join us in the Linux Packaging channel on Mycroft Chat.