OpenVoiceOS - A bare minimal (production type of) OS based on Buildroot

Just some more teasers of what is coming :innocent:

2 Likes

Further, overdue status report:

Mk2:

Today, September 3rd, @j1nx successfully activated the mic array in the Mk2’s audio board (the SJ-201).

Our thanks and congratulations to MycroftAI on selecting this hardware and designing this board. It’s a high-quality mic array, set up to cope well with its proximity to powerful speakers, and it’s got buttons. We couldn’t have done better.

The problem we were experiencing had less to do with not being MycroftAI, and more to do with running the newest Linux kernel. MycroftAI would likely experience the same problem in an upcoming version - however, j1nx’s driver is expected to work on other systems and devices, so it will be submitted not only to MycroftAI, but also to the microphone manufacturer. This was no mean feat, and @j1nx is due thanks and congratulations!

Also, Mk2s no longer start at max volume! They should start out at 50%.

Skill Store Skill Synchronized with Skill System:

Our initial prototype for the Skill Stores’ visual skill is working, in the way initial prototypes work.

That’s a mouthful, so let me clarify: it’s a Skill that acts as a frontend to OSM, our multi-marketplace skill manager. It’s not the one where you say, “Hey Mycroft, install .” It’s the one where you launch an App Store interface which you can browse with your eyes and your fingers and a search box, and install skills by tapping them.

A tabbed interface represents different skill stores and repositories. Within each tab, users will find a swipeable catalog. Tap a skill, and a popup displays its description above an ‘Install’ button. Tap outside the popup, and it disappears.

The Skill Store Skill is not ready to distribute, but we are pleased to have a working prototype. Testers can expect access within a couple of weeks, possibly sooner.

Other app store integrations exist for other @Aix blessed systems, but, to the best of my knowledge, this is the first visual, Mycroft-first, totally jailbroken skill store… skill.

Peter Piper picked a peck of pickled peppers.

General:

  • Brightness and volume sliders hooked up, in both directions. If you slide, the volume changes. If you tell Mycroft to change the volume, the slider changes. Congratulations to @Aix, and whichever of my fellow Pythonistas hooked it up to the skill.

  • Mycroft’s “internal” volume control changes system volume, as well

  • Other main menu options:

    • Select your home screen: under the swipe-down menu > settings, you can select our home screen, the default Mk2 home screen, and Jarbas mode parrots! Other home screen skills are possible.

    • Rotate the screen! (clockwise)

    • Weather bug fixed for some users

  • “Change the wallpaper” changes the wallpaper. This has been around for a while, but I don’t think we announced it.

Upcoming:

  • Over the past few days, several of our devs have been working at a feverish pace on improvements to OSM, and another round of improvements is on the way

  • Tester access to the aforementioned Skill Store Skill

  • We anticipate that work will begin soon on support for Matrix (mics)

Users who wish to follow our progress more closely are reminded that we are now on Matrix (chat) at #openvoiceos-general:matrix.org.

There’s video of j1nx testing the SJ-201 in all its glory, but the forum doesn’t accept mp4s, so the curious will have to wait for some of it to reach YouTube.

3 Likes

Here you go!

Amazing piece of tech, that sj201 daughterboard. :muscle:

2 Likes
1 Like

Very cool!

By the way, is the pause after “hey mycroft” to wait for a reaction required, or can I continue with the command without pause?

At the moment there is a bit of performance regression because of the included OSM skill and the stop skills for dev testing. That’s why it doesn’t really flow as it should and did.

In the end it should react a bit faster making it more flow natural.

Last little teaser for now. Really need to get back to work. :stuck_out_tongue_closed_eyes:

The team is working hard on the new audio system, that in the end will look like this;

2 Likes

Wow, very cool! What is the skill that plays the music? Thanks.

-Mike M

One of the cool things about UI handlers in Mycroft is that any media skill can bring up those controls.

@mike99mac That is the simple-youtube skill from @JarbasAl We decided to include it by default within the image (till we got told not to :wink: )

But yeah, the stream comes from youtube, but if I install deezer/spotify/bandcamp all together, it just plays the best match and you don’t know where it comes from until you swipe the screen to select something different.

Below another video of the first run wizard, to configure your OpenVoiceOS device the Mycroft way.

3 Likes

the common play UI is now all plugged in and going strong, repeat/shuffle buttons are not yet integrated however

I’m sure there will be a proper demo video shortly

2 Likes

The OpenVoiceOS team is pleased to announce Precise Lite prealpha, and a companion plugin! This is Precise on TFLite, presenting a much smaller footprint. It’s suitable for featherweight devices, and we also plan to ship it with OVOS itself.

A modest amount of setup is required; in addition to installing the wake word engine or plugin, you’ll need to configure Mycroft to use it, and point it at a model. See below for an example config snippet.

Like most Mycroft plugins, the Precise Lite plugin is believed to be compatible with any up-to-date Mycroft device. Mark II, Mycroft-git, Chatterbox, Plasma Bigscreen, you should all be able to use the plugin. I know there are a lot of us running 2GB Pi-based devices. It’s a real boon for us!

The primary author on this one was @jarbas. Special thanks to: Bartmoss St. Clair and eltocino.

To test, in your Mycroft environment,

mycroft-pip install git+https://github.com/OpenVoiceOS/ovos-ww-plugin-precise-lite

(Or if you run it systemwide / on a release of OpenVoiceOS)

pip install git+https://github.com/OpenVoiceOS/ovos-ww-plugin-precise-lite

and, assuming you’ve cloned the models repo into ~/precise-lite-models, see the following:

# mycroft.conf

 "hotwords": {
    "hey mycroft": {
      "module": "ovos-precise-lite",
      "model": "~/precise-lite-models/wakewords/en/hey_mycroft.tflite",
      "sensitivity": 0.5,
      "trigger_level": 3
 ...

Like mentioned above, the next pre-alpha image of OpenVoiceOS will ship/default configured with this precise-lite version of the wakeword listener. We are also currently looking into adding the “wake up” wake-word as secondairy precise-lite wake word to replace the pocketsphinx one, however we might reach out to you all to help out in gathering the required audio data to train an initial first version of it.

Other related news; The next pre-release will have a major bump of all required packages;

  • Based on buildroot 2021.02.05 LTS
  • Latest linux kernel 5.10.60-v8 LTS
  • Latest XMOS VocalFusion kernel driver for linux 5.10
  • KDE Framework (KF5) bumped to version 5.82 (kf5-kio / kf5-kirigami2)
  • About any other package bumped and or latest and greatest.

It will also have some not yet finished goodies such as a local configuration webbased dashboard;

The video GUI player has been worked on, to follow the same appearance as the above showed music GUI screens

and to top it up, the first run wizard - local selection procedure is now finished / implemented, providing a very easy and simple way to run Mycroft fully local and/ or without internet.

3 Likes

The included weather skill with animated background based on the weather report and time of day (dark/light mode).

And yes, typical Dutch weather…

2 Likes

Instant listen feature that @JarbasAl worked on and is default within OpenVoiceOS.

In short it means it basically immediately start listening for the intent when it hears the wake word which includes the sound that is actually being recorded as the wake word.

And as the SJ201 hat of the Mark2 is such a great little device, even when we do not disable the listening sound (although we made it a bit less intrusive and shorter in this video) it perfectly understands what you saying as all playing sound is removed by the DSP.

(The above weather skill demo already kind of gave it away, although with the “longer” listening sound)

3 Likes

Feature or Bug? Should he say second timer when mycroft opened the third? :smiley:

And to your Video: Awesome! I love to see your progress! Nice work.

Report that to the Mycroft A.I. chaps :wink:

It is a bit confusing but it is indeed the second timer for 5 minutes

actually a crude feature. the id is sorted by timer length (if unnamed).

Nice one - weather is looking slick and the instant listen is a great improvement.

In terms of the Timer behaviour - definitely interested in feedback. We’ve gone with more verbose responses to start with because it can then get confusing if you say something like “cancel the 5 minute timer”, or when the first 5 minute timer expires and it says “your 5 minute timer is up”. So the response on the “rice” here, is to help remind the user that there are actually two 5 minute timers.

Sorting by timer length is a newer thing too. Again keen on feedback of whether it’s helpful, confusing, or you don’t care as long as it’s consistent?

Installing ``ovos-precise-lite on a picroft went pretty smoothly and it was notably more responsive to the wakeword although I still had to wait until the beep to be able to issue commands.

The problem I now have is that mycroft.client.speech is now taking around 175% cpu compared to 40% for precise-engine and 100% for pocketsphinx, I did a full reboot to be sure and got the same cpu usage.

I did get t```he error:

ImportError: liblapack.so.3: cannot open shared object file: No such file or directory

on my picrofy install meant I needed to install libatlas3-base and then reinstall numpy:

sudo apt install libatlas3-base
~/mycroft-core/bin/mycroft-pip install --upgrade numpy

I also needed to add

"phonemes": "HH EY . M AY K R AO F T",

to the mycroft.conf (as per the precise and pocketsphinx sections).

After this and a $HOME/mycroft-core/start-mycroft.sh voice restart Mycroft worked. I think that it’s worth mentioing the package dependency in the install instructions.

1 Like

Thanks for that and sound very plausible as I seen other similar reports about the need of (re)installing numpy AFTER lapack.

@AIIX I think you are very interested in this as you had the same high CPU issue.

1 Like