Mycroft on Respeaker Core V2

Mycroft Project Modding
#32

Hi guys.

I’m having totally a lot of fun with the mycroft project now. Its awesome and runs well on my respeaker core V2.0.

I am having a little issue here, though…

I want to install a custom wake-word for the robot, and fudging around with precise looked a little daunting, so I am trying snowboy.
starting with the default snowboy.umdl file, and just altering the config. here is what I did…

“listener”: {
“wake_word”: “snowboy”
},

“hotwords”: {
“snowboy”: {
“module”: “snowboy”,
“local_model_file”: “home/respeaker/.mycroft/snowboy.umdl”,
“sensitivity”: 0.1
}
},

It seems to be using snowboy now, but the problem is that it is triggered all the time.
Any little sound triggers the wake up, even when I reduce the sensitivity down to 0.1.

If anyone have played around with snowboy, this would be a great help, thanks in advance

#33

OK, so I abandoned my quest to make snowboy work, and decided to use the CMUsphinx instead.

It mostly works, but it really was a necessity to get my robot to respond to its own name!

Here are the results:

1 Like
#34

Wow, very cool!
I plan on building a Spotmicro, most likely in the OpenQuadruped design. Like Charlie robot with Mycroft integration of course.

2 Likes
#35

Are you happening to use ROS for your robot? I’m building my own robot and am looking for a solution to integrate mycroft with ROS. There’s one package that I’ve found, http://wiki.ros.org/mycroft_ros, but it hasn’t had much love lately. I’m not sure there are others.

#36

Yes, my Open-Quad robot will run with ROS - I am a ROS novice though. Thanks for the pointer to Mycroft-ROS.

#37

Same here… its a bit of a big learning curve, but slowly I’m getting there. I finally have it reliably navigating through the doorway and down the hall. It’s my third generation build since the beginning of the year and it runs much better. Here’s a video of it (I had to blur family photos or my wife wouldn’t let me post it publicly)… cables are messy because its temporary installation until I get a Rock Pi X to run everything on it.

1 Like
#38

I’m having an issue with the install. I followed the directions as precisely as I can (twice) and both times after running:

cd ~/mycroft-core
./start-mycroft.sh debug

I get the following error message:

respeaker@v2:~/mycroft-core$ ./start-mycroft.sh debug
Already up-to-date.
Starting all mycroft-core services
Initializing…
Starting background service bus
CAUTION: The Mycroft bus is an open websocket with no built-in security
measures. You are responsible for protecting the local port
8181 with a firewall as appropriate.
Starting background service skills
Starting background service audio
Starting background service voice
Starting background service enclosure
Starting cli
Traceback (most recent call last):
File “/usr/lib/python3.5/runpy.py”, line 193, in _run_module_as_main
“main”, mod_spec)
File “/usr/lib/python3.5/runpy.py”, line 85, in _run_code
exec(code, run_globals)
File “/home/respeaker/mycroft-core/mycroft/client/text/main.py”, line 21, in
from .text_client import (
File “/home/respeaker/mycroft-core/mycroft/client/text/text_client.py”, line 21, in
from mycroft.tts import TTS
File “/home/respeaker/mycroft-core/mycroft/tts/init.py”, line 20, in
from .tts import TTSFactory, TTS, TTSValidator, PlaybackThread
File “/home/respeaker/mycroft-core/mycroft/tts/tts.py”, line 467, in
class TTSFactory:
File “/home/respeaker/mycroft-core/mycroft/tts/tts.py”, line 473, in TTSFactory
from mycroft.tts.mimic_tts import Mimic
File “/home/respeaker/mycroft-core/mycroft/tts/mimic_tts.py”, line 40, in
import distutils.spawn
File “”, line 969, in _find_and_load
File “”, line 958, in _find_and_load_unlocked
File “”, line 666, in _load_unlocked
File “”, line 577, in module_from_spec
File “/home/respeaker/mycroft-core/.venv/lib/python3.5/site-packages/_distutils_hack/init.py”, line 82, in create_module
return importlib.import_module(’._distutils’, ‘setuptools’)
File “/usr/lib/python3.5/importlib/init.py”, line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File “”, line 981, in _gcd_import
File “”, line 931, in _sanity_check
SystemError: Parent module ‘setuptools’ not loaded, cannot perform relative import
respeaker@v2:~/mycroft-core$

Any suggestions?

#39

There was recently a major update to mycroft-core (20.2.5), maybe that did break something - I haven’t fired up my RC2 in the last few days, so I might have missed some incompatibility - will check that later…

#40

found this solution:

export SETUPTOOLS_USE_DISTUTILS=stdlib

from this page:

1 Like
#41

So I have some new issues. I’m having a hard time getting it to survive a reboot. I started from scratch and just loaded respeaker after getting connect to the network (no upgrade/update because for whatever reason, doing so breaks firefox), made the configuration changes, and did an audio test and it works. If I reboot, and do the audio test, it seems to record something, but its squeaks instead of clear voices. Any idea what could be going wrong?

#42

After many hours, and multiple attempts are restarting from scratch (redoing the image), it seems there might be some kind of contention problem on boot. When I boot and start mycroft, the speaker mic level is blank (no bar or movement). I do get log messages coming in from the background services, but Mycroft doesn’t respond to any voice. I quit and restart (which I guess just loads cli at this point) and the mic level is there and it moves, but the value is really low (10-100).

After quitting and doing some combination of killing and starting pulseaudio and stopping and starting respeakerd, I’m able to use parecord to make a recording that plays back … without doing so, I get either silence or occasional squeeks in the recording. When I’m able to do a recording, I can load mycroft fresh and it works, but mic levels are in the thousands. It seems like either something in the boot process seems to be hanging up or capturing the audio device? I need to figure out the proper sequence of starting/stoping the processes and see if that can narrow it down.

Edit (Because discourse won’t let me post 4 times in a row)…

Maybe this will help someone in the future since there seems nothing unique about my setup other than I’m doing this now (Sep '20) and not back in Jan '19. Plenty could have actually changed since then.

First, @Dominik, in your first post, between the update and upgrade, can you add an ‘apt-mark hold’ to prevent firefox from being updated.

sudo apt update
sudo apt-mark hold firefox-esr
sudo apt upgrade

Without this, after you do the upgrade, firefox errors out everytime you try to launch it.

Second, to fix the error message with setuptools, I added the export command I mentioned above to my .bashrc file. I don’t know enough about linux to know if there is a better way of doing it or if doing it this way won’t work with autostarting or not…

Which leads me to the third, I assume I cannot autostart mycroft because I have to manually stop and restart respeakerd before I can get mycroft to work after a boot. I’m not sure what’s causing the problem because I’ve rebuilt from image about 4 times today and got the same results each time… to get around it, I issue the following right after I ssh in:

sudo systemctl stop respeakerd
sudo systemctl start respeakerd

Then when I launch mycroft, and patiently wait for it to be fully loaded, it works.

Finally, I was really surprised to see how much CPU was used while mycroft was running… it sits at 75% cpu usage on average the entire time. I think it will work for my application, but its not a lot of headroom.

#43

@madgrizzle thanks for posting your findings.

When Firefox is installed you most likely are using the “lxqt” Respeaker-image with full GUI/desktop. I recommend to use the “IoT” version which does not have the GUI and use less resources. I have accidently mentioned the “lxqt” instead of the “iot” image in my first post. Unfortunately the forum software does not let me edit it any longer. :frowning:

I remember rarely seeing the issue myself. Maybe it is possible to change the startup procedure and add a pause and a restart of respeaker-daemon - I will think about how to do it…

This is a) because GUI services are running (see above) and b) Mycrofts Precise wake word listener - which is a cpu-hogger…

#44

Thanks… when I first starting out, I escaped (ctrl-c) out of doing the Alexa integration part of the respeaker install and thought that might have been the problem (like I didn’t complete the install or something). I’m now sure that’s not the issue so if I end up having to reload again, I’ll switch to the iot version. As for cpu load, it’s definitely the precise taking up the resources… can see it in htop. I’m hoping it won’t cause a problem with getting the ROS skill running.

#45

That’s odd, maybe because of the age? As it turns out, all that needs to be done is change it from “lxqt” to “iot”. I restarted today from scratch with the iot version, even had it build mimic, and it worked without any deviations from your instructions… no issues - knock on wood… and it loads from boot with no problem. Maybe @gez-mycroft has the ability to mod it?

#46

I’m not sure this is the appropriate place to comment on this, but I’ll do it anyway.

I managed to install ROS Melodic (bare bones package) on the respeaker by building it from source. I followed the directions from wiki.ros.org but had to make two tweaks:

rosdep install --from-paths src --ignore-src --rosdistro melodic -y

changed to:

rosdep install --from-paths src --ignore-src --rosdistro melodic --skip-keys=‘sbcl’ -y

I also had to add sketch-backports to my source list because rosdep dependcy install step failed to find python-pycryptodome. Not sure the proper way, but I edited /etc/apt/sources.list and added the following:

deb http://deb.debian.org/debian stretch-backports main

and then I installed python-pycryptodome using:

sudo apt-get -t stretch-backports install python-pycryptodome

I also created a 2GB swapfile on my SD card because the first time I tried to install ROS, it failed (I also unknowingly had mycroft still running as well). But I figure a swap file isn’t a bad thing if you have the space on the SD card and after doing so (and also shutting down mycroft) it succeeded to build.

So next step is to work on the skill to output ROS messages based upon the commands mycroft hears.

1 Like
#47

This may cause the SD card to wear out fast. Maybe you can put the swapfile on the RC2 internal 4GB eMMC memory instead?

Why not create a new topic in section “modding” with title e.g. “Mycroft on robots” or “Mycroft and ROS”? :wink:

#48

Any idea how to do that?

It was more of a question as to which forum to post it to since it really wasn’t related to Mycroft (getting ROS to run on the respeaker) but this thread had evolved to include robot discussion, I figured I’d mention it here. If I do manage to get the Mycroft-ROS install running, I’ll start a new thread on how I did it. It didn’t work last night as it seems to include its own modified mycroft instance inside a virtualenv. I think it was intended to ‘add’ mycroft node to an existing ROS install rather than just add a ROS skill to an existing mycroft install.

#49

It took me some time to finish assembly, but here it is - Mark-IIR (early Mark-II prototype based on Respeaker Core V2):

IMG_7065x
IMG_7065x720×960 137 KB

Unfortunately the loudspeaker outputs some “electronic noise” as soon as it is powered on, requires probably some shielding/grounding…

4 Likes
#50

I’m attempting this procedure now. With the following changes:

  • I’m running from a 32GB Samsung EVO uSD
  • I allocated the entire onboard eMMC as swap

Your procedure for installing respeakerd throws an error and doesn’t complete. However, the latest IoT image allows installation of respeakerd from apt. I assumed that’s OK.

The parecord test works. paplay does not, though pacat does, with a rate different from that used during parecord. Odd.

The Mycroft setup throws an error because one of the setup.py scripts requires Python 3.7, minimum, while RC2 (Debian stretch) only has 3.5. So I’m building 3.7.3 from source, the tests are … running, yes, they are still running… It is sloooow.

Q: If I build mimic during setup, can I go back and configure mycroft to use an in-house mimic3 server instead?

#51

This will build Mimic1, not Mimic3. You may install Mimic3 with a bit of manual resolving dependencies but it will be slow on the RCv2.