Mycroft on Respeaker Core V2

Archive - Mycroft Project Modding
42

After many hours, and multiple attempts are restarting from scratch (redoing the image), it seems there might be some kind of contention problem on boot. When I boot and start mycroft, the speaker mic level is blank (no bar or movement). I do get log messages coming in from the background services, but Mycroft doesn’t respond to any voice. I quit and restart (which I guess just loads cli at this point) and the mic level is there and it moves, but the value is really low (10-100).

After quitting and doing some combination of killing and starting pulseaudio and stopping and starting respeakerd, I’m able to use parecord to make a recording that plays back … without doing so, I get either silence or occasional squeeks in the recording. When I’m able to do a recording, I can load mycroft fresh and it works, but mic levels are in the thousands. It seems like either something in the boot process seems to be hanging up or capturing the audio device? I need to figure out the proper sequence of starting/stoping the processes and see if that can narrow it down.

Edit (Because discourse won’t let me post 4 times in a row)…

Maybe this will help someone in the future since there seems nothing unique about my setup other than I’m doing this now (Sep '20) and not back in Jan '19. Plenty could have actually changed since then.

First, @Dominik, in your first post, between the update and upgrade, can you add an ‘apt-mark hold’ to prevent firefox from being updated.

sudo apt update
sudo apt-mark hold firefox-esr
sudo apt upgrade

Without this, after you do the upgrade, firefox errors out everytime you try to launch it.

Second, to fix the error message with setuptools, I added the export command I mentioned above to my .bashrc file. I don’t know enough about linux to know if there is a better way of doing it or if doing it this way won’t work with autostarting or not…

Which leads me to the third, I assume I cannot autostart mycroft because I have to manually stop and restart respeakerd before I can get mycroft to work after a boot. I’m not sure what’s causing the problem because I’ve rebuilt from image about 4 times today and got the same results each time… to get around it, I issue the following right after I ssh in:

sudo systemctl stop respeakerd
sudo systemctl start respeakerd

Then when I launch mycroft, and patiently wait for it to be fully loaded, it works.

Finally, I was really surprised to see how much CPU was used while mycroft was running… it sits at 75% cpu usage on average the entire time. I think it will work for my application, but its not a lot of headroom.

43

@madgrizzle thanks for posting your findings.

When Firefox is installed you most likely are using the “lxqt” Respeaker-image with full GUI/desktop. I recommend to use the “IoT” version which does not have the GUI and use less resources. I have accidently mentioned the “lxqt” instead of the “iot” image in my first post. Unfortunately the forum software does not let me edit it any longer. :frowning:

I remember rarely seeing the issue myself. Maybe it is possible to change the startup procedure and add a pause and a restart of respeaker-daemon - I will think about how to do it…

This is a) because GUI services are running (see above) and b) Mycrofts Precise wake word listener - which is a cpu-hogger…

44

Thanks… when I first starting out, I escaped (ctrl-c) out of doing the Alexa integration part of the respeaker install and thought that might have been the problem (like I didn’t complete the install or something). I’m now sure that’s not the issue so if I end up having to reload again, I’ll switch to the iot version. As for cpu load, it’s definitely the precise taking up the resources… can see it in htop. I’m hoping it won’t cause a problem with getting the ROS skill running.

45

That’s odd, maybe because of the age? As it turns out, all that needs to be done is change it from “lxqt” to “iot”. I restarted today from scratch with the iot version, even had it build mimic, and it worked without any deviations from your instructions… no issues - knock on wood… and it loads from boot with no problem. Maybe @gez-mycroft has the ability to mod it?

46

I’m not sure this is the appropriate place to comment on this, but I’ll do it anyway.

I managed to install ROS Melodic (bare bones package) on the respeaker by building it from source. I followed the directions from wiki.ros.org but had to make two tweaks:

rosdep install --from-paths src --ignore-src --rosdistro melodic -y

changed to:

rosdep install --from-paths src --ignore-src --rosdistro melodic --skip-keys=‘sbcl’ -y

I also had to add sketch-backports to my source list because rosdep dependcy install step failed to find python-pycryptodome. Not sure the proper way, but I edited /etc/apt/sources.list and added the following:

deb Index of /debian stretch-backports main

and then I installed python-pycryptodome using:

sudo apt-get -t stretch-backports install python-pycryptodome

I also created a 2GB swapfile on my SD card because the first time I tried to install ROS, it failed (I also unknowingly had mycroft still running as well). But I figure a swap file isn’t a bad thing if you have the space on the SD card and after doing so (and also shutting down mycroft) it succeeded to build.

So next step is to work on the skill to output ROS messages based upon the commands mycroft hears.

1 Like
47

This may cause the SD card to wear out fast. Maybe you can put the swapfile on the RC2 internal 4GB eMMC memory instead?

Why not create a new topic in section “modding” with title e.g. “Mycroft on robots” or “Mycroft and ROS”? :wink:

48

Any idea how to do that?

It was more of a question as to which forum to post it to since it really wasn’t related to Mycroft (getting ROS to run on the respeaker) but this thread had evolved to include robot discussion, I figured I’d mention it here. If I do manage to get the Mycroft-ROS install running, I’ll start a new thread on how I did it. It didn’t work last night as it seems to include its own modified mycroft instance inside a virtualenv. I think it was intended to ‘add’ mycroft node to an existing ROS install rather than just add a ROS skill to an existing mycroft install.

49

It took me some time to finish assembly, but here it is - Mark-IIR (early Mark-II prototype based on Respeaker Core V2):

IMG_7065x
IMG_7065x720×960 137 KB

Unfortunately the loudspeaker outputs some “electronic noise” as soon as it is powered on, requires probably some shielding/grounding…

4 Likes
50

I’m attempting this procedure now. With the following changes:

  • I’m running from a 32GB Samsung EVO uSD
  • I allocated the entire onboard eMMC as swap

Your procedure for installing respeakerd throws an error and doesn’t complete. However, the latest IoT image allows installation of respeakerd from apt. I assumed that’s OK.

The parecord test works. paplay does not, though pacat does, with a rate different from that used during parecord. Odd.

The Mycroft setup throws an error because one of the setup.py scripts requires Python 3.7, minimum, while RC2 (Debian stretch) only has 3.5. So I’m building 3.7.3 from source, the tests are … running, yes, they are still running… It is sloooow.

Q: If I build mimic during setup, can I go back and configure mycroft to use an in-house mimic3 server instead?

51

This will build Mimic1, not Mimic3. You may install Mimic3 with a bit of manual resolving dependencies but it will be slow on the RCv2.

52

Yes, thanks, I presumed it built Mimic 1, although, so far, it doesn’t build at all, even with Python 3.7.3 onboard.

Yes, the RC2 is quite slow, but that is why we have networks: to offload heavy tasks to hardware better suited for the work!

I asked because I have a quad core i5 based server that seems to produce m/l timely results with Mimic 3 in server mode (although there’s a problem downloading voices; mimic3-download hangs every time).

The RC2 (or a Pi3, etc) could offload TTS tasks to a remote system if there’s a way to configure it which won’t break updates to mycroft-core. I don’t see anything in the documentation that provides for this, but I haven’t read everything and I’m only two days into this effort.

You wouldn’t happened to have cut an RC2 sd image with reasonably current mycroft-core running by any chance? The procedure above no longer works, and Seeed appears to have abandoned RC2 completely even though one can still buy them on Amazon.

Edit: alternatively, if you could tell me the git hash for the mycroft-core revision you’re using, that might work.

53

The Mimic3 http-server is compatible with MaryTTS. Older mycroft-core releases, should support that.

Unfortunately not. I didn’t fire up my RC2 for quite a long time, it is still on mycroft-core from January '21.

54

Do you remember what version you used? 20.08 was Sep 2020. The next release was Feb 2021.

It’s sad; Python is touchier about versions than Java. One doesn’t need a crystal ball to know where that road leads. Like Java, it’s great for prototyping or quick-dirty, but breaks if a squirrel sneezes in Borneo.

Thanks Dominik!

55

To get this working in 2022, version 20.8.1 might be about as far as you can go. Here is what I did. Your mileage may vary. At least this version includes the STT and TTS plugin support, which I considered critical on the very low-power RC2 modules.

cd /home/respeaker && rm -fr .cache mycroft-core

That will clean up from previous failures. Now grab v20.8.1:

git clone -b release/v20.8.1 https://github.com/MycroftAI/mycroft-core.git

Python 3.5 is deprecated and won’t build this directly, so you have to edit mycroft-core/dev_setup.sh, line 436 (v20.8.1) and change from whence the required pip will be retrieved. Change:

curl https://bootstrap.pypa.io/get-pip.py | "${VIRTUALENV_ROOT}/bin/python" - 'pip==20.0.2'

to

PYVER=$(python3 --version | awk '{ print $2 }' | cut -d. -f1,2)
if [ "$PYVER" = "3.5" ] ; then
    curl https://bootstrap.pypa.io/pip/3.5/get-pip.py | "${VIRTUALENV_ROOT}/bin/python" - 'pip==20.0.2'
else
    curl https://bootstrap.pypa.io/get-pip.py | "${VIRTUALENV_ROOT}/bin/python" - 'pip==20.0.2'
fi

Then run the setup as detailed. I answered N to master, and N to updates. Y to everything else. You’ll get warned frequently during the build that pip can be upgraded! It’s a trick. Version 20.0.2 is what’s needed.

1 Like
56

Fired up my RCv2 - 20.8.0 it is. Mycroft asks me to update to 21.2.2, but I won’t do that (now)…

In the Respeaker-Wiki i have found a link to the “image builder”, maybe this can be used to create a RC2-image with a more current Debian OS?

57

Well, you can absolutely upgrade that to 20.8.1! :smiley:

20.8.1 runs just fine.

That build-image tool will be useful if I port Debian bullseye to the RC2, get respeakerd to work on newer bits etc., but I’m not very interested in doing that for an abandoned product like the RC2.

I might be ADHD. Cuz now that Mycroft works (well) with the RC2’s six-mic array, I’m playing with running Mimic3 on a Khadas Vim3 SBC and trying to push all the heavy lifting onto the Vim3’s dedicated NPU.

And that raises several dozen other questions e.g., can I get a sub-50-watt wad of electronics to do completely off-net “ai” assistant tasks? Can I fit it all into a 3d-printed Skyrim ice troll skull and move the lower jaw with a servo? Where can I get three 27mm glass eyes for that skull? What servo should I use to make the glass eyes track whoever is speaking?

Priorities, eh?

1 Like
58

OR… Upgrade that crusty Debian stretch image to Debian buster (Debian 10).

  • Install Seeed’s IoT image.

  • Set up BOTH ethernet and wifi (lots of people report network problems on the RC2 after the upgrade, presumably because the network device modules weren’t loaded during the upgrade?) I configured both and had zero problems.

    sudo apt update;sudo apt upgrade -y

  • Reboot

    sudo apt install respeakerd -y ; sudo apt autoremove -y

  • Upgrade to Debian Buster COMMENT OUT THE SEEED PPA IN /etc/apt/sources.list (last lines) before the apt update There is no buster repo for Seeed’s packages.

  • Follow Dominik’s instructions (except for the respeakerd build).

  • Now you have the latest Mycroft build and you’re future-proof for a few weeks.

respeaker@v2:~$ cat /etc/issue
Debian GNU/Linux 10

respeaker@v2:~$ ps -ef | grep respeakerd
respeak+  2182     1  0 15:32 ?        00:00:00 /bin/bash /usr/bin/respeakerd_safe
respeak+  2216  2182 75 15:32 ?        00:32:39 /usr/bin/respeakerd --source=alsa_input.platform-sound_0.seeed-8ch
respeak+ 16366 15706  0 16:15 pts/1    00:00:00 grep respeakerd
respeaker@v2:~$
1 Like