Mycroft on Respeaker Core V2


#1

Mycroft on Respeaker Core V2

Respeaker Core V2 (short: RC2) is a Single Board Computer with a technical specification similar to a Raspberry Pi3 - but with some more goodies:

  • builtin microphone array (6 mics) which supports
    • beamforming
    • noise suppression
    • echo cancellation
  • led pixel ring
  • user button
  • 4GB eMMC memory (so you don’t need a SD-card)
  • JST2-connector to connect a small loudspeaker

Overall the RC2 is a nice package to run Mycroft on…

Preparation

RC2 runs with Debian 9 (Stretch). Seeedstudio (the vendor of RC2) provides prebuilt images where drivers for the onboard sound card are already installed and configured.

First you must decide if you want to install the OS to the onboard eMMC memory or run it directly from SD-card.
Go to fangcloud and navigate to respeakerv2 > debian > YYYYMMDD.
For the eMMC option download the
respeaker-debian-9-lxqt-flasher-********-4gb.img.xz.
In case you prefer to run the OS from the SD-card you should download the respeaker-debian-9-lxqt-sd-********-4gb.img.xz.

Burn the downloaded image to an SD-card (e.g. using Etcher), then insert the SD-card in the RC2 card slot and power it on. When you have choosen the eMMC option it will take approximately 10 minutes to copy the data from SD-card to the onboard memory.

Configure WiFi

After the OS install you must connect the RC2 to a PC via USB cable using the micro USB port labeled “OTG”. Then run a serial console program using 115200 baud rate, 8Bits, Parity None, Stop Bits 1, Flow Control None.
Login with user “respeaker” and password “respeaker”. Then run the setup with sudo nmtui and enter your WiFi credentials.

More detailed instructions for preparation and WiFi setup can be found here

Mycroft setup

OS update

First you should update the operating system:

sudo apt update
sudo apt upgrade

Respeaker daemon

Next step is installing respeakerd which provides the software stack for beamforming, noise reduction and echo cancellation:

curl https://raw.githubusercontent.com/respeaker/respeakerd/master/scripts/install_all.sh|bash

Next edit /etc/respeaker/respeakerd.conf and change the line starting with mode to mode = pulse

Then edit /etc/pulse/default.pa and add following lines

load-module module-pipe-source source_name="respeakerd_output" format=s16le rate=16000 channels=1
set-default-source respeakerd_output

Finally start the respeaker daemon with sudo systemctl start respeakerd

You may want to perform an audio test by:

parecord --channels=8 --rate=16000 --format=s16le hello2.wav
paplay hello2.wav

Mycroft install

Now your RC2 is ready to install Mycroft. This is done by a git-install:

cd ~/
git clone https://github.com/MycroftAI/mycroft-core.git
cd mycroft-core
bash dev_setup.sh

During the setup you will be asked some questions, here is a rundown of my recommended answers:

  • Y)es, run on the stable ‘master’ branch
  • Y)es, automatically check for updates
  • N - skip Mimic build
  • Y - Adding Mycroft commands to your PATH

NOTE: When you have chosen to install the OS on the eMMC you have only 4GB available! Whenever running dev_setup.sh you should consider adding the option -sm to skip the install of Mimic as this will several hundreds MB additional memory…

After setup is finished you are ready to run Mycroft for the first time:

cd ~/mycroft-core
./start-mycroft.sh debug

Respeaker-IO-Skill

I have written a Mycroft-skill that supports the LED pixel ring and the user button. You can install the skill by

mycroft-msm install https://github.com/domcross/respeaker-io-skill.git

Advanced options

Here are some advanced options, inspired by the blog of Mycroft community user @j1nx

Use RAM-disk for IPC

Mycroft uses a message bus over which the different Mycroft services communicate - the so called “inter process communication” (IPC). This will result in a lot of read and write operation to the file system which can “wear out” your SD-card. To avoid that you can redirect the IPC to a RAM-Disk:

sudo mkdir /ramdisk

The edit /etc/fstab and add following line to the end:

tmpfs /ramdisk tmpfs rw,nodev,nosuid,size=20M 0 0

sudo mkdir /etc/mycroft

Edit (new) file `/etc/mycroft/mycroft.conf:

{
  "ipc_path": "/ramdisk/mycroft/ipc/"
}

Automatically start Mycroft while booting

Until now you have to manually start Mycroft each time you have rebooted your RC2. Here is how to automatically run by using systemd:

Edit (new) file /etc/systemd/system/mycroft.service:

[Unit]
Description=Mycroft personal AI
After=respeakerd.service

[Service]
User=respeaker
WorkingDirectory=/home/respeaker/mycroft-core
ExecStart=/home/respeaker/mycroft-core/start-mycroft.sh all
ExecStop=/home/respeaker/mycroft-core/stop-mycroft.sh
Type=forking
Restart=always
RestartSec=3

[Install]
WantedBy=multi-user.target

User respeaker must be member of groups pulse and pulse-access, otherwise sound output will no work:

sudo usermod -a -G pulse respeaker
sudo usermod -a -G pulse-access respeaker

Then enable the “Mycroft service” by

sudo systemctl enable mycroft.service


Auto startup question from a noob
Mark II Architecture Change - A Note from the CEO
#2

That looks like a nice piece of hardware!

Thanks for sharing your learnings and writing that skill to make it even easier too


#3

Great write-up and thanks for the “ping”!

Forked your skill, because somewhere in the near future I want to do the same for their 4-Mic Pi HAT.

Anyhow, will put that device on my todo/to buy list. Looks very interesting. How are the;

  • beamforming
  • noise suppression
  • echo cancellation

Performing?


#4

I did only tests in a “laboratory stage” so far where the RC2 is sitting “naked” on my workdesk:

When running the respeaker-daemon in debug mode I can see that different directional information is shown when I change my position while speaking, so the beamforming/direction-of-arrival algorithm does work.

In another test I triggered a skill that plays MP3-stream and Mycroft still recognized the wake-word and triggered the desired skill/intent - the echo cancellation does work as well. My Mark-1 and my PiCroft with Google-AIY-hat in the same scenario (playing music stream in with similar loudness) stayed silent as they couldn’t hear me any more…

I am now looking for a case where the RC2 fits into, then I will perform real word tests…

By the way: the ReSpeaker Pro Case is for the Core V1 only - the RC2 has larger dimensions and does not fit into it (found it out the hard way…)


#5

This sounds like an important development, because I’m finding the picroft distribution to be essentially useless for music without echo cancellation. Have you found any drawbacks compared to the Raspberry pi?


#6

I did not perform actual benchmark testing but my impression is that overall performance of the RC2 is similar to the Raspberry Pi 3B. Drawbacks: as stated above there is no case for the RC2 and it does not have extensive GPIO support as the Raspberry has.

If you want echo-cancellation with Mycroft on a Raspberry Pi you might want to look at Respeaker 6mic circular array or the 4mic linear array - both have hardware loopback channel which enables the echo-cancellation (all other Respeaker mic hats for Raspberry do NOT have hw-loopback and therefore no echo-cancellation).


#7

Has anyone actually had success with a raspberry pi hearing them through the respeaker while music is playing? That would be reason to buy the respeaker, but frankly I don’t care much about the lights.


#8

For that you need echo-cancellation (EC). Looking at Respeaker devices only the 4-mic linear array and the 6-mic array can do EC (the 2-mic and 4-mic circular can’t do EC as they have no hardware loopback).

Using one of these, follow the instructions for installation of the seeedvoicecard-driver from the Wiki, then install respeakerd (you probably need to change some configuration settings depending on your mic-array model).


#9

@Dominik
Hey can you tell me what is about this 4-mic speaker:
http://wiki.seeedstudio.com/ReSpeaker_4_Mic_Array_for_Raspberry_Pi/

Does ist support echo-cancellation, beamforming or noise suppression?


#10

Hi @suisat,
The “circular” 4-mic array does have DOA (Direction of Arrival - beamforming), VAD (Voice Activity Detection) and KWS (Keyword Spotting or Keyword Search) but does NOT have echo-cancellation due to hardware limitations (no loopback channel).

The “linear” 4-mic array has other hardware including loopback channel which enables echo-cancellation, but lacks the LED pixel ring.

In case you want echo-cancellation and LED pixel ring you should look at the “circular” 6-mic array.