Mycroft Community Forum

Mycroft with Respeaker 2-Mic

I just setup a R Pi 3B+ with a Respeaker 2-mic PiHat and installed Picroft.

I’ve installed the ReSpeaker drivers following their wiki and I can record and play back sounds with arecord -Dhw:1 -f cd test.wav
and aplay -Dhw:1 test.wav

I tried:

“play_wav_cmdline”: “aplay %1”,
“play_mp3_cmdline”: “mpg123 %1”,

and

“play_wav_cmdline”: “aplay -Dhw:1 %1”,
“play_mp3_cmdline”: “mpg123 -a hw:1 %1”,

in my mycroft.conf file

But Mycroft will not respond when I saw “Hey Mycroft” and it will not play audio when I use “Mycroft-say-to” in the CLI.

I can post audio and voice logs if needed.

I haven’t used these microphones, but when I added an I2S microphone a few weeks ago, I had a hard time getting it to work with Mycroft. My solution was to install Picroft and choose the Google Voicehat option. The installer fails to finish cleanly, it constantly wanted to reboot and ask again what sound out to use. After a few reboots, I simply selected to default (1) sound out. This allowed the routine to finish. Now the I2S microphone and speaker work. Neither of which is the Google Voicehat.

So it looks like Mycroft uses Pulse audio but Pulse uses Alsa to manage the audio devices?

When I did a pactl info I see:

Default Sink: alsa_output.platform-soc_audio.analog-stereo
Default Source: alsa_output.platform-soc_audio.analog-stereo.monitor

However when I go into alsamixer the default audio device is PulseAudio. Just going to go out on a limb here and say that perhaps this might be my issue? Pulse is pointing to Alsa for devices and Alsa is pointing to Pulse as the default device…

So I may have sorted this out but I won’t know for sure until I can test it later today.

I dug up some old notes from when I had the ReSpeaker-2mic on my Pi Zero and found that I had edited the /boot/config.txt file.

At the bottom of the file I changed

`# Enable audio (loads snd_bcm2835)
dtparam=audio=on
dtoverlay=i2s-mmap
dtparam=i2s=on

to

`# Enable audio (loads snd_bcm2835)
#dtparam=audio=on
dtparam=audio=off
dtoverlay=i2s-mmap
dtparam=i2s=on
dtoverlay=seeed-2mic-voicecard

So basically I disabled all the on board audio and left only the Respeaker card. After rebooting and getting back into the Mycroft CLI I noticed the Mic level was jumping around and changing which was something that wasn’t happening before.

1 Like

So I now have Mycroft responding, well sort of. I can say “Hey Mycroft” then there is a long pause, the “I’m Listening” chime, then I can ask a question.
Mycroft never understands what I say, in fact I always get “List index out of range” errors in the CLI.

My Voice.log file:

2019-12-07 02:08:50.467 | INFO | 854 | main:handle_wakeword:59 | Wakeword Detected: hey mycroft
Playing WAVE ‘/home/pi/mycroft-core/mycroft/res/snd/start_listening.wav’ : Signed 16 bit Little Endian, Rate 48000 Hz, Stereo
2019-12-07 02:08:56.162 | INFO | 854 | main:handle_record_begin:37 | Begin Recording…
2019-12-07 02:09:04.447 | INFO | 854 | main:handle_record_end:43 | End Recording…
2019-12-07 02:09:06.297 | ERROR | 854 | mycroft.client.speech.listener:transcribe:239 | list index out of range
2019-12-07 02:09:06.302 | ERROR | 854 | mycroft.client.speech.listener:transcribe:240 | Speech Recognition could not understand auto

My Audio.log file:
This is the closest entry I could find

2019-12-07 02:08:39.138 | INFO | 851 | mycroft.audio.speech:mute_and_speak:120 | Speak: I don’t understand, but I’m learning new things everyday.
Playing WAVE ‘/tmp/mycroft/cache/tts/Mimic/bbe44ad9dc362342ef2162a3d7fe94a6.wav’ : Signed 16 bit Little Endian, Rate 44100 Hz, Mono

The long pause between “hey mycroft” and the chime might be related to the RPi being (to) busy. A Rpi with precise-engine running is always “busy”, but it should not be to busy.

I wonder what the output is of the linux command;

cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor

The “List index out of range” sounds like your mic volume is to low. Have a look with “alsamixer”, change to view the Seed voice card and see what the levels are of the microphone. If they are low, you coudl raiase them and play around a bit to find the better optimal setting for them.

@j1nx

Here is the output:

(.venv) pi@picroft:~ $ cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor
ondemand

I have no idea what that means.

As for the mic volume, it’ll pick up my wife and kids talking, print exactly what they say in the CLI and then let us know it didn’t understand us.

But if we ask a specific question that Mycroft should know we get the “List index out of range” message, even if I get them to ask the question :slight_smile:

The only thing I’ve installed on the Pi is Pycroft and the Respeaker 2-mic software.

The ondemand governor means it increases and decreases the CPU clockspeed on demand. This is fine for most cases, but I have seen that with precise-engine it keeps throtling up and down as I believe the threshold for it is at 95%.

I have it set to performance, but we will skip that for now. I asked to double check it wasn’t configured to “powersafe”. Because that meant it wasn’t running at full speed.

Anyhow, if the listener records you perfectly and also prints it, but doesn’t know what to do with it,. that sounds like you changed the language of mycroft? Did you?

Bit of monkey business going on I believe.

Well I wouldn’t say it records me perfectly. It pretty much ignores anything I say after “Hey Mycroft” if anyone else in my family says anything after the listening chime it will usually print what they say unless it’s a Mycroft command.

As for changing the language. I don’t think I changed anything but the voice and the wake word, which I quickly changed back when it didn’t work. Less things in the mix to sort out.

Sounds about right for anything I try.

So I just tried “Hey Mycroft” and then I didn’t say anything and I got the same error message as when I try to say something.

20:34:46.113 | ERROR | 854 | mycroft.client.speech.listener:transcribe:239 | list index out of range
`> ~~~~ | 854 | mycroft.client.speech.listener:transcribe:240 | Speech Recognition could not understand audio

I don’t know if that means anything or not.

ok edit 2, So I decided to watch the mic meter in the CLI, it jumps around with the house noises, spikes when I say 'Hey Mycroft" and then drops and freezes when it’s supposed to be listening to a spoken command. It’s almost like it’s getting muted. I assumed the muting mentioned in the CLI was for the output…

I wonder if a similar trick as here;

Might help you out. Especially the change in asound.conf to force everything to PulseAudio.

@j1nx I made the changes you recommended in that thread. I haven’t had the opportunity to test them out yet though. I’ll give it a try and let you know what happens.

edit:
Ok, I was just able to test it using TTS (as I am at work currently). I am able to trigger Mycroft with “Hey Mycroft” but then I get about 5 secs before it errors out.

13:45:31.193 | INFO | 856 | main:handle_wakeword:59 | Wakeword Detected: hey mycroft
75
13:45:31.754 | INFO | 850 | VolumeSkill | MUTING!
13:45:31.755 | INFO | 850 | AlarmSkill | on started…
13:45:31.792 | INFO | 850 | VolumeSkill | Volume before mute: 75
13:45:31.793 | INFO | 850 | VolumeSkill | 75
13:45:31.739 | INFO | 856 | main:handle_record_begin:37 | Begin Recording…
13:45:34.613 | INFO | 856 | main:handle_record_end:43 | End Recording…
0
13:45:34.693 | INFO | 850 | AlarmSkill | on ended…
13:45:36.308 | ERROR | 856 | mycroft.client.speech.listener:transcribe:239 | list index out of range
13:45:36.312 | ERROR | 856 | mycroft.client.speech.listener:transcribe:240 | Speech Recognition could not understand audio

So other than “Hey Mycroft” nothing is being said.

Hey I’ve been screwing around with installing mycroft on raspberry pi with a 2 mic hat and this is what I have come up with in the past month or so since I’ve started trying in my free time:

How to install Mycroft with a ReSpeaker 2-mic hat on a Raspberry Pi 3B+

  1. Install image from: https://drive.google.com/uc?id=1FdPUZhgoo_J8-MF4Z-xBl8nnf_6fJbrf&export=download

  2. Format card with SD Card Formatter

  3. Use balenaEtcher to flash image to sd Card

  4. Boot up raspberry pi with power to both hat and board

  5. Let it go through the first boot process

  6. It will ask you after the first boot process if you want to go through the guided install or to just go straight to the command line. Choose to go straight to the command line.

  7. From note (B), follow steps 2.1 and 2.2.

  8. Run pactl list sinks and pactl list sources and note what number corresponds with the Description: seeed-2mic-voicecard Analog Stereo for the sources and sinks command (should be 1 and 2 respectively).

  9. Run pactl set-default-sink $sink_card_number and pactl set-default-source $source_card_number where $sink_card_number and $source_card_number are replaced from the numbers you took note of in step 7 (again, should be 1 and 2 respectively).

  10. Run sudo reboot for good measure.

  11. Run sudo nano /etc/mycroft/mycroft.conf and edit the file to have the following code:

  12. “Play_wav_cmdline”: “paplay %1”,

  13. Confirm functionality by running ./mycroft-core/start-mycroft.sh audiotest -l from your home directory. When it says “DONE RECORDING. PLAYING BACK…”, let it try to run until it fails, do not try to control+c out of it. Sometimes it takes a few seconds to start the Daemon(?).

  14. Pair device

Notes:

  1. https://mycroft-ai.gitbook.io/docs/using-mycroft-ai/get-mycroft/picroft
  2. http://wiki.seeedstudio.com/ReSpeaker_2_Mics_Pi_HAT/
  3. https://stackoverflow.com/questions/42141004/qt-multimedia-cannot-find-lpulse

There are ~3 seconds of recording. It should pickup your voice command.

Perhaps, really forcing the souce to be set to the seed card might work. Have a look at @mrhavoc9999 his post. Maybe that works as well.

I believe sound out is working. Precise-engine as well, but as soon as it starts recording, it looks like it uses the wrong microphone or some sort of other monkey business.

I had the same problem at one point where it would pick up when I said “Hey Mycroft”, but after that, it wouldn’t record anything I said. I think the problem is because one of the skills mutes a sound card or something by default after it hears the wake word, which causes the microphone to not pickup anything. After it’s done trying to record the utterance and it spits out that list index error, it goes back to normal because it unmutes the sound card. I’m not sure how to fix it outright, but I know after restarting and following the directions I outlined in my post it didn’t do that anymore.
I hope that helps :slight_smile: I hope you get it working because I’ve been having trouble with it too. Right now it’s with getting Spotify to play from the same device, but I’m sure I’ll get it working eventually.

@mrhavoc9999 [quote=“mrhavoc9999, post:12, topic:7635”]
set-default-sink $sink_card_number and pactl set-default-source $source_card_number
[/quote]
I did this bit and I am happy to report it is now working!

Big thank you to @j1nx and @mrhavoc9999 for your help!

edit: Do you have a problem with the headphone volume being set to 0 when you reboot/restart your RPi?

@rlongfield Great news. Interested to know if you had multiple sources/sinks or just one of both. If the latter is the case, still strange that it didn’t make the only available the default automatically.

To comment on your volume. Try setting everything up with alsamixer. Close it and run the command;

sudo alsactl store

It should save your settings to /var/ which get’s restored at a reboot.

@j1nk

I gave that command a try. I’ll give my Pi a reboot and see if it sticks.

There was only 1 device listed for Sinks but 2 for Sources.

Source #0
State: IDLE
Name: alsa_output.platform-soc_sound.analog-stereo.monitor
Description: Monitor of seeed-2mic-voicecard Analog Stereo
Driver: module-alsa-card.c
Sample Specification: s32le 2ch 48000Hz
Channel Map: front-left,front-right
Owner Module: 6
Mute: no

and

Source #1
State: RUNNING
Name: alsa_input.platform-soc_sound.analog-stereo
Description: seeed-2mic-voicecard Analog Stereo
Driver: module-alsa-card.c
Sample Specification: s32le 2ch 48000Hz
Channel Map: front-left,front-right
Owner Module: 6
Mute: no

I set my Source to 1.

@j1nx

Seems that this command had no effect on this. The headphone volume still resets to zero on a reboot.

In regards to your edit, I don’t have that problem since I use the JST 2.0 plug, but I’ll take a look when I get home and attempt with the headphone jack :slight_smile:

Very strange that you have two sources, the first being a Monitor for the seed card…

I am a bit confused, which config files are now adjusted to what.

Could you walk us through which files you have editted with which content? This also for other users that might run into the same later on in time.

On a second note: Do you mind, me asking for a favour? I wonder if you have a spare SDcard at hand and would like to test an image for me? I am working on a hobby project that creates an out of the box experience for Mycroft. It is far from complete, but has respeaker support, although I am using the 4-mic hat, but the 2-hat should be supported as it uses the same drivers from seeed.

I just pushed a new alpha4 version online, and wondering if you could give it a go and report back. If so, this goes offtopic so perhaps better to move it to my thread here on the forums.

Anyhow; the download for it;
https://www.j1nx.nl/downloads/mycroftos-0-1-0-alpha4-rpi3/

Thanks in advanced, whatever your answer is :slight_smile: