Mycroft Community Forum

Mycroft with Respeaker 2-Mic

So I now have Mycroft responding, well sort of. I can say “Hey Mycroft” then there is a long pause, the “I’m Listening” chime, then I can ask a question.
Mycroft never understands what I say, in fact I always get “List index out of range” errors in the CLI.

My Voice.log file:

2019-12-07 02:08:50.467 | INFO | 854 | main:handle_wakeword:59 | Wakeword Detected: hey mycroft
Playing WAVE ‘/home/pi/mycroft-core/mycroft/res/snd/start_listening.wav’ : Signed 16 bit Little Endian, Rate 48000 Hz, Stereo
2019-12-07 02:08:56.162 | INFO | 854 | main:handle_record_begin:37 | Begin Recording…
2019-12-07 02:09:04.447 | INFO | 854 | main:handle_record_end:43 | End Recording…
2019-12-07 02:09:06.297 | ERROR | 854 | mycroft.client.speech.listener:transcribe:239 | list index out of range
2019-12-07 02:09:06.302 | ERROR | 854 | mycroft.client.speech.listener:transcribe:240 | Speech Recognition could not understand auto

My Audio.log file:
This is the closest entry I could find

2019-12-07 02:08:39.138 | INFO | 851 | mycroft.audio.speech:mute_and_speak:120 | Speak: I don’t understand, but I’m learning new things everyday.
Playing WAVE ‘/tmp/mycroft/cache/tts/Mimic/bbe44ad9dc362342ef2162a3d7fe94a6.wav’ : Signed 16 bit Little Endian, Rate 44100 Hz, Mono

The long pause between “hey mycroft” and the chime might be related to the RPi being (to) busy. A Rpi with precise-engine running is always “busy”, but it should not be to busy.

I wonder what the output is of the linux command;

cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor

The “List index out of range” sounds like your mic volume is to low. Have a look with “alsamixer”, change to view the Seed voice card and see what the levels are of the microphone. If they are low, you coudl raiase them and play around a bit to find the better optimal setting for them.

@j1nx

Here is the output:

(.venv) pi@picroft:~ $ cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor
ondemand

I have no idea what that means.

As for the mic volume, it’ll pick up my wife and kids talking, print exactly what they say in the CLI and then let us know it didn’t understand us.

But if we ask a specific question that Mycroft should know we get the “List index out of range” message, even if I get them to ask the question :slight_smile:

The only thing I’ve installed on the Pi is Pycroft and the Respeaker 2-mic software.

The ondemand governor means it increases and decreases the CPU clockspeed on demand. This is fine for most cases, but I have seen that with precise-engine it keeps throtling up and down as I believe the threshold for it is at 95%.

I have it set to performance, but we will skip that for now. I asked to double check it wasn’t configured to “powersafe”. Because that meant it wasn’t running at full speed.

Anyhow, if the listener records you perfectly and also prints it, but doesn’t know what to do with it,. that sounds like you changed the language of mycroft? Did you?

Bit of monkey business going on I believe.

Well I wouldn’t say it records me perfectly. It pretty much ignores anything I say after “Hey Mycroft” if anyone else in my family says anything after the listening chime it will usually print what they say unless it’s a Mycroft command.

As for changing the language. I don’t think I changed anything but the voice and the wake word, which I quickly changed back when it didn’t work. Less things in the mix to sort out.

Sounds about right for anything I try.

So I just tried “Hey Mycroft” and then I didn’t say anything and I got the same error message as when I try to say something.

20:34:46.113 | ERROR | 854 | mycroft.client.speech.listener:transcribe:239 | list index out of range
`> ~~~~ | 854 | mycroft.client.speech.listener:transcribe:240 | Speech Recognition could not understand audio

I don’t know if that means anything or not.

ok edit 2, So I decided to watch the mic meter in the CLI, it jumps around with the house noises, spikes when I say 'Hey Mycroft" and then drops and freezes when it’s supposed to be listening to a spoken command. It’s almost like it’s getting muted. I assumed the muting mentioned in the CLI was for the output…

I wonder if a similar trick as here;

Might help you out. Especially the change in asound.conf to force everything to PulseAudio.

@j1nx I made the changes you recommended in that thread. I haven’t had the opportunity to test them out yet though. I’ll give it a try and let you know what happens.

edit:
Ok, I was just able to test it using TTS (as I am at work currently). I am able to trigger Mycroft with “Hey Mycroft” but then I get about 5 secs before it errors out.

13:45:31.193 | INFO | 856 | main:handle_wakeword:59 | Wakeword Detected: hey mycroft
75
13:45:31.754 | INFO | 850 | VolumeSkill | MUTING!
13:45:31.755 | INFO | 850 | AlarmSkill | on started…
13:45:31.792 | INFO | 850 | VolumeSkill | Volume before mute: 75
13:45:31.793 | INFO | 850 | VolumeSkill | 75
13:45:31.739 | INFO | 856 | main:handle_record_begin:37 | Begin Recording…
13:45:34.613 | INFO | 856 | main:handle_record_end:43 | End Recording…
0
13:45:34.693 | INFO | 850 | AlarmSkill | on ended…
13:45:36.308 | ERROR | 856 | mycroft.client.speech.listener:transcribe:239 | list index out of range
13:45:36.312 | ERROR | 856 | mycroft.client.speech.listener:transcribe:240 | Speech Recognition could not understand audio

So other than “Hey Mycroft” nothing is being said.

Hey I’ve been screwing around with installing mycroft on raspberry pi with a 2 mic hat and this is what I have come up with in the past month or so since I’ve started trying in my free time:

How to install Mycroft with a ReSpeaker 2-mic hat on a Raspberry Pi 3B+

  1. Install image from: https://drive.google.com/uc?id=1FdPUZhgoo_J8-MF4Z-xBl8nnf_6fJbrf&export=download

  2. Format card with SD Card Formatter

  3. Use balenaEtcher to flash image to sd Card

  4. Boot up raspberry pi with power to both hat and board

  5. Let it go through the first boot process

  6. It will ask you after the first boot process if you want to go through the guided install or to just go straight to the command line. Choose to go straight to the command line.

  7. From note (B), follow steps 2.1 and 2.2.

  8. Run pactl list sinks and pactl list sources and note what number corresponds with the Description: seeed-2mic-voicecard Analog Stereo for the sources and sinks command (should be 1 and 2 respectively).

  9. Run pactl set-default-sink $sink_card_number and pactl set-default-source $source_card_number where $sink_card_number and $source_card_number are replaced from the numbers you took note of in step 7 (again, should be 1 and 2 respectively).

  10. Run sudo reboot for good measure.

  11. Run sudo nano /etc/mycroft/mycroft.conf and edit the file to have the following code:

  12. “Play_wav_cmdline”: “paplay %1”,

  13. Confirm functionality by running ./mycroft-core/start-mycroft.sh audiotest -l from your home directory. When it says “DONE RECORDING. PLAYING BACK…”, let it try to run until it fails, do not try to control+c out of it. Sometimes it takes a few seconds to start the Daemon(?).

  14. Pair device

Notes:

  1. https://mycroft-ai.gitbook.io/docs/using-mycroft-ai/get-mycroft/picroft
  2. http://wiki.seeedstudio.com/ReSpeaker_2_Mics_Pi_HAT/
  3. https://stackoverflow.com/questions/42141004/qt-multimedia-cannot-find-lpulse

There are ~3 seconds of recording. It should pickup your voice command.

Perhaps, really forcing the souce to be set to the seed card might work. Have a look at @mrhavoc9999 his post. Maybe that works as well.

I believe sound out is working. Precise-engine as well, but as soon as it starts recording, it looks like it uses the wrong microphone or some sort of other monkey business.

I had the same problem at one point where it would pick up when I said “Hey Mycroft”, but after that, it wouldn’t record anything I said. I think the problem is because one of the skills mutes a sound card or something by default after it hears the wake word, which causes the microphone to not pickup anything. After it’s done trying to record the utterance and it spits out that list index error, it goes back to normal because it unmutes the sound card. I’m not sure how to fix it outright, but I know after restarting and following the directions I outlined in my post it didn’t do that anymore.
I hope that helps :slight_smile: I hope you get it working because I’ve been having trouble with it too. Right now it’s with getting Spotify to play from the same device, but I’m sure I’ll get it working eventually.

@mrhavoc9999 [quote=“mrhavoc9999, post:12, topic:7635”]
set-default-sink $sink_card_number and pactl set-default-source $source_card_number
[/quote]
I did this bit and I am happy to report it is now working!

Big thank you to @j1nx and @mrhavoc9999 for your help!

edit: Do you have a problem with the headphone volume being set to 0 when you reboot/restart your RPi?

@rlongfield Great news. Interested to know if you had multiple sources/sinks or just one of both. If the latter is the case, still strange that it didn’t make the only available the default automatically.

To comment on your volume. Try setting everything up with alsamixer. Close it and run the command;

sudo alsactl store

It should save your settings to /var/ which get’s restored at a reboot.

@j1nk

I gave that command a try. I’ll give my Pi a reboot and see if it sticks.

There was only 1 device listed for Sinks but 2 for Sources.

Source #0
State: IDLE
Name: alsa_output.platform-soc_sound.analog-stereo.monitor
Description: Monitor of seeed-2mic-voicecard Analog Stereo
Driver: module-alsa-card.c
Sample Specification: s32le 2ch 48000Hz
Channel Map: front-left,front-right
Owner Module: 6
Mute: no

and

Source #1
State: RUNNING
Name: alsa_input.platform-soc_sound.analog-stereo
Description: seeed-2mic-voicecard Analog Stereo
Driver: module-alsa-card.c
Sample Specification: s32le 2ch 48000Hz
Channel Map: front-left,front-right
Owner Module: 6
Mute: no

I set my Source to 1.

@j1nx

Seems that this command had no effect on this. The headphone volume still resets to zero on a reboot.

In regards to your edit, I don’t have that problem since I use the JST 2.0 plug, but I’ll take a look when I get home and attempt with the headphone jack :slight_smile:

Very strange that you have two sources, the first being a Monitor for the seed card…

I am a bit confused, which config files are now adjusted to what.

Could you walk us through which files you have editted with which content? This also for other users that might run into the same later on in time.

On a second note: Do you mind, me asking for a favour? I wonder if you have a spare SDcard at hand and would like to test an image for me? I am working on a hobby project that creates an out of the box experience for Mycroft. It is far from complete, but has respeaker support, although I am using the 4-mic hat, but the 2-hat should be supported as it uses the same drivers from seeed.

I just pushed a new alpha4 version online, and wondering if you could give it a go and report back. If so, this goes offtopic so perhaps better to move it to my thread here on the forums.

Anyhow; the download for it;
https://www.j1nx.nl/downloads/mycroftos-0-1-0-alpha4-rpi3/

Thanks in advanced, whatever your answer is :slight_smile:

@j1nx
Probably a good idea to note what changes I’ve made in this process. At some point I’m going to do something dumb, break something, and then need to start over…

I’ll check on the SD card. I believe I have one but I’m not 100% sure. If I do I’d be happy to test out your build.

As Requested here is the extent of the files I edited and what was edited.

/boot/config.txt
from

`# Enable audio (loads snd_bcm2835)
dtparam=audio=on
dtoverlay=i2s-mmap
dtparam=i2s=on

to

`# Enable audio (loads snd_bcm2835)
dtparam=audio=off
dtoverlay=i2s-mmap
dtparam=i2s=on
dtoverlay=seeed-2mic-voicecard

/etc/mycroft/mycroft.conf

“play_wav_cmdline”: “paplay %1”
“play_mp3_cmdline”: “mpg123 %1”

/etc/asound.conf

`# Use PulseAudio by default
pcm.!default {
type pulse
fallback “sysdefault”
hint {
show on
description “Default ALSA Output (currently PulseAudio Sound Server)”
}
}

ctl.!default {
type pulse
fallback “sysdefault”
}

/etc/pulse/daemon.conf

resample-method = ffmpeg
default-sample-format = s24le
default-sample-rate = 48000
alternate-sample-rate = 44100

/etc/pulse/default.pa

unload-module module-suspend-on-idle
unload-module module-role-cork
load-module module-role-ducking

pactl set-default-sink 0
pactl set-default-source 1

2 Likes

So I installed some skills tonight and tried to have Mycroft respond to “Hey Jarvis”
Then I installed the Cocktails, Timer, Home Assistant, and Tune-In skills.

Then I tried to use Mycroft.

“Hey Jarvis” didn’t work so I switched back to “Hey Mycroft”

I rebooted and then tried a “Hey Mycroft”. I got the listening chime, I asked a question and watched the CLI. Mycroft parsed the question properly but did not play any audio despite the CLI showing it was saying something.

Things were working fine before I made the changes. I removed the skills I added and rebooted but still no audio is being played. Also I noticed in the CLI that the Mic is silenced after a question is asked and doesn’t start working until I type something into the CLI.

So quick update, I double checked all my previous changes and everything was still there.
Then I remembered that I changed the voice to British Male. I switched it back to American Male, rebooted, and Mycroft started working properly…