Mycroft on Respeaker Core V2

Looks like I have to add another topic on my to-do list :laughing:

I can not rule out that that the respeaker-io skill causes the respeakerd service to malfunction as I simply mixed some code examples from the respeaker wiki pages for that…

Hi @Dominik,

no big deal. Your code was for the Respeaker Core V2, I have a Respeaker 6-mic array. I found this: ReSpeaker 4-Mic Array HAT Mycroft A.I. Skill (ReSpeaker 4-Mic Array HAT Mycroft A.I. Skill) and the developer says he built it on your work. So maybe her can point you in the direction of the problem? This works on the Rasp Pi 4.

Regards,

Stephen

1 Like

Hi @Dominik, I’ve received my V2 unit and followed these instructions. The microphone works brilliantly, very clear recordings.

I have no audio output of any kind though. Not from Mycroft, which uses pulse, or from Raspotify, which uses ALSA.

aplay -l shows a single output device, the seeed card. But a pacmd list-sinks shows only a null device. Or rather, it complains that no daemon is running. After starting pulse manually and using the pacmd list-sinks command again, it shows the null.

The seeed wiki seems to indicate that the seeed voicecard audio output ALSA device actually redirects the audio to pulse. This would explain why a service that doesn’t use pulse and rather relies on ALSA directly, would still fail to produce any sound.

What pulse sink devices do you see on yours? Did you have to take any extra steps to get audio output working?

I am using the iot image from seeed, which is cli only. I’m considering reflashing the lxqt image and starting again. I had hoped to build for rapid start times and low memory demand though.

Don’t know if this helps but here are some outputs

    $ aplay -l
    **** List of PLAYBACK Hardware Devices ****
    card 0: seeed8micvoicec [seeed-8mic-voicecard], device 1: 100b0000.i2s1-rk3228-hifi rk3228-hifi-1 []
      Subdevices: 1/1
      Subdevice #0: subdevice #0
$ pactl list sinks
Sink #0
	State: SUSPENDED
	Name: auto_null
	Description: Dummy Output
	Driver: module-null-sink.c
	Sample Specification: float32le 2ch 48000Hz
	Channel Map: front-left,front-right
	Owner Module: 13
	Mute: no
	Volume: front-left: 23593 /  36% / -26.62 dB,   front-right: 23593 /  36% / -26.62 dB
	        balance 0.00
	Base Volume: 65536 / 100% / 0.00 dB
	Monitor Source: auto_null.monitor
	Latency: 0 usec, configured 0 usec
	Flags: DECIBEL_VOLUME LATENCY 
	Properties:
		device.description = "Dummy Output"
		device.class = "abstract"
		device.icon_name = "audio-card"
	Formats:
		pcm
$ pactl list source
Source #0
	State: SUSPENDED
	Name: auto_null.monitor
	Description: Monitor of Dummy Output
	Driver: module-null-sink.c
	Sample Specification: float32le 2ch 48000Hz
	Channel Map: front-left,front-right
	Owner Module: 13
	Mute: no
	Volume: front-left: 65536 / 100% / 0.00 dB,   front-right: 65536 / 100% / 0.00 dB
	        balance 0.00
	Base Volume: 65536 / 100% / 0.00 dB
	Monitor of Sink: auto_null
	Latency: 0 usec, configured 0 usec
	Flags: DECIBEL_VOLUME LATENCY 
	Properties:
		device.description = "Monitor of Dummy Output"
		device.class = "monitor"
		device.icon_name = "audio-input-microphone"
	Formats:
		pcm

Source #1
	State: SUSPENDED
	Name: respeakerd_output
	Description: Unix FIFO source /tmp/music.input
	Driver: module-pipe-source.c
	Sample Specification: s16le 1ch 16000Hz
	Channel Map: mono
	Owner Module: 22
	Mute: no
	Volume: mono: 65536 / 100% / 0.00 dB
	        balance 0.00
	Base Volume: 65536 / 100% / 0.00 dB
	Monitor of Sink: n/a
	Latency: 0 usec, configured 0 usec
	Flags: DECIBEL_VOLUME LATENCY 
	Properties:
		device.string = "/tmp/music.input"
		device.description = "Unix FIFO source /tmp/music.input"
		device.icon_name = "audio-input-microphone"
	Formats:
		pcm

When I check status of respeakerd by sudo systemctl status respeakerd I get a strange result:

   Loaded: loaded (/lib/systemd/system/respeakerd.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Thu 2020-03-05 20:23:11 CET; 4min 53s ago
  Process: 1902 ExecStart=/usr/bin/respeakerd_safe (code=exited, status=1/FAILURE)
 Main PID: 1902 (code=exited, status=1/FAILURE)

Nevertheless Mycroft works (wakeword and intents).

I am using the “iot” image as well. The reference to the “lxqt” in my entrance posting is actually a typo (unfortunately I am not able to edit it anymore).

Thanks for taking the time to reply. Unfortunately that output is exactly what I was seeing, so no clues there.

I did get pulse audio running in system mode, which provided some better looking output from pactl list sinks, i could see what looked like a proper audio device. Alas, still no sound output!

I’ve actually reflashed the lxqt image, but haven’t had much time for the project since then.

On my way to my own Mark-IIr

3D printing of housing is finished. Now some glueing and soldering while start-stop-watching the assembly video

A big thank you to @derick-mycroft for providing the STL files!

4 Likes

Very cool !

Which screen did you choose?

This a Mark-IIR which does not have a built-in screen.

When everything works out fine i am planning to add a Waveshare 4.3" LCD (the same that is used for the Mark-II-RPI)

Ah, right! Didn’t do my homework :smiley:

Hope to get my hand on one of those boards myself sooner then later. Read good things about it. Looking forward to you build log :wink:

First of all, thank you for the hard work Dominik :slight_smile: ,

I have the problem that the respeaker core v2 doesn’t record, the mic and bluetooth speakers working fine, the mic Level goes up and seems to be fine and I checked alsamixer too.

I have to mention after the first Installation it worked fine, but on the next morning when I started the respeaker it didn’t work anymore properly and I didn’t change anything.

When I start the debug Mode, it doesn’t show any error I don’t know what to do or where to look :).

Can please someone help me i’m a beginner, thanks :).

Hi Dominik,

Also want to thank you for your great work.
I’ve had a respeaker core V2.0 for over a year and wanted to use it on a robotics command and control project I’ve been working on for a while.
Mycroft has come a long way and seems like a great active project, which is why I thought it would be great to try installing it on the respeaker core V2.0.

I worked through your installation instructions, and it seemed to be working, but then it stopped once it was authenticated to the main server.
I now start it and get nothing, not even errors.

When I run the pulse audio test, it seems to record a file, but can not play.

It seems like the problem is with pulse audio, but I’m not sure where to start.

Thanks in advance!!!

2 Likes

Hi again,

I just reinstalled the respeaker package and that fixed things.
Not sure what happened, but I’m all good now.

1 Like

Good to hear a fresh install got it working.

Interested to see the broader project you’re working on sometime too :slight_smile:

A quick look at the robot. Charlie.
Hopefully voice commands will control most of the behavior.

3 Likes

Damn that looks like a beast! Looking forward to seeing it evolve :smiley:

Hi guys.

I’m having totally a lot of fun with the mycroft project now. Its awesome and runs well on my respeaker core V2.0.

I am having a little issue here, though…

I want to install a custom wake-word for the robot, and fudging around with precise looked a little daunting, so I am trying snowboy.
starting with the default snowboy.umdl file, and just altering the config. here is what I did…

“listener”: {
“wake_word”: “snowboy”
},

“hotwords”: {
“snowboy”: {
“module”: “snowboy”,
“local_model_file”: “home/respeaker/.mycroft/snowboy.umdl”,
“sensitivity”: 0.1
}
},

It seems to be using snowboy now, but the problem is that it is triggered all the time.
Any little sound triggers the wake up, even when I reduce the sensitivity down to 0.1.

If anyone have played around with snowboy, this would be a great help, thanks in advance

OK, so I abandoned my quest to make snowboy work, and decided to use the CMUsphinx instead.

It mostly works, but it really was a necessity to get my robot to respond to its own name!

Here are the results:

1 Like

Wow, very cool!
I plan on building a Spotmicro, most likely in the OpenQuadruped design. Like Charlie robot with Mycroft integration of course.

2 Likes

Are you happening to use ROS for your robot? I’m building my own robot and am looking for a solution to integrate mycroft with ROS. There’s one package that I’ve found, http://wiki.ros.org/mycroft_ros, but it hasn’t had much love lately. I’m not sure there are others.

Yes, my Open-Quad robot will run with ROS - I am a ROS novice though. Thanks for the pointer to Mycroft-ROS.