ReSpeaker 4-Mic Array HAT Mycroft A.I. Skill

j1nx · January 31, 2019, 11:41am

ReSpeaker 4-Mic Array HAT Mycroft A.I. Skill.
This skill is to enable and control the Seeed - ReSpeaker 4-Mic Array for Raspberry Pi. ReSpeaker 4-Mic Array for Raspberry Pi is a quad-microphone expansion board for Raspberry Pi designed for AI and voice applications and provides a super cool LED ring, which contains 12 APA102 programable LEDs.

DISCLAIMER: At the moment this is NOT tested by me (as of yet), but as multiple people asked for a ReSpeaker 4-Mic Array HAT skill in the past, already created this topic to open the dialogue and possible feedback / testing.

CREDITS: ALL credits go to @Dominik as he created the almost exact same skill for the ReSpeaker Core V2. I only changed some small code to flip the library used (pixel-ring) to use the 4-Mic Array instead.

How to install respeaker-4mic-hat-skill

Install respeaker-4mic-hat-skill by …

SSH into your Mycroft, then

mycroft-msm install GitHub - j1nx/respeaker-4mic-hat-skill: Mycroft A.I. unofficial - ReSpeaker 4Mic HAT Skill.

In case the requirements are not installed automatically you must perform the following;

mycroft-pip install pixel-ring gpiozero

Turn on SPI on the Raspberry Pi

To make sure the RPi can control the LEDs, you need to enable SPI;

For Raspbian based systems (Picroft)

sudo raspi-config
Go to “Interfacing Options”
Go to “SPI”
Enable SPI
Exit raspi-config
? reboot ?

Manually

Edit the file config.txt on the boot partition of your Raspberry Pi
Make sure the following content is there;

dtparam=spi=on

Ideas & Additions for later

Finish / Start enabling and disabling the pixel ring

“Hey Mycroft”, “Enable pixel ring”
“Hey Mycroft”, “Disable pixel ring”

Change LED mode

“Hey Mycroft”, “Set pixel ring to Alexa”
“Hey Mycroft”, “Set pixel ring to Google”

Add Mycroft LED mode

“Hey Mycroft”, “Set pixel ring to Mycroft”

DOA (Direction Of Arrival) of keyword

Where feedback should be directed a

Feedback on the skill should be provided through:

replying to this thread
Issue on Github
Mycroft Chat in channel ~skills

j1nx · January 31, 2019, 7:14pm

WARNING !!! Just installed on Raspbian and 18.8.12 but it hangs the RPI !!!

DO NOT INSTALL (yet)

j1nx · January 31, 2019, 7:16pm

Now only need to figure out how to debug this?

a) I don’t know Python yet
b) Loading the skill brings the RPi on it’s knees

Running out of time for now. Must be somewhere in spidev <-> gpiozero as only hardware thingies can bring computers on its knees I think.

Dominik · January 31, 2019, 8:49pm

I would try to debug it like following:

LOG.debug("initialising")
power = LED(5)
LOG.debug("power set to LED(5)")
power.on()
LOG.debug("power on")

Maybe user mycroft is missing permissions:
sudo usermod -g gpio mycroft

j1nx · February 1, 2019, 9:40am

Yeah, will start playing around with the ezample code first;

github.com

respeaker/pixel_ring/blob/master/examples/respeaker_4mic_array.py

"""
Control pixel ring on ReSpeaker 4 Mic Array

pip install pixel_ring gpiozero
"""

import time

from pixel_ring import pixel_ring
from gpiozero import LED

power = LED(5)
power.on()

pixel_ring.set_brightness(10)

if __name__ == '__main__':
    while True:

        try:

This file has been truncated. show original

Highly doubt it is the Mycroft part, or your code in that regard.

When I first run the Skill on MycroftOS I was missing lsusb libs, which brought up errors in the section that checks which HAT/Hardware is present via pyusb. Now that error is gone and the PI was bought on it’s knees.

Because of that searched that old Raspbian based SDcard, updated everything and tested there as well. Same thing, so suspect it hangs on either that checking hardware thing with lsusb/pyusb or one step further in actually controllinig the SPI via spidev/gpiozero.

Oh well, a good first project to start learning some Python. Did my fair share of coding with other languages in the past, so can easily read the code. Will see how far I can get…

suisat · April 7, 2019, 4:52pm

I am not able to bring it to run on my rpi-3b+ with picroft stable.

But I am unsure if it is because of my setup or your skill.
Can I send you any informations/logs?

EDIT:
With the commands here I think I bring it to run. I am running that skill on a RPi3B+ with Picroft stable.

But… The lights only show up once on the startup of mycroft.
Here is a video of that: Watch me

gez-mycroft · April 9, 2019, 12:29am

Hi suisat,

Are there errors showing up in your voice or audio logs? This is located at /var/log/mycroft/voice.log.

If the CLI is accepting commands you can tell it to “create a support ticket” and it will upload all your logs to termbin and email you a link. This can be much easier than fetching them manually.

In your other post you said “Mycroft is hearing me but only when I do some inputs” do you mean, only after typing things into the CLI?

suisat · April 12, 2019, 6:14am

Nothin in the skill-logs about that.
After isntall the j1nx skill for respeaker 4mic hat I had some issues about some pip-requirements but after install that, there gone.

But yesterday if foudn out how to run the LED’s.

Install the git skill from j1nx (msm install GIT-URL or within home.mycroft.ai)
Start mycroft
perhaps you need to say once “enable pixel ring” or “disable pixel ring”
Restart mycroft and while booting up, check the pixel-ring on your respeaker. It should pop up for a second.
Press CTRL+ALT+F2 and open another raspi-terminal
Type in python and then:

from pixel_ring import pixel_ring
from gpiozero import LED
power = LED(5)
power.on()
pixel_ring.set_brightness(10)
pixel_ring.wakeup()

As long as your python shell is running, it should work like a charm

Anything does not work in j1nx skill. Perhaps it has something todo with “self.en.write(1)” in the “def shutdown” function but I am npt sure. I try to replace the line with:

def shutdown(self):
     LOG.debug("shutdown")
     power = LED(5)
     power.off()

But without success.

@j1nx or @Dominik Did you have any idea why it does not run on my mycroft-stable instance on my Raspberry-PI-3b+ . It seems that mycroft has any problems with running or keeping the gpio power.

I would be very happy about any hints.

@gez-mycroft
How can I enable full-detailed-logging of the skill? I read this Log Files - Mycroft AI and yesterday I did add thisline to the init.py (respeaker-4mic-hat-skill/__init__.py at master · j1nx/respeaker-4mic-hat-skill · GitHub):

LOGGER = getLogger(__name__)

But that just enable the output like “initialising” or “speak”.

Here is a new video how beatiful it could be:

https://mega.nz/embed#!60xBzYRa!3hMBEOKGaC9JLziIXMPjOCZGCxyOtz1m78PzuZhMRc8

gez-mycroft · April 12, 2019, 7:07am

Hey, looking at the code j1nx has imported the logging utility as “LOG” so if you see on line 28 there is:
LOG.debug("initialising")

This is what’s causing the output. If you want to log something else, you need to add in some extra LOG.debug()

Slightly unrelated but fun side fact: the logger is now included in the base MycroftSkill class so we no longer need to import it at all. You can call this within the Skill using self.log.debug()

suisat · April 12, 2019, 7:11am

I will add this to the init.py and write here something if I can see more in the skill.log.

THX

j1nx · April 17, 2019, 8:29am

@suisat and @gez-mycroft First of all, my apologies for my late response / abscence. Momentarily it is rather busy with my daytime job, so haven’t much played with mycroft, nor being active online.

First things first;

This plugin was quickly, let say “blind” coded without any Python knowledge what so ever. I just saw @Dominik his Skill for the ReSpeaker V2 box and looking at his code I discovered that the Python libraries it used, already had everything in place for the “other” MIC arrays. That code also was properly commented, so it looked like a quick revamp to get it working on the 4-MIC array. Apparently is isn’t

I still don’t have much / any time to investigate or fix / adjust, but here are my thoughts on the above testing and information.

With the commands here I think I bring it to run. I am running that skill on a RPi3B+ with Picroft stable.

Yes, this is what I already expected some what. These commands can be added to a “requirements.sh” next to the “requirements.txt” file. Similar as this one from @Dominik ;

github.com

domcross/respeaker-io-skill/blob/master/requirements.sh

#!/bin/bash

# mraa libs are now added in the local skill folder
#sudo apt-get -y install python3-mraa=1.9.0 libmraa1=1.9.0
sudo apt-get -y install python3-mraa libmraa1
sudo usermod -a -G input respeaker
exit 0

However the first command; “mycroft-pip install RPi.GPIO” should be removed and added to the “requirements.txt” file instead;

pixel-ring
gpiozero
RPi.GPIO

I don’t know if both; “gpiozero” and “RPi.GPIO” are needed, or that I made a mistake and “gpiozero” needs to be changed to “RPi.GPIO”

Then on to the strangest thing of all, you needed to type this into another terminal / python CLI;

from pixel_ring import pixel_ring
from gpiozero import LED
power = LED(5)
power.on()
pixel_ring.set_brightness(10)
pixel_ring.wakeup()

All this code is within the Skill init.py file as shown here;

github.com

j1nx/respeaker-4mic-hat-skill/blob/master/init.py

# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#    http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# All credits go to domcross (Github https://github.com/domcross)

import time

from mycroft.messagebus.message import Message
from mycroft.skills.core import MycroftSkill, intent_handler
from adapt.intent import IntentBuilder
from mycroft.util.log import LOG

This file has been truncated. show original

So it looks like the code within the Skill is not properly loaded or initialised. I have no clue why as I don’t know Python yet.

What I can think of is that, perhaps the Skill initialisation fails because I deleted the " settingsmeta.json" ?!?! The content of it looked like only placeholders, so deleted it. It does not do much yet, but considering that you apparantly also can enable/disable the pixelring by commands, hence toggling it is needed to get stuff working, perhaps this file need to be there as well ?

Perhaps @forslund can have a look, if that could be the cause of it. Or perhaps you could test it by creating the file with the following content, similar as @Dominik ;

github.com

domcross/respeaker-io-skill/blob/master/settingsmeta.json

{
    "skillMetadata": {
        "sections": [
            {
                "name": "Options << Name of section",
                "fields": [
                    {
                        "name": "internal_python_variable_name",
                        "type": "text",
                        "label": "Setting Friendly Display Name",
                        "value": "",
                        "placeholder": "demo prompt in the input box"
                    }
                ]
            },
            {
                "name": "Login << Name of another section",
                "fields": [
                    {
                        "type": "label",

This file has been truncated. show original

So far, my ideas and thoughts. Hope to start cracking with Mycroft again any time soon now…

j1nx · April 17, 2019, 3:00pm

Perhaps, this;

Is/ was also related with this;

https://github.com/j1nx/respeaker-4mic-hat-skill/blob/master/init.py#L19

forslund · April 17, 2019, 6:33pm

Hmm, that shouldn’t make a difference the skill doesn’t seem to wait for the “message” message that the PR re-enabled…

The missing settingsmeta.json shouldn’t™ be an issue either.

From a quick glance I’m afraid I don’t see anything obviously wrong

Dominik · April 17, 2019, 7:18pm

No, RPi.GPIO and gpiozero are two different libraries for the same thing (accessing GPIO), where the gpiozero is the more conveniant one to program with. You are using gpiozero, so you can safely remove the RPi.GPIO requirement.

As already stated before: maybe user ‘mycroft’ is lacking permissions to access GPIO?
try sudo usermod -g gpio mycroft

suisat · April 23, 2019, 12:47pm

There is no user mycroft in the pycroft stable release.

forslund · April 25, 2019, 4:49pm

The newer picroft images doesn’t use the separate mycroft user. Try doing the suggested change for the pi user: sudo usermod -g gpio pi

suisat · April 25, 2019, 8:04pm

Yeah, I did that already. Without success.

Perhaps it has smething todo with the the order of the skills that get loaded when Mycroft starts.
Is there a chance to force mycroft to load the skill at the end?

suisat · July 18, 2019, 2:57pm

@j1nx sorry for my afk’nes

Did you think we can together bring this skill to a stable rls?

j1nx · July 19, 2019, 9:55am

@suisat No worries, I was (still am a bit) busy aswell.

I am about to reflash my sd with the latest picroft image to start tackling this plugin again. So yes, let’s work together to figure out how to fix this. Assume it is just a small thing we are overlooking and we should be able to get it fixed.

Hope to get start cracking after the weekend.

mtwannabe · August 30, 2019, 10:32pm

Any more updates on this? I’d love to see this working so i can replace the microphone in mine. I really love the microphones in the newer Alexa Echo (yes we play with them all) and would love my Mycroft to not feel left out.