Build an open future with us.

Invest in Mycroft and become a community partner.

Speech Rec For MyCroftPi Using Kinect

#1

I was thinking of downloading the Raspberry Pi MyCroft image. However I was not interested in using any ol’ microphone for speech rec. I wanted to use a 1st gen Microsoft Kinect for the mic. The Kinect has a microphone array so it is excellent microphone for speech recognition.

My problem is this. I do not know how to install the Kinect to be able to do this. I have looked around and most people who talk about Pi and Kinect are only concerned with the video portion of this.

Would one of you kindly walk me thru how to install the Kinect?

Much appreciated.

#2

Does anyone know how to do this? Havent had any answers yet.

#3

Hey Dominique,

It’s something I’m wanting to do. I have a 1st gen Kinect, but running into issues getting it working on the pi. It’s definitely not the easiest option, but I’ll post once I find out more.

#4

Thanks. I would like to get this working as well. I think the Kinect using the microphone array it has onboard would make an excellent speech rec microphone.

Someone turned me onto this but I do not know how to implement it.

#5

Hey guys, I have just bought a Kinect and I am trying to set it up to use as a microphone for picroft however, I am not having much luck with it. Did either of you ever get it working?

#6

Hi there, I’m pretty much doing the same thing - I have Kinect (v1414) working under both Raspbian & Ubuntu mate (16.04) and you need to do a few things to get it working :-

Raspbian - will work for cam but NOT mic
Ubuntu Mate - Will work for Cam & Mic - preferred solution

  1. Install Libfreenect & deps following this (do lsusb to check OS can see Kinect)
    https://sites.google.com/site/hntuan94/home/kinect/-2-connect-kinect-to-raspberry-pi-2-using-libfreenect

  2. I then test in X to see cam, etc,
    test in X-Windows using “python demo_cv2_async.py” from libfreenect/wrappers/python/

  3. Get the Microsoft SDK & Firmware
    apt-get install kinect-audio-setup

  4. Check you can see everything - note the hw: info for each
    aplay -l
    arecord -l

  5. I added ~/.asoundrc file following instructions here
    https://developers.google.com/assistant/sdk/guides/library/python/embed/audio

  6. Probably reboot now is a good time

  7. Set volumes & check devices are seen (if level 0, press M to unmute and < > to increase vol)
    alsamixer
    alsactl store

  8. Check with a test wav file
    aplay filename.wav

  9. Can you record too ?
    arecord --duration=5 -t wav testfile.wav

All seems to work ok for me on Rpi3 with Ubuntu mate, however, I can’t get it to work in Mycroft !!!
I only installed last night (took all night to compile) - so if I sort this today I’ll let you know.

If anyones mic or output in Mycroft with Kinect is working let me know !!!

This works a dream for Google Assistant - but I want a custom wakeword (and snowboy doesn’t work well and I rather support Mycroft).

Good luck !

#7

ok, found a typo in my conf, I had “play_mp3_cmdline” instead of wav - so make sure it looks like this in the mycroft.conf file :-

// Mechanism used to play WAV audio files
// Override: SYSTEM
//“play_wav_cmdline”: “paplay %1 --stream-name=mycroft-voice”,
“play_wav_cmdline”: “aplay -Dhw:0,0 %1”,

Then both Mic & output work (speakers from 3.5mm jack, Kinect Mic).

However mic input seems to freeze after a few mins ?

EDIT:
All working fine now, fresh Ubuntu 16.04 install, Kinect or USB headset working for Mycroft (run in GUI console !), libfreenect & servo controls all working, Kinect depth & video work too :slight_smile: