Build an open future with us.

Invest in Mycroft and become a community partner.

Virtual Devices


#1

Hello, everybody :smiley:
In the last days I am thinking about a “Virtual Device” system for mycroft devices.
So when I connect ESP32’s to a single picroft (running as server) (with WIFI) and send back the recorded audio data, the pycroft processes the audio and sends back the result (TTS, music,…). The microcontroller would then output this.
Here a microcontroller would be a virtual device and is treated like a normal device that I can add to my account.
Since I am familiar with programming, I would do it myself.
What I need is information on where to start and suggestions for improvement.


#2

You should have a look at Mycroft Messagebus docs - while the functioanility you want is currently not there (yet), it nevertheless gibes you an idea how to hook up the Esp-devices with Mycroft.


#3

I’ve been considering something like this myself.

Easiest would be to create a separate mycroft client (running in parallell with mycroft-speech-client) that hosts a simple socket server which the ESP32 can connect to and send wav-data to.

I created this example a long time ago: https://github.com/forslund/mycroft-wave-client

This one reads from a file but the same architecture can be used to read from a socket.

If you can get the ESP32 to record audio and send using a standard socket I can help with the mycroft side if you like.

I vote the ESP32 device should be called μCroft (microcroft/mucroft) :smiley:


#4

Totally calling those sherlocks


#5

Got looking for microphones for my small devices. Do you have any recommendations? I’ve been looking at https://www.electrokit.com/uploads/productfile/41016/adafruit-agc-electret-microphone-amplifier-max9814.pdf


#6

Okay I will PM you when I am ready :smiley:
Also voting for that name :+1:

Unfortunately no.


#7

I’m definitely not as far into this as most here, apologies if my thoughts don’t apply to this… but I had pondered if this would work as a way to do multi-person voice recognition?

For example you run three μCrofts… one to recognize your voice, one for another person (in my example it would be my wife), and the 3rd for all others… all reporting back to a main picroft… with unique accounts to allow different actions…


#8

The idea is that every uCroft works as a virtual device so like a normal picroft.
Why multiple voices wouldn’t be able?


#9

From what I have read, I thought multiple voice recognition wasn’t possible on the standard setup? Sorry, like I said, just getting started looking into MyCroft and have pulled the trigger on setting up a device yet.


#10

Hi there @Senkrad, unfortunately this isn’t possible at the moment, but multiple speaker recognition is something we would really like to do longer term.

This would require a few technical pieces to be in place;

  • We would need a way for people to record their voices - voice samples - so that Mycroft could then compare the recorded sample with someone speaking to do a match.

  • We would need a Wake Word listener that compared the recording of a Wake word with the voice sample on file to determine “who” was speaking

  • We would also need a way to handle the case where the voice could not be matched to a sample on record.

So, it’s doable, but requires a fair bit of work.