Wake word detection

Community Projects Neon AI's Voice AI Platform
1

I have switched the wake word over from “hey neon” to “hey mycroft”
Despite that, when the TV is on, when our dog barks, I get so many false positives of the wake word detection and then of course “you have to say that another way”

Could a more precise standard WW be implemented in the mk2 neon version?
One where the user doesn’t have to fiddle with learning models etc?

Inquiring mind wants to know.
Currently we switch off the mic when we start the TV.
Kind of defeats the purpose of having a local voice AI

2

I know @mikejgray has been playing with OpenWakeWord models with some success. We also have a draft community project for a web UI to collect wake word audio samples.

If anyone is interested in working on this, I can flesh out that draft issue to get the data collection and model training started for a better “Hey Neon” (and possibly incremental training of "Hey Mycroft). :slight_smile:

1 Like
3

It is nice to see a lot of updates, both visible and under the hood.
EG:
I like the new kernel, I like the flexibility for tinkering with BT and local TTS & STT options.
But for the non-tinkering people, a more solid base of having a better wake work detection is imo an essential needed upgrade.
Just an upgrade that makes the units in the field perform (way) better.

What exactly do you mean with “If anyone is interested in working on this”
If another person does the actual work, you will include it in the next release ?

4

I agree this would be a significant upgrade that would benefit all users in a pretty visible way, compared to some of the other updates that help with speed and stability but don’t necessarily translate to a big visible change.

To better explain what I have in mind here, I see a few pieces we need to get to a better wake word model.

  1. More samples. This is what the linked issue is relevant to; something like a public web UI where users can record themselves saying the wake word and submit the samples to a public dataset (kind of like CommonVoice).
  2. A WW model training pipeline to use all of the submitted samples. My current idea is to use Open Wake Word where we can supplement the collected samples with other generated data.
  3. Once we have that pipeline in place, images can pull the latest generated output model for each release.
5

I have a “Hey Neon” model I’ve been using lately that works pretty well. It’s possibly not quite sensitive enough, but I haven’t had any false activations so far. I also don’t keep the unit near my TV :slight_smile:

Happy to share the model, including some more sensitive ones I haven’t tried, along with instructions to run them in the Mark 2!

2 Likes
6

If anyone does want to give this a try, here are instructions.

First, download the model: https://raw.githubusercontent.com/NeonGeckoCom/neon-iris/dev/neon_iris/wakeword_models/hey_neon/hey_neon_high.tflite

Next, copy the model to your Mark 2’s /home/neon directory. scp is probably the best choice here.

Then, update your ~/.config/neon/neon.yaml file to include the following:

extra_dependencies:
  voice:
    - openwakeword~=0.5,<0.6
hotwords:
  hey_mycroft:
    active: false
    listen: false
  hey_neon:
    module: ovos-ww-plugin-openwakeword
    models:
      - /home/neon/hey_neon_high.tflite
    inference_framework: tflite
    active: true
    listen: true
skills:
  default_skills:
    - openwakeword~=0.5,<0.6

Finally, reload the Neon services. This is required to make sure the openwakeword package downgrades sufficiently if you’re running a late February version of Neon. Future readers, you may not need to reload anything, since the changes will be picked up by the voice service automatically.

Have fun! This should be much less sensitive, but may not be sensitive enough.

2 Likes
7

Looking good so far!
Wife is watching tv, I am keeping an eye on voice.log for WW detection !

2 Likes
8

I’m getting a lot of false positive wake ups - most commonly TV news but sometimes just conversations around the kitchen table.

Thanks for this, I’m giving it a go

On the instructions

  • I downloaded the model by using wget on my Mark 2 in an ssh session. That seemed a lot simpler.
  • is there a command line way to restart services? (I did it from the touchscreen menu)
9

Definitely less sensitive, initially I didn’t think it was working. It’ll get the full test this evening with the TV news.

10

I haven’t had a single false positive wake up in the last 24 hours, which is great. As noted elsewhere - this is a big and very noticeable improvement for non-technical people, other members of my family …

OTOH for me, sometimes I can’t get the wakeword recognised and I hit the top circle button instead.

If I look in the voice.log does hitting the big circle button appear as ‘wakeword detected’?

If the wakeword is detected, I’m expecting the home screen to be highlighted around the edge, is that right? If I don’t see the highlighting I understand that as the wakeword not being detected, is that right?

Just trying to make sense of my experience compared with what I see in the voice.log. Happy to test any suggestions.

Thanks

11

CPU idle time on my MK2 went from just over 80% with the old model to 67.5% with the new model. For sure the CPU is working a lot harder.
Me and wife took turns to try and the ww recognised.
We stopped after a while, but indeed: no more false positives during TV time.
Blessing & a curse

1 Like
12

This is very experimental, so thank you two for being the guinea pigs here! OpenWakeWord does have a training option where you can provide a few samples of the user saying the wakeword, then overlay that with the model. I haven’t tried it yet and there isn’t any Neon/OVOS-compatible code for it yet. But I do intend to set this up as a skill when I get some time…

“Hey Neon, meet my friend”
“Hey Neon, I want to introduce you to someone”
“Hey Neon, please take instructions from this person”
“Hey Neon, please listen to this person”

1 Like
13

I haven’t tried it yet and there isn’t any Neon/OVOS-compatible code for it yet.

there is a OVOS plugin for OpenWakeWord, from the author himself!

or did you mean an integration with the training code itself?

1 Like
14

I mean the integration with the part that lets you fine tune it. They’re using the plugin already

1 Like
15

I’ve just noticed something … doh!

neon/yaml already has sections for extra_dependencies and hotwords …

Does ‘include’ mean add or replace the existing sections?

I also noticed under hotwords that in my file, wake_up was set to false. It doesn’t sound right, so I changed it to true.

Thanks

16

You can append that to ~/.config/neon/neon.yaml. There is some configuration documentation here but basically the user path here will take priority over any default/system configuration that exists at other paths.

Keep in mind that the ~/.config path is where you should make changes, the others (i.e. /etc/neon/neon.yaml) will be overwritten by updates

17

Thanks.

Now I’ve tried both ways.

So far, what I’m seeing, either way, is that after restarting services my MK2 responds to the wakeword once or twice, then stops responding, nothing appears in the voice log … unless I press the big top button, then it responds normally to voice commands.

Is there something/somewhere I should be looking to see what the problem is?

18

  1. You can see relevant logs in ~/logs/voice.log; grep -i error ~/logs/voice.log is a quick way to see if there are any errors there.

  2. If you can also add to neon.yaml:

    listener:
  record_wake_words: true

    then any recognized wake word will go in ~/.local/share/neon/listener/wake_words. You can play back some samples just to be sure that what Neon hears is reasonable (i.e. loud enough, not a lot of captured background noise). It seems unlikely to be a microphone problem if the STT works normally but something to check…

19

Thanks for coming back to me.

What’s puzzling me is that there are no errors at all in voice.log and when my MK2 stops responding to the wakeword, there is nothing at all written to the voice.log …

unless I hit the top button, then I see ‘wakeword detected’ and ‘speech finished’ and 'INFO - transcribed … ’ (just to repeat I haven’t spoken the wakeword, I’ve hit the top button instead)

To make sure I haven’t messed up the neon.yaml file, I’ve gone back to the original file and appended the extra part in the instructions above. This has made no difference. My MK2 responds normally to 'Hey Neon … ’ once after reloading services, then doesn’t respond at all, but does respond normally to hitting the top button.

I can see a line in the voice.log when I update the neon.yaml file
voice - ovos_dinkum_listener.service:reload_configuration:973 - INFO - Reload Completed but this doesn’t get the MK2 responding to the wakeword. It only works if I ‘restart services’ from the pull down menu on the home screen.

I’ve added your #2. suggestion to neon.yaml and I’ll have a look at what that does too.