Custom Wake-Word not Working when Implemented into Mycroft

Hello,

I’ve been working to create a new wake word (Hey Tux) using Precise. I did the Source Install that was instructed here in Train your own model by the MycroftAI team.

I used the instructions here How to train your own wake word to create TensorFlow models that should be able to be utilized by Mycroft as a custom wake word. I have 190,000 not-wake-words along with 150 wake-words for creating the model, and I have 43,00 not-wake-words and 49 wake-words for testing the model. After long hours of acquiring the data and training the model, I achieved 99% accuracy.

Precise-Listen works perfectly, and I now have the converted model - the two files:

  1. Hey-Tux.pb
  2. Hey-Tux.pb.params

Following the instructions here Using a Custom Wake Word, I edited my User level configuration to tell Mycroft to use my file and then reloaded.

I even checked if my syntax was correct using this tool and it said that everything was fine. I don’t have any issues with the microphone or AVX since I went through all the troubleshooting here.

Afterwards when I started Mycroft services (including CLI) using debug, Mycroft was not responding anymore. To check if my Wake-Word had been accepted, I typed “Who are you” to which Mycroft responded “I’m tux, an open-source A.I.”. This would suggest that Mycroft implemented the new changes, but somehow it doesn’t recognize when it is being called (which doesn’t make sense to me). I’ve read multiple forums and chats, but none have thus far helped me solve my problem. The only reason why this seems to be happening may be here. It is mentioned that Mycroft uses precise 0.2, but I don’t know how to check my version of precise to prove or dismiss whether this is the case.

I would appreciate all the help I can get. Thanks in advance!

I’m running Ubuntu 18.04.

1 Like

What’s in /var/log/mycroft/voice.log from startup?

Also might be worth getting a few more samples from those two false negs for additional training data if possible.

You can check out the 0.2.0 version here: https://github.com/MycroftAI/mycroft-precise/releases/tag/v0.2.0 if you haven’t already.

I’m not sure how exactly to access the voice.log? Is “/var/log/mycroft/” the directory where it is stored? Or do I need to type this in the virtual environment?

In regards to those two false negatives… Should I record more wake words from those people?

Lastly, I did some digging in my mycroft-precise folder and it turns out I am running the version 0.3.0 (the one I used to create the Hey-Tux.pb files). I used the link you gave me to retrieve version 0.2.0 (but the issue hasn’t resolved yet due to me getting the error " cannot import name ‘create_parser’ " when I run precise-train on version 0.2.0

If you run “less /var/log/mycroft/voice.log” you can review the whole file or “tail -40f /var/log/mycroft/voice.log” when starting up, you’ll get to watch it scroll by live.

Yes, if you can get similar vocals from those folks would be useful.
You’ll need to make a new venv for .2.0.

I’d also cut down the number of nww’s you have by maybe half for now, the ratio is pretty wildly leaning towards the not wake words. Once you have it working, make sure you’re saving wake words, then you can build your dataset and also track false positives to improve your model with.

I managed to get everything working. The issue was that I was running precise version 0.3.0, and I used the link you gave me to get version 0.2.0. I also recorded more wake words, but now the model the model is not as accurate. It is activating on noises that are similar to “Hey Tux”, but that’s another issue. I created a new venv for 0.2.0 as you said, and (although it is slower) but it works. I think I’ll stick with my nww because my wake word is only 2 syllables, so it activates easily (in comparison to other wake words), so I need to give precise as many examples of nww as I can to recognize as when not to wake up. (However if you think that is a bad idea, do let me know!)

I also ran “tail -40f /var/log/mycroft/voice.log” as per your request (thank you for teaching me how to get it) and I got:

2020-05-04 22:20:50.908 | INFO | 19244 | mycroft.client.speech.listener:create_wake_word_recognizer:323 | Creating wake word engine
2020-05-04 22:20:50.909 | INFO | 19244 | mycroft.client.speech.listener:create_wake_word_recognizer:346 | Using hotword entry for hey tux
2020-05-04 22:20:50.910 | WARNING | 19244 | mycroft.client.speech.listener:create_wake_word_recognizer:348 | Phonemes are missing falling back to listeners configuration
2020-05-04 22:20:50.912 | WARNING | 19244 | mycroft.client.speech.listener:create_wake_word_recognizer:352 | Threshold is missing falling back to listeners configuration
2020-05-04 22:20:50.914 | INFO | 19244 | mycroft.client.speech.hotword_factory:load_module:386 | Loading “hey tux” wake word via precise
2020-05-04 22:20:52.308 | INFO | 19244 | mycroft.client.speech.listener:create_wakeup_recognizer:360 | creating stand up word engine
2020-05-04 22:20:52.310 | INFO | 19244 | mycroft.client.speech.hotword_factory:load_module:386 | Loading “wake up” wake word via pocketsphinx
2020-05-04 22:20:52.429 | INFO | 19244 | mycroft.messagebus.client.client:on_open:67 | Connected
2020-05-04 22:21:07.510 | INFO | 19244 | mycroft.session:get:74 | New Session Start: de5c5c2e-0e5a-4ed9-a04c-1c809dcf3eb0
2020-05-04 22:21:07.513 | INFO | 19244 | main:handle_wakeword:67 | Wakeword Detected: hey tux
2020-05-04 22:21:08.130 | INFO | 19244 | main:handle_record_begin:37 | Begin Recording…
2020-05-04 22:21:11.179 | INFO | 19244 | main:handle_record_end:45 | End Recording…
2020-05-04 22:21:12.437 | ERROR | 19244 | mycroft.client.speech.listener:transcribe:239 | list index out of range
2020-05-04 22:21:12.439 | ERROR | 19244 | mycroft.client.speech.listener:transcribe:240 | Speech Recognition could not understand audio
2020-05-04 22:21:21.775 | INFO | 19244 | main:handle_wakeword:67 | Wakeword Detected: hey tux
2020-05-04 22:21:22.175 | INFO | 19244 | main:handle_record_begin:37 | Begin Recording…
2020-05-04 22:21:25.172 | INFO | 19244 | main:handle_record_end:45 | End Recording…
2020-05-04 22:21:26.473 | ERROR | 19244 | mycroft.client.speech.listener:transcribe:239 | list index out of range
2020-05-04 22:21:26.475 | ERROR | 19244 | mycroft.client.speech.listener:transcribe:240 | Speech Recognition could not understand audio
2020-05-04 22:21:29.586 | INFO | 19244 | main:handle_wakeword:67 | Wakeword Detected: hey tux
2020-05-04 22:21:29.985 | INFO | 19244 | main:handle_record_begin:37 | Begin Recording…
2020-05-04 22:21:33.775 | INFO | 19244 | main:handle_record_end:45 | End Recording…
2020-05-04 22:21:34.540 | INFO | 19244 | main:handle_utterance:72 | Utterance: [‘who are you’]
2020-05-04 22:21:43.950 | INFO | 19244 | main:handle_wakeword:67 | Wakeword Detected: hey tux
2020-05-04 22:21:44.378 | INFO | 19244 | main:handle_record_begin:37 | Begin Recording…
2020-05-04 22:21:46.257 | WARNING | 19244 | mycroft.messagebus.client.client:on_error:79 | Could not send message because connection has closed
2020-05-04 22:21:46.260 | ERROR | 19244 | mycroft.messagebus.client.client:on_error:88 | Exception closing websocket: WebSocketConnectionClosedException(‘Connection is already closed.’,)
2020-05-04 22:21:46.262 | WARNING | 19244 | mycroft.messagebus.client.client:on_error:91 | Message Bus Client will reconnect in 5 seconds.
Exception in thread Thread-17:
Traceback (most recent call last):
File “/usr/lib/python3.6/threading.py”, line 916, in _bootstrap_inner
self.run()
File “/usr/lib/python3.6/threading.py”, line 864, in run
self._target(*self._args, **self._kwargs)
File “/home/shahzeb/mycroft-core/.venv/lib/python3.6/site-packages/precise_runner/runner.py”, line 197, in _handle_predictions
prob = self.engine.get_prediction(chunk)
File “/home/shahzeb/mycroft-core/.venv/lib/python3.6/site-packages/precise_runner/runner.py”, line 65, in get_prediction
return float(self.proc.stdout.readline())
ValueError: could not convert string to float:

The more nww you can add that trigger it, the better your results are in my experience.

this:

2020-05-04 22:21:43.950 | INFO | 19244 | main:handle_wakeword:67 | Wakeword Detected: hey tux

indicates it’s hearing your wake word (successfully, I hope).

Not to hijack this post, but I’ve been having the same issues. I’ve been able to follow this discussion through switching to v0.2.0, and now I’m stuck with the same " cannot import name ‘create_parser’ " error. Looks like I need to make a new venv, maybe a dumb question but how do I do that?

You can type:

cd mycroft-precise-0.2.0
./setup.sh

This would create your venv, and then to run your venv, you should type:

source .venv/bin/activate

After doing this, all precise commands should work (i.e. precise-collect, or precise-listen, etc)

1 Like

I am working on a clean download of mycroft-precise-0.2.0, so I already had to do all that. I thought this meant I had to remove and re-create the venv, but I guess not. Any idea what else could be causing this error?

Oh yeah now I remember. I had the same issue when I got version 0.2.0. It is caused by misalignment of versions (similar to how Precise version 0.3.0 won’t work with Mycroft Core). How I managed to eliminate the error was:

(First activate the venv)
cd mycroft-precise-0.2.0
source .venv/bin/activate

Then to eradicate the error:
pip install ‘prettyparse<1.0’

This should fix your bug. I also faced another issue with incompatible version of keras. I solved it by:
pip install keras==2.1.5

(Just a reminder that all these commands are executed in the virtual environment - venv)

Ah that and the keras version fixed it, thanks for the help.

1 Like