Custom hot word stopped to work

baconator · September 17, 2019, 4:01pm

Something is different about how your model is getting used in mycroft vs. how it’s getting used when you’re trying precise-listen. I can’t diagnose that remotely without much more info.

If your model that you uploaded works for you on precise-listen, great. All you should have to do is upload it to your precise host and set the config for it.

One thing to try is running the precise-engine on the precise host with the model, and pass it wav files of your wake word. If that completes as expected, then it seems like the mic would be the issue. For mic issues, you’d have to verify the input, volume, etc., and check any other mycroft config settings that might be in place for it.

Dodo_Ivanecky · September 17, 2019, 4:51pm

ok, now I understand you. Yes, I fully agree with you, something is different on mycroft/precise level. All I found out so far is, that my Mycroft is not using the same precise as I use when I use precise-listen. That’s why I started with suspect of differences in some audio processing.
I have also no clue what precise host is. But I have the keyword, so I will look at it. Thanks.

baconator · September 17, 2019, 7:12pm

precise host is the computer you’re running precise on.

Dodo_Ivanecky · September 18, 2019, 1:29pm

I am running everything on one computer. So there is just one audio input. The same for both.

baconator · September 18, 2019, 3:18pm

" Mycroft is not using the same precise as I use when I use precise-listen."

Can you elaborate more on that?

Dodo_Ivanecky · September 18, 2019, 3:22pm

Mycroft uses mycrof-precise installed as a package via pip install. For Precise I am talking about all the time I used git clone installation. So without any pip package installation.

baconator · September 18, 2019, 3:32pm

also if you want to send a single (working) example of your wake word over that might be useful as well.

Dodo_Ivanecky · September 19, 2019, 9:32am

#Projects/mycroft-precise$ precise-test hallo-scotty3.pb hallo-scotty/
Loading wake-word…
Loading not-wake-word…
2019-09-19 11:23:24.523353: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2019-09-19 11:23:24.544040: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2904000000 Hz
2019-09-19 11:23:24.544352: I tensorflow/compiler/xla/service/service.cc:150] XLA service 0x1b4ce80 executing computations on platform Host. Devices:
2019-09-19 11:23:24.544376: I tensorflow/compiler/xla/service/service.cc:158] StreamExecutor device (0): ,
Data:
=== False Positives ===
=== False Negatives ===
=== Counts ===
False Positives: 0
True Negatives: 40
False Negatives: 0
True Positives: 15
=== Summary ===
55 out of 55
100.00%
0.00% false positives
0.00% false negatives

All the test data are here:
https://drive.google.com/file/d/1--Xsai_pqeBzfH-PFGHMdTRAgv_lhaFA/view?usp=sharing

baconator · September 20, 2019, 3:35am

I added a few words and modeled that for 1k steps.

Can you see if that works any better for you?

Dodo_Ivanecky · September 20, 2019, 8:13am

Hm, what have you done?
On precise-listen I do not see any difference. It wors for me the same way as my models. But under mycroft it started to work. No as well as under precise-listen, but ut does work. So I really wanna understand what is the trick/problem.
Thanks

Dodo_Ivanecky · September 20, 2019, 10:33am

Actually, your model is not really usable. It has too many Falso positives … Despite the fact that test on my test data does not show it

baconator · September 20, 2019, 1:46pm

On your (github) copy of precise, check out commit b3e6efa. Remove any previous models/logs (keep your audio clips), and try training again.

Dodo_Ivanecky · September 20, 2019, 1:56pm

I didi it. And it wors perfectly as before. What shell I achieve? Not working precise-listen?

baconator · September 20, 2019, 5:43pm

I don’t know. What you’re describing makes no sense. If it works for mycroft, that’d be what the end goal should be?

Dodo_Ivanecky · September 21, 2019, 4:31pm

Frankly, I have all the time feeling you do not understand what the problem is. I try one more times:

I have my own model. It works perfectly with precise-test or precise-listen. I am absolutely satisfy with give model.
When I want to use the model with entire Mycroft, so I configure Mycroft to use for the how word my model, it works terribly wrong. So in this case I run in mycroft-core directory ./start-myscroft.sh debug

Is it clear now? I do not know how shell I explain it to you. So I do not think the problem is my model.

j1nx · September 21, 2019, 8:32pm

Just chiming in here, but are you sure the precise version of 1) is the same as 2) ?

mycroft uses precise-runner binary version 0.2.1 while the defailt version of the github sources are already at 0.3.1

models created on 0.3.1 don’t work on 0.2.1

So are you sure you are using both the 0.3.1 binaries and models within mycroft and not only the model made via 0.3.1 without changing/updating the binaries?

Dodo_Ivanecky · September 23, 2019, 7:49am

No, I am not sure and I did mention it already. Look at the older messages. But I always had to react to model related reply. I asked ven explicit question if there were any important changes in precise between the versions …

j1nx · September 23, 2019, 2:01pm

Precise is not really my cup of tea, @baconator is way more experienced in that regard.

Reading is not my cup of tea either but then again, your’s not either as my comment is stated within the README You must have read it as you managed to install from source, which is even a bridge to far for me

github.com

MycroftAI/mycroft-precise/blob/dev/README.md

# Mycroft Precise

*A lightweight, simple-to-use, RNN wake word listener.*

Precise is a wake word listener.  The software monitors an audio stream ( usually a microphone ) and when it recognizes a specific phrase it triggers an event.  For example, at Mycroft AI the team has trained Precise to recognize the phrase "Hey, Mycroft".  When the software recognizes this phrase it puts the rest of Mycroft's software into command mode and waits for a command from the person using the device.  Mycroft Precise is fully open source and can be trined to recognize anything from a name to a cough.

In addition to Precise there are several proprietary wake word listeners out there.  If you are looking to spot a wakeword Precise might be a great solution, but if it's too resource intensive or isn't accurate enough here are some [alternative options][comparison].

[comparison]: https://github.com/MycroftAI/mycroft-precise/wiki/Software-Comparison

## Supported Operating Systems

Precise is designed to run on Linux.  It is known to work on a variety of Linux distributions including Debian, Ubuntu and Raspbian.  It probably operates on other \*nx distributions.

## Training Models

### Communal models

Training takes lots of data. The Mycroft community is working together to jointly
build datasets at:

This file has been truncated. show original

If you would like your models to run on an older version of precise, like the stable version the binary install uses, check out the master branch.

It doesn’t matter. Hope we managed to get it sorted for you. Both @baconator and me are just community members, trying to help out where we can. The Dev’s if MycroftAI are currently working very hard to get 19.08 out…

baconator · September 23, 2019, 3:12pm

There’s almost certainly a version difference. There are changes between the .2 and .3 versions.

Dodo_Ivanecky · September 24, 2019, 11:31am

Great. So simple. I took the master branch from precise and everything started to work. Thanks j1nx!!!
However then I did recommended update and everything stopped to work. But that’s a different story