Help us improve Wake Word precision: Introducing Precise Tagger

Originally published at: http://mycroft.ai/blog/introducing-precise/

In November 2017, we introduced Precise - a Wake Word listener to replace the PocketSphinx engine. Precise was recently promoted to be the default Wake Word listener for all Mycroft Devices. You might remember that Precise and PocketSphinx work in different ways. PocketSphinx recognises a Wake Word based on a speech to text translation - that is, it tries to recognize words. Precise takes a different approach. Instead of trying to match words, it matches based on sounds - think of a wave form rather than a typed string.

In order to make Precise more accurate, Matthew from our Developer Team has recently released Precise Tagger. It’s a tool to train the neural network that helps Precise distinguish between four different conditions:

  • False positive: Precise detects a phrase as a Wake Word, when it is not a Wake Word
  • Correct negative: Precise detects that a phrase is not a Wake Word correctly
  • Correct positive: Precise detects correctly that a phrase is a Wake Word
  • False negative: Precise does not detect a phrase as a Wake Word, when it is a Wake Word
It's super easy to help train Precise to be more accurate.

Precise Tagger will provide you with a Wake Word sample, contributed by one of the over two thousand Mycroft Community members who have opted in to providing completely anonymized open data. You will be presented with three options to ‘tag’ the sample - just like the below image. Simple!

precise-tagger-button-options

Go to Precise Tagger and help us make Precise Wake Word listener even more accurate

At the moment the tagging counter doesn’t mean a lot - it’s not yet integrated into home.mycroft.ai, but we’re considering how we will built this into a rewards program for Mycroft Community contributors. But still, who doesn’t want to be top of the leaderboard?!

1 Like

What about a skill for this ?

Hey Microft - train precision.

OK - then sound - waiting for yes/No/Almost - next sound…until saying stop training

Great suggestion @andlo!
You might want to write it up here on our Skill Suggestions board;
https://community.openconversational.ai/c/skill-suggestions

Hi, Is there any information regarding the type of neural net, BAM, How many nodes, etc being used?

@Wolfgange is best positioned to answer this as the Lead on this project :slight_smile:

I am tagging words from time to time, a few hundreds of samples at a time. Looking at the statistics, If I read them correctly, the accuracy of the recognition is not getting better, in fact it stays the same pkus or minus some random error, although almost 80% of the original 100000 samples have been tagged.
So either -

  • the initial training data was already optimal
  • the new data is not taken into account, or is not accurate.
  • I misread the information

Which is which?

@Wolfgange might have some thoughts on this one

1 Like

The model we are currently using with Precise isn’t improving significantly. If anyone would like to experiment with different models they can take a look at the precise github page. Otherwise, me or @Mn0491 will continue trying to improve the model when we get some spare time so that it will fit all the new data better.

2 Likes