Getting Mycroft to listen within a skill


#1

hello All,

I am working on a list skill, a skill that can create a text file and add and remove, items from the file, as well as clear everything or delete the file. i am planning to use it a a house hold shopping list (so the kids will let us know when we are out of things) as well as a general list generator. eventually i want to have it so it will save the lists that i specify to the cloud (like google docs) so i can retrieve the list from the store.

My question is "is there a way to get mycroft to listen within my skill?"
so when i ask it to delete a list it will ask me if i am sure and listen for the yes or no response so i do not accidentally delete the file.

thank you in advance.

Mark


#2

Using the already accepted API: you can request play a message (skill methods “speak” or “speck_dialog”) adding to the command the argument “expect_response=True”.

This flag causes mycroft to play the message and, immediately, continue as if a “hey, mycroft” has been listen. You have then 10 seconds to speech a new command.

Another option --disclaimer, I’m its author-- still in “pull request” stage is to use the local pocketsphinx client. This client does the voice recognition locally and adds some new features:

  • session ids to correlate all messages of a session skill.

  • use a specific grammar for a specific record: it is not a good strategy to use a generic STT when the only expected answers are “yes”/“no”.

  • use a local STT: remote STTs have (will have) a price of around 1$/100 recognitions. It is a waste of money if only recognition of sequence of “yes”/“no” words is need.


#3

hello PasabaPorAqui,

that is awesome thank you! is there anyplace that lists all of the functions for mycroft?

i played with pocketsphinx trying to accomplish my overall task for my project because i was having troubles finding a way to get access to the google APIs’ cheep. that is when i stumbled across mycroft, witch seems to be a perfect match for what i am doing, i found that pocketsphinx is great for a limited vocabulary but once i started to get up in to a large vocabulary it had problems distinguishing some words and became slower and slower. so i really like your idea of using pocketsphinx for use within mycroft for things just like this :slight_smile:

Thank you very much for your answer, i will give your skill a try!

Mark


#4

I’ve a similar experience. After some very disappointing tests with Google English STT (ok, could be it was caused by my perfect spanglish … or not), I decided not be ambitious. I do not need talk about philosophy with Mycroft.

What I need is small commands (“play music”, “add to shop list”, …). For these commands a jsgf grammar file is perfect, reaching near than 100% recognition.

Only some of these commands can trigger an answer followed by a free speech record. By example “mycroft, look at wikipedia” can be followed answer “what I must look for?” and a free speech phrase that needs to be converted using a generic language model. For this reason, the pull request done adds the possibility of switch between pocketsphinx grammars (or change the STT service provider).

In some other cases the answer will be followed by a record not to be translated by the STT. By example, “mycroft, remind me in ten minutes” can be followed by “what I must remind you?” followed by a phrase not to be translated, because after ten minutes mycroft will play exactly the recorded sound without processing nor understanding.
( yes, I’m tired of burn the chicken, I want say to mycroft “remind me in 5 minutes: stop the cooking fire”. I do not need that mycroft understands “stop the cooking fire”, just that replays it).

I see very difficult nowadays find an STT that understands all the thousands of commercial product names. For this reason, I would considerer a simple skill started with “mycroft, add to shopping list” followed by play “what I must add?” and record of a free phrase.


#5

that sounds exactly like what i need i was just not having much luck with my word/phrase list. I will have to look in to sgf grammar file that looks very interesting i just do not understand enough about any of this yet.

this is my first time programming in an environment like this and first time programming in python (i am old school assembler and the basic i took in high school lol) i have had some experience with C++ but not a bunch so i am taking an online python course just the basics it is helping and this is really helping because i am now motivated to make it work. spending hours messing around trying to get the code to do what i want lol.
do you know of a place that has the Mycroft specific commands and calls in a list with a bit of an explanation on syntax and how they should be used?

Mark


#6

Could be some day the necessary changes are integrated in official mycroft code but, in the while, what I have is:

New speech client with the features said: download from here

Modify start to replace official voice client with this one. Change line from:

"voice") SCRIPT=${TOP}/mycroft/client/speech/main.py ;;

to

"voice") SCRIPT=${TOP}/mycroft/client/lspeech/main.py ;;
  1. In order to use several new features in the skills, download base class from here and store it as …/mycroft/skills/multi_thread_skill.py

With that, you can start development of your skill. An example with comments can be seen here.

In order to use a reduced grammar for commands, config file in listener block must contain:

  "listener": {
    ...
    "producer": "pocketsphinx",
    "grammar": "jsgf"
}

and create a file es.jsgf (or en-US.jsgf) with the supported commands:

#JSGF V1.0;

grammar prueba;
    
public <prueba> = <cmnd1> | <cmnd2> | <cmnd3> | <cmnd4> ;
<cmnd1> = apaga la música ;
<cmnd2> = pon música ;
<cmnd3> = avisa <when> | <when> avisa ;
<cmnd4> = graba un mensaje ;
<when> = en <n1> ( minuto | minutos ) ;
<n0> = un | dos | tres | cuatro | cinco | seis | siete | ocho | nueve ;
<n1> = <n0> | treinta y <n0> ;