Mycroft not listening after anserwing?

Plugins & Skills Feedback / Support
1

Hi,

I am using Picroft 18.8.4. (enclosure version 1.4.0)

In the doc I find example of conversation like this one:

User: Where is John Cleese from?
Mycroft: He’s from England
User: What’s the weather like over there?
Mycroft: Raining and 14 degrees…

Because I need to say the wake word before each question, I don’t find this kind of conversation.

Mycroft doesn’t seem to return to listening just after answering.

This a log sample.

12:14:13.360 - SKILLS - DEBUG - {“context”: null, “data”: {“utterance”: “hey mycroft”, “session”: “0d26c4f9-9fda-40c5-ac9c-59679d64e6f3”}, “type”: “recognizer_loop:wakeword”}
12:14:15.523 - SKILLS - DEBUG - {“context”: {“ident”: “1541070853.3456345137989080”, “client_name”: “mycroft_listener”}, “data”: {“lang”: “en-US”, “utterances”: [“who is john cleese”], “session”: “0d26c4f9-9fda-40c5-ac9c-59679d64e6f3”}, “type”: “recognizer_loop:utterance”}
12:14:15.566 - SKILLS - DEBUG - {“data”: {“lang”: “en-US”, “utterance”: “who is john cleese”}, “context”: {“ident”: “1541070853.3456345137989080”, “client_name”: “mycroft_listener”}, “type”: “intent_failure”}
12:14:15.567 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): api.mycroft.ai:443
12:14:15.573 - mycroft.skills.padatious_service:handle_fallback:133 - DEBUG - Padatious fallback attempt: who is john cleese
12:14:15.584 - SKILLS - DEBUG - {“data”: {“handler”: “fallback”}, “context”: {“ident”: “1541070853.3456345137989080”, “client_name”: “mycroft_listener”}, “type”: “mycroft.skill.handler.start”}
12:14:15.609 - urllib3.connectionpool - DEBUG - Starting new HTTP connection (1): duckduckgo.com:80
12:14:15.735 - urllib3.connectionpool - DEBUG - http://duckduckgo.com:80 “GET /?o=x&q=john+cleese HTTP/1.1” 200 None
12:14:15.744 - mycroft-fallback-duck-duck-go_mycroftai:respond:120 - DEBUG - Query: john cleese
12:14:15.747 - mycroft-fallback-duck-duck-go_mycroftai:respond:121 - DEBUG - Type: answer
12:14:15.770 - SKILLS - DEBUG - {“data”: {“expect_response”: false, “utterance”: “John Marwood Cleese is an English actor, voice actor, comedian, screenwriter, and producer”}, “context”: {“ident”: “1541070853.3456345137989080”, “client_name”: “mycroft_listener”}, “type”: “speak”}
12:14:15.773 - SKILLS - DEBUG - {“data”: {“fallback_handler”: “DuckduckgoSkill.respond_to_question”, “handler”: “fallback”}, “context”: {“ident”: “1541070853.3456345137989080”, “client_name”: “mycroft_listener”}, “type”: “mycroft.skill.handler.complete”}
12:14:15.787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): api.mycroft.ai:443
12:14:15.804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): api.mycroft.ai:443
12:14:15.805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): api.mycroft.ai:443
12:14:16.114 - urllib3.connectionpool - DEBUG - https://api.mycroft.ai:443 “POST /v1/device/3c44026f-9c7f-490e-9147-8404b6c9e94f/metric/timing HTTP/1.1” 200 278
12:14:16.317 - urllib3.connectionpool - DEBUG - https://api.mycroft.ai:443 “POST /v1/device/3c44026f-9c7f-490e-9147-8404b6c9e94f/metric/timing HTTP/1.1” 200 266
12:14:16.330 - urllib3.connectionpool - DEBUG - https://api.mycroft.ai:443 “POST /v1/device/3c44026f-9c7f-490e-9147-8404b6c9e94f/metric/timing HTTP/1.1” 200 296
12:14:16.349 - urllib3.connectionpool - DEBUG - https://api.mycroft.ai:443 “POST /v1/device/3c44026f-9c7f-490e-9147-8404b6c9e94f/metric/timing HTTP/1.1” 200 265
12:14:16.397 - SKILLS - DEBUG - {“type”: “recognizer_loop:audio_output_start”, “context”: null, “data”: {}}
12:14:20.008 - SKILLS - DEBUG - {“data”: {}, “context”: null, “type”: “mycroft-date-time.mycroftai:TimeSkillupdate_display”}
12:14:21.108 - SKILLS - DEBUG - {“type”: “recognizer_loop:audio_output_end”, “context”: null, “data”: {}}
12:14:23.015 - SKILLS - DEBUG - {“data”: {}, “context”: null, “type”: “mycroft-reminder.mycroftai:reminder”}
12:14:25.603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): api.mycroft.ai:443
12:14:26.097 - urllib3.connectionpool - DEBUG - https://api.mycroft.ai:443 “POST /v1/device/3c44026f-9c7f-490e-9147-8404b6c9e94f/metric/timing HTTP/1.1” 200 261
12:14:28.273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): api.mycroft.ai:443
12:14:28.816 - urllib3.connectionpool - DEBUG - https://api.mycroft.ai:443 “GET /v1/device/3c44026f-9c7f-490e-9147-8404b6c9e94f/skill HTTP/1.1” 304 0
12:14:30.021 - SKILLS - DEBUG - {“data”: {}, “context”: null, “type”: “mycroft-date-time.mycroftai:TimeSkillupdate_display”}
12:14:32.401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): api.mycroft.ai:443
12:14:32.878 - urllib3.connectionpool - DEBUG - https://api.mycroft.ai:443 “GET /v1/device/3c44026f-9c7f-490e-9147-8404b6c9e94f/skill HTTP/1.1” 304 0
12:14:33.313 - SKILLS - DEBUG - {“context”: null, “data”: {}, “type”: “recognizer_loop:record_begin”}
12:14:33.328 - AlarmSkill - INFO - on started…
12:14:35.032 - SKILLS - DEBUG - {“data”: “UpdateRemote”, “context”: null, “type”: “mycroft-configuration.mycroftai:ConfigurationSkillupdate_remote”}
12:14:35.050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): api.mycroft.ai:443
12:14:35.204 - SKILLS - DEBUG - {“context”: null, “data”: {}, “type”: “recognizer_loop:record_end”}
12:14:35.208 - AlarmSkill - INFO - on ended…
12:14:35.218 - SKILLS - DEBUG - {“context”: null, “data”: {“utterance”: “hey mycroft”, “session”: “0d26c4f9-9fda-40c5-ac9c-59679d64e6f3”}, “type”: “recognizer_loop:wakeword”}
12:14:35.522 - urllib3.connectionpool - DEBUG - https://api.mycroft.ai:443 “GET /v1/device/3c44026f-9c7f-490e-9147-8404b6c9e94f/setting HTTP/1.1” 304 0
12:14:35.538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): api.mycroft.ai:443
12:14:35.589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): api.mycroft.ai:443
12:14:36.021 - urllib3.connectionpool - DEBUG - https://api.mycroft.ai:443 “GET /v1/device/3c44026f-9c7f-490e-9147-8404b6c9e94f/location HTTP/1.1” 304 0
12:14:36.099 - urllib3.connectionpool - DEBUG - https://api.mycroft.ai:443 “GET /v1/device/3c44026f-9c7f-490e-9147-8404b6c9e94f/skill HTTP/1.1” 304 0
12:14:36.560 - SKILLS - DEBUG - {“context”: {“ident”: “1541070875.205196-974118156”, “client_name”: “mycroft_listener”}, “data”: {“lang”: “en-US”, “utterances”: [“what’s the weather”], “session”: “0d26c4f9-9fda-40c5-ac9c-59679d64e6f3”}, “type”: “recognizer_loop:utterance”}
12:14:36.599 - SKILLS - DEBUG - {“data”: {“target”: null, “tags”: [{“start_token”: 2, “key”: “weather”, “entities”: [{“data”: [[“weather”, “mycroft_weather_mycroftaiWeather”]], “confidence”: 1.0, “key”: “weather”, “match”: “weather”}], “end_token”: 2, “from_context”: false, “match”: “weather”}], “intent_type”: “mycroft-weather.mycroftai:handle_current_weather”, “mycroft_weather_mycroftaiWeather”: “weather”, “utterance”: “what’s the weather”, “confidence”: 0.5}, “context”: {“target”: null, “ident”: “1541070875.205196-974118156”, “client_name”: “mycroft_listener”}, “type”: “mycroft-weather.mycroftai:handle_current_weather”}
12:14:36.609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): api.mycroft.ai:443
12:14:36.623 - SKILLS - DEBUG - {“data”: {“name”: “WeatherSkill.handle_current_weather”}, “context”: {“target”: null, “ident”: “1541070875.205196-974118156”, “client_name”: “mycroft_listener”}, “type”: “mycroft.skill.handler.start”}
12:14:36.641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): api.mycroft.ai:443
12:14:37.106 - urllib3.connectionpool - DEBUG - https://api.mycroft.ai:443 “POST /v1/device/3c44026f-9c7f-490e-9147-8404b6c9e94f/metric/timing HTTP/1.1” 200 299
12:14:37.279 - urllib3.connectionpool - DEBUG - https://api.mycroft.ai:443 “GET /v1/owm/weather?lang=en&lon=-4.097899&lat=47.997542 HTTP/1.1” 200 438
12:14:37.299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): api.mycroft.ai:443
12:14:37.930 - urllib3.connectionpool - DEBUG - https://api.mycroft.ai:443 “GET /v1/owm/forecast/daily?lang=en&lon=-4.097899&lat=47.997542 HTTP/1.1” 200 2002
12:14:37.948 - SKILLS - DEBUG - {“data”: {}, “context”: null, “type”: “enclosure.mouth.events.deactivate”}
12:14:37.958 - SKILLS - DEBUG - {“data”: {“img_code”: 0, “temp”: “11”}, “context”: null, “type”: “enclosure.weather.display”}
12:14:37.963 - SKILLS - DEBUG - {“data”: {“expect_response”: false, “utterance”: “It’s currently clear sky and 11 degrees. Today’s forecast is for a high of 12 and a low of 8.”}, “context”: {“target”: null, “ident”: “1541070875.205196-974118156”, “client_name”: “mycroft_listener”}, “type”: “speak”}
12:14:37.991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): api.mycroft.ai:443
12:14:38.513 - urllib3.connectionpool - DEBUG - https://api.mycroft.ai:443 “POST /v1/device/3c44026f-9c7f-490e-9147-8404b6c9e94f/metric/timing HTTP/1.1” 200 263
12:14:38.617 - SKILLS - DEBUG - {“type”: “enclosure.eyes.blink”, “context”: null, “data”: {“side”: “b”}}
12:14:38.624 - SKILLS - DEBUG - {“type”: “recognizer_loop:audio_output_start”, “context”: null, “data”: {}}
12:14:40.055 - SKILLS - DEBUG - {“data”: {}, “context”: null, “type”: “mycroft-date-time.mycroftai:TimeSkillupdate_display”}

What do you think of that ?

2

Hi there @henridbr, I’m trying to better understand the problem here.

Are you trying to use Conversation Context, and are you expecting that the response will be captured without the Wake Word?

For example:

You are running version 18.8.2 of Mycroft Core.
Would you like to upgrade to the latest version?
(no Wake Word required)

“Yes”

If this is the type of conversation you’re trying to achieve, then yes Conversation Context is the way to go.

You may also need the speak() method with get_response set to True
https://mycroft-core.readthedocs.io/en/stable/source/mycroft.html#mycroft.MycroftSkill.speak

I’m not sure if this helps at all?

3

Thank you @KathyReid
My question is about the use of the wake word.
The example from the doc shown above doesn’t use it between the first answer and the second question.
When discussing it’s natural to get someone’s attention by a call but if you have to do that every time you have a problem with this person.
Once Mycroft has answered to the first question I suppose that it is waiting for a new one without the need of the wake word.
Context maintains the utterance inside the skill. It’s not the point. It seems that Mycroft needs the wake word between each question too.
In other words, if the doc retranscribes a true discussion with Mycroft, I have to search why my Picroft works a different way.

4

you dont want mycroft to listen 24/7 and send all audio for STT processing

the wake word ensures it only listens when you want, i get the idea behind listening all the time after first interaction, i believe @steve.penrod tried this in a previous project and the results were not great, since very often you make a single question this means you will get a bunch of fake activations

if you know for sure you are are expecting the conversation to continue you can trigger listening, either by sending a bus message “mycroft.mic.listen” or if you just spoke something you can pass expect_response = True in the self.speak() method

other options for continuous dialog are using the get_response method, this one will ensure mycroft tries to listen up to 5 times (or was it 3?)

an example i can give you is the cave adventure skill, the interaction does not play out great with it always listening (still working on that one), sometimes the user needs some time to think before answering right away and the experience is not smooth

I will play a bit with continuous listening like you described to see how the user experience is, but i do not think this should be a default setting for privacy reasons

i think the responsability to trigger automatic listening falls to the skill designer in a case by case basis

5

minor correction, speak method takes expect_response = True to start listening after speech ends

get_response is a method to retrieve the user utterance as text inside an intent

6

Thank you @Jarbas_Ai for these details I have to study and for all your work and good advises you give to us. Thank you @KathyReid for your patience. I am falling in all the road holes under my feet.

  1. Discussion transcription

This is how conversations are transcribed in the doc:

User: Where is John Cleese from?
Mycroft: He’s from England
User: What’s the weather like over there?
Mycroft: Raining and 14 degrees…

There is no sign to show that wake word or button press which activate the recording before user’s question are omitted. My English accent is far from good and often I am using the mycroft-client-cli to avoid “I don’t understand what you say”. In this case History records are equivalent to usual transcriptions in the doc. Once this admitted, it’s not an issue for me. It’s the problem of who has written the doc and his/her objectives, but doc might be true though not absolute truth.

  1. Delays

I am using Picroft. I think my perception is amplified by the time spent between the wake word recognition and the recording process. Looking at the log above, it seems about 2 seconds at least. It’s a very long time when speaking.
Looking at some videos I don’t see actors waiting so long between wake word and question. They say “Hey Mycroft, what time is it” whereas I have to say “Hey Mycroft” [wait 2 seconds] “what time is it”.
Perhaps it is the difference between Raspberry with Picroft and Mark I or Mark II ?