Mark 1 STT fails with 500 errors, skills work manually via python console - FIXED

Archive - Mycroft Project This category is being retired - Support
1

Hi,

My Mark 1 responds normally to wake words, however nothing happens - e.g. talk beep/ mouth animation, records, but no response from any skills.

If I bypass STT by manually typing a request into the Python console, skills work AOK.

After some digging in the logs, it looks like the speech recording WAV is being uploaded, but the STT is failing somehow with what looks like a HTTP 500 error:

tail -f /var/log/mycroft-speech-client.log

12:31:32.495 - mycroft.client.speech.mic:listen:515 - DEBUG - Recording...
12:31:32.497 - mycroft.client.speech.mic:_upload_file:344 - DEBUG - Uploading /tmp/mycroft_wake_words/hey-mycroft.cc65c58d358ea17c4ced03054e2d70c6.1517488292489.80553ce6-c585-4ad7-98aa-5fc800e5414f.123d8aaa-e6a7-49ae-8ddf-39d640c56535.0.wav...
12:31:32.504 - mycroft.client.speech.main:handle_record_begin:35 - INFO - Begin Recording...
12:31:34.621 - mycroft.client.speech.main:handle_record_end:40 - INFO - End Recording...
12:31:34.669 - mycroft.client.speech.main:handle_wakeword:56 - INFO - Wakeword Detected: hey mycroft
12:31:34.724 - requests.packages.urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): api.mycroft.ai
12:31:35.349 - requests.packages.urllib3.connectionpool - DEBUG - https://api.mycroft.ai:443 "POST /v1/stt?lang=en-US&limit=1 HTTP/1.1" 500 35
12:31:35.403 - requests.packages.urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): api.mycroft.ai
12:31:35.558 - mycroft.client.speech.mic:listen:510 - DEBUG - Waiting for wake word...
12:31:36.047 - requests.packages.urllib3.connectionpool - DEBUG - https://api.mycroft.ai:443 "POST /v1/stt?lang=en-US&limit=1 HTTP/1.1" 500 35
12:31:36.062 - requests.packages.urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): api.mycroft.ai
12:31:36.561 - requests.packages.urllib3.connectionpool - DEBUG - https://api.mycroft.ai:443 "POST /v1/device/9b3a9118-fcea-4c24-81d9-ab4254421bbc/metric/timing HTTP/1.1" 200 239

I’m not certain but it looks like config update HTTPS calls work (200), but all STT fails (500).

I have recently updated both core and skills:

sudo apt-get update && sudo apt-get install mycroft-core
sudo msm update

Any ideas (other than re-flash the SD card - would prefer to learn, not just format) please?

Thanks,

James

2

My guess this was a server issue - Speech To Text (STT) is now working. No restart needed, just time.

For reference, here’s a successful log - note the connectionpool - DEBUG lines end with HTTP status codes of 200 rather than 500.

tail -f /var/log/mycroft-speech-client.log

17:23:45.319 - mycroft.client.speech.mic:listen:515 - DEBUG - Recording...
17:23:45.321 - mycroft.client.speech.mic:_upload_file:344 - DEBUG - Uploading /tmp/mycroft_wake_words/hey-mycroft.cc65c58d358ea17c4ced03054e2d70c6.1517505825310.2045076b-1c54-4a6f-a422-f45133f95a54.123d8aaa-e6a7-49ae-8ddf-39d640c56535.0.wav...
17:23:45.328 - mycroft.client.speech.main:handle_record_begin:35 - INFO - Begin Recording...
17:23:47.418 - mycroft.client.speech.main:handle_record_end:40 - INFO - End Recording...
17:23:47.427 - mycroft.client.speech.main:handle_wakeword:56 - INFO - Wakeword Detected: hey mycroft
17:23:47.526 - requests.packages.urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): api.mycroft.ai
17:23:48.332 - requests.packages.urllib3.connectionpool - DEBUG - https://api.mycroft.ai:443 "POST /v1/stt?lang=en-US&limit=1 HTTP/1.1" 200 None
17:23:48.372 - mycroft.client.speech.mic:listen:510 - DEBUG - Waiting for wake word...
17:23:49.075 - mycroft.client.speech.listener:transcribe:165 - DEBUG - STT: what's the weather
17:23:49.078 - mycroft.client.speech.main:handle_utterance:61 - INFO - Utterance: [u"what's the weather"]
3

Thanks so much FloatingBoater - this was definitely an STT issue on our end.
You can see our open and transparent Incident Report at:

I will add your logs to the Incident Report for future reference, thank you for posting them.
Best, Kathy

1 Like
4

Excellent - I’m rather impressed with the root cause analysis doc.

It’s well up the standard of ITIL-based service management used in major multinationals :clap:

In case a similar error ever happens again, I’ve raised an enhancement request:

Thanks Kathy!

1 Like
5

Thanks so much for that feedback @FloatingBoater, we have a very multi-talented team here. I have several ITIL qualifications, and used to lead root cause analysis for a large college. Know nothing about Tensorflow though, so that’s where our amazing dev team, headed by Top Bloke, @steve.penrod, come in :slight_smile: