Skill-weather fails with Python error after HTTPS GET (FIXED)

FloatingBoater · August 24, 2017, 5:26pm

Hi,

After updating my Mark 1 to mycroft-core 0.8.21 (with nightly updates), the skill-weather has stopped working.

As a greybeard former Unix sysadmin and software engineer, I have two goals here:

Help report and trace a bug
Learn and document the debug process - perhaps something to add to a FAQ?

After a look around resource, I’m unsure what the best workflow is for skill bugs, so hope someone can point me in the right direction.

Thanks!

Issue Description

After a recent update, the weather skill has stopped working. The utterance is recognised, but Mycroft does not reply.
###Environment

Mycroft Mark 1
mycroft-core 0.8.21
Nightly cron job to ‘apt-get update’ active

Steps to Reproduce

Say ‘Hey Mycroft, What’s the weather?’
Say ‘Hey Mycroft, What’s the weather in Austin?’
###Expected result
A spoken weather forecast for my local area, as pre-configured some time ago.
Weather symbols on the LED matrix
###Actual Result
Mouth changes to listening
After a few seconds, the mouth goes blank (or back to the time skill clock display)
No speech, no response.
##Investigation
After some poking around the forums, I found how to check the Mark 1 software version, and fixed the auto-update roof cron job. I also found the main log files, and noticed errors appeared about the same time as the main skill-weather Python script timestamp.
I know a little of GitHub from FOSS tinkering, but don’t know the preferred workflow to report / diagnose skill issues.

Some basic checks:

pi@mark_1:~ $ apt list --installed |grep mycroft-core

WARNING: apt does not have a stable CLI interface yet. Use with caution in scripts.

mycroft-core/unknown,now 0.8.21 armhf [installed]
pi@mark_1:~ $ ls -la /opt/mycroft/skills/skill-weather
total 96
drwxr-xr-x  7 mycroft mycroft  4096 Aug 24 16:22 .
drwxrwxrwx 30 mycroft mycroft  4096 Aug 21 16:37 ..
drwxr-xr-x  3 mycroft mycroft  4096 May 17 01:30 dialog
drwxr-xr-x  8 mycroft mycroft  4096 Aug 24 16:22 .git
-rw-r--r--  1 mycroft mycroft 21875 Aug 21 14:04 __init__.py
-rw-r--r--  1 mycroft mycroft 16685 Aug 24 16:22 __init__.pyc
-rw-r--r--  1 mycroft mycroft  9141 Aug 21 14:04 LICENSE.md
-rw-r--r--  1 mycroft mycroft   125 May 17 01:30 README.md
drwxr-xr-x  3 mycroft mycroft  4096 May 17 01:30 regex
-rw-r--r--  1 mycroft mycroft    52 May 17 01:30 requirements.txt
-rw-r--r--  1 mycroft mycroft     2 Aug 24 16:22 settings.json
drwxr-xr-x  3 mycroft mycroft  4096 May 17 01:30 test
drwxr-xr-x  3 mycroft mycroft  4096 May 17 01:30 vocab

Mycroft logs appears to live in /var/log, and I found the test console which also reproduces the issue:

python -m mycroft.client.text.main --quiet
tail -f /var/log/mycroft-*
vim /var/log/mycroft-skills.log

So, tracing the skill-weather failure gives an error mesage:

2017-08-24 16:03:58,236 - Skills - DEBUG - {"type": "recognizer_loop:utterance", "data": {"lang": "en-US", "session": "3c447d2c-1974-48a9-9625-c6c61f4ad80e", "utterances": ["what's the weather"]}, "context": null}
2017-08-24 16:03:58,280 - Skills - DEBUG - {"type": "WeatherSkill:CurrentWeatherIntent", "data": {"confidence": 0.5, "target": null, "intent_type": "WeatherSkill:CurrentWeatherIntent", "Weather": "weather", "__tags__": [{"end_token": 2, "start_token": 2, "from_context": false, "entities": [{"confidence": 1.0, "data": [["weather", "Weather"], ["Weather", "Concept"]], "key": "weather", "match": "weather"}], "key": "weather", "match": "weather"}], "utterance": "what's the weather"}, "context": {"target": null}}
2017-08-24 16:03:58,287 - requests.packages.urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): api.mycroft.ai
2017-08-24 16:03:58,763 - requests.packages.urllib3.connectionpool - DEBUG - https://api.mycroft.ai:443 "GET /v1/owm/weather?q=Cramlington%2C+England%2C+United+Kingdom&lang=en HTTP/1.1" 200 451
2017-08-24 16:03:58,778 - requests.packages.urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): api.mycroft.ai
2017-08-24 16:03:59,226 - requests.packages.urllib3.connectionpool - DEBUG - https://api.mycroft.ai:443 "GET /v1/owm/forecast/daily?q=Cramlington%2C+England%2C+United+Kingdom&lang=en HTTP/1.1" 200 1978
2017-08-24 16:03:59,237 - skill-weather__init__ - ERROR - Error: 'NoneType' object has no attribute 'to_JSON'
2017-08-24 16:04:05,698 - Display Manager (mycroft.client.enclosure) - DEBUG - Setting active skill to TimeSkill

This suggests to me either the HTTPS call is failing, and/or the skill isn’t catching the return data.
I have seen some reports of issues with web configuration of custom locations (I’ve used the Mycroft.ai WWW settings to setup Cramlington, England), so I also tried an explicit US location picked from other register_vocab calls in the log - Austin:
> 2017-08-24 16:51:04,747 - Skills - DEBUG - {“type”: “WeatherSkill:CurrentWeatherIntent”, “data”: {“confidence”: 0.6666666666666666, “target”: null, “intent_type”: “WeatherSkill:CurrentWeatherIntent”, “Weather”: “weather”, “Location”: “austin”, “tags”: [{“end_token”: 2, “start_token”: 2, “from_context”: false, “entities”: [{“confidence”: 1.0, “data”: [[“weather”, “Weather”], [“Weather”, “Concept”]], “key”: “weather”, “match”: “weather”}], “key”: “weather”, “match”: “weather”}, {“end_token”: 4, “start_token”: 4, “confidence”: 1.0, “from_context”: false, “entities”: [{“confidence”: 1.0, “data”: [[“austin”, “Location”]], “key”: “austin”, “match”: “austin”}], “key”: “austin”, “match”: “austin”}], “utterance”: “what’s the weather in austin”}, “context”: {“target”: null}}
> 2017-08-24 16:51:04,753 - requests.packages.urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): api.mycroft.ai
> 2017-08-24 16:51:05,287 - requests.packages.urllib3.connectionpool - DEBUG - https://api.mycroft.ai:443 “GET /v1/owm/weather?q=austin&lang=en HTTP/1.1” 200 444
> 2017-08-24 16:51:05,302 - requests.packages.urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): api.mycroft.ai
> 2017-08-24 16:51:05,763 - requests.packages.urllib3.connectionpool - DEBUG - https://api.mycroft.ai:443 “GET /v1/owm/forecast/daily?q=austin&lang=en HTTP/1.1” 200 1989
> 2017-08-24 16:51:05,775 - skill-weather__init__ - ERROR - Error: ‘NoneType’ object has no attribute ‘to_JSON’

So - pre-configured, or explicit locations give the same 'ERROR - Error: ‘NoneType’ object has no attribute ‘to_JSON’ in what is probably

I don’t know if the installed skill code is the latest version in GitHub, but I think the error is thrown in ‘/opt/mycroft/skills/skill-weather/init.py’:

My thought is to backup the skill files and then start adding LOG.info() calls into the Python code to get a feel for where the error is occurring, but don’t see other text in ‘/var/log/mycroft-skills.log’ suggesting there may be a log level setting or another file.

Any ideas please?

Mn0491 · August 24, 2017, 6:37pm

Hello @FloatingBoater, this is a very comprehensive post! Thank you for putting your time into that.

What seems to be happening is that the function _get_forecast is returning a NONE type object. https://github.com/MycroftAI/skill-weather/blob/master/init.py#L491. something must be failing in that skill for it to return a NONE type.

All LOG’s in skills will be logged out to the /var/log/mycroft-skills.log.

As far as the best workflow for reporting bugs, you can do that in the issues tab of the github repo. https://github.com/MycroftAI/skill-weather/issues.

Hope this helps!

FloatingBoater · August 25, 2017, 11:17am

Hi @Mn0491,

Thanks for your help - I’ve registered with GitHub and reported the issue, and was pleased to find the same MarkDown formatting works:

github.com/MycroftAI/skill-weather

Skill-weather fails with Python error after HTTPS GET

opened 08:45AM - 25 Aug 17 UTC

closed 06:22PM - 29 Aug 17 UTC

FloatingBoater

Original linked here: https://community.mycroft.ai/t/skill-weather-fails-with-p…ython-error-after-https-get/2070 Hi, After updating my Mark 1 to mycroft-core 0.8.21 (with nightly updates), the skill-weather has stopped working. As a greybeard former Unix sysadmin and software engineer, I have two goals here: 1. Help report and trace a bug 2. Learn and document the debug process - perhaps something to add to a FAQ? After a look around resource, I'm unsure what the best workflow is for skill bugs, so hope someone can point me in the right direction. Thanks! ## Issue Description After a recent update, the weather skill has stopped working. The utterance is recognised, but Mycroft does not reply. ### Environment 1. Mycroft Mark 1 1. mycroft-core 0.8.21 1. Nightly cron job to '`apt-get update`' active ### Steps to Reproduce 1. Say 'Hey Mycroft, What's the weather?' 1. Say 'Hey Mycroft, What's the weather in Austin?' ### Expected result 1. A spoken weather forecast for my local area, as pre-configured some time ago. 1. Weather symbols on the LED matrix ### Actual Result 1. Mouth changes to listening 1. After a few seconds, the mouth goes blank (or back to the time skill clock display) 3. No speech, no response. ## Investigation After some poking around the forums, I found how to check the Mark 1 software version, and fixed the auto-update roof cron job. I also found the main log files, and noticed errors appeared about the same time as the main skill-weather Python script timestamp. I know a little of GitHub from FOSS tinkering, but don't know the preferred workflow to report / diagnose skill issues. Some basic checks: > pi@mark_1:~ $ apt list --installed |grep mycroft-core > > WARNING: apt does not have a stable CLI interface yet. Use with caution in scripts. > > mycroft-core/unknown,now 0.8.21 armhf [installed] > pi@mark_1:~ $ ls -la /opt/mycroft/skills/skill-weather > total 96 > drwxr-xr-x 7 mycroft mycroft 4096 Aug 24 16:22 . > drwxrwxrwx 30 mycroft mycroft 4096 Aug 21 16:37 .. > drwxr-xr-x 3 mycroft mycroft 4096 May 17 01:30 dialog > drwxr-xr-x 8 mycroft mycroft 4096 Aug 24 16:22 .git > -rw-r--r-- 1 mycroft mycroft 21875 Aug 21 14:04 __init__.py > -rw-r--r-- 1 mycroft mycroft 16685 Aug 24 16:22 __init__.pyc > -rw-r--r-- 1 mycroft mycroft 9141 Aug 21 14:04 LICENSE.md > -rw-r--r-- 1 mycroft mycroft 125 May 17 01:30 README.md > drwxr-xr-x 3 mycroft mycroft 4096 May 17 01:30 regex > -rw-r--r-- 1 mycroft mycroft 52 May 17 01:30 requirements.txt > -rw-r--r-- 1 mycroft mycroft 2 Aug 24 16:22 settings.json > drwxr-xr-x 3 mycroft mycroft 4096 May 17 01:30 test > drwxr-xr-x 3 mycroft mycroft 4096 May 17 01:30 vocab Mycroft logs appears to live in /var/log, and I found the test console which also reproduces the issue: > python -m mycroft.client.text.main --quiet > tail -f /var/log/mycroft-* > vim /var/log/mycroft-skills.log So, tracing the skill-weather failure gives an error mesage: > 2017-08-24 16:03:58,236 - Skills - DEBUG - {"type": "recognizer_loop:utterance", "data": {"lang": "en-US", "session": "3c447d2c-1974-48a9-9625-c6c61f4ad80e", "utterances": ["what's the weather"]}, "context": null} > 2017-08-24 16:03:58,280 - Skills - DEBUG - {"type": "WeatherSkill:CurrentWeatherIntent", "data": {"confidence": 0.5, "target": null, "intent_type": "WeatherSkill:CurrentWeatherIntent", "Weather": "weather", "__tags__": [{"end_token": 2, "start_token": 2, "from_context": false, "entities": [{"confidence": 1.0, "data": [["weather", "Weather"], ["Weather", "Concept"]], "key": "weather", "match": "weather"}], "key": "weather", "match": "weather"}], "utterance": "what's the weather"}, "context": {"target": null}} > 2017-08-24 16:03:58,287 - requests.packages.urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): api.mycroft.ai > 2017-08-24 16:03:58,763 - requests.packages.urllib3.connectionpool - DEBUG - https://api.mycroft.ai:443 "GET /v1/owm/weather?q=Cramlington%2C+England%2C+United+Kingdom&lang=en HTTP/1.1" 200 451 > 2017-08-24 16:03:58,778 - requests.packages.urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): api.mycroft.ai > 2017-08-24 16:03:59,226 - requests.packages.urllib3.connectionpool - DEBUG - https://api.mycroft.ai:443 "GET /v1/owm/forecast/daily?q=Cramlington%2C+England%2C+United+Kingdom&lang=en HTTP/1.1" 200 1978 > 2017-08-24 16:03:59,237 - skill-weather__init__ - ERROR - Error: 'NoneType' object has no attribute 'to_JSON' > 2017-08-24 16:04:05,698 - Display Manager (mycroft.client.enclosure) - DEBUG - Setting active skill to TimeSkill This suggests to me either the HTTPS call is failing, and/or the skill isn't catching the return data. I have seen some reports of issues with web configuration of custom locations (I've used the Mycroft.ai WWW settings to setup Cramlington, England), so I also tried an explicit US location picked from other register_vocab calls in the log - Austin: > 2017-08-24 16:51:04,747 - Skills - DEBUG - {"type": "WeatherSkill:CurrentWeatherIntent", "data": {"confidence": 0.6666666666666666, "target": null, "intent_type": "WeatherSkill:CurrentWeatherIntent", "Weather": "weather", "Location": "austin", "__tags__": [{"end_token": 2, "start_token": 2, "from_context": false, "entities": [{"confidence": 1.0, "data": [["weather", "Weather"], ["Weather", "Concept"]], "key": "weather", "match": "weather"}], "key": "weather", "match": "weather"}, {"end_token": 4, "start_token": 4, "confidence": 1.0, "from_context": false, "entities": [{"confidence": 1.0, "data": [["austin", "Location"]], "key": "austin", "match": "austin"}], "key": "austin", "match": "austin"}], "utterance": "what's the weather in austin"}, "context": {"target": null}} > 2017-08-24 16:51:04,753 - requests.packages.urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): api.mycroft.ai > 2017-08-24 16:51:05,287 - requests.packages.urllib3.connectionpool - DEBUG - https://api.mycroft.ai:443 "GET /v1/owm/weather?q=austin&lang=en HTTP/1.1" 200 444 > 2017-08-24 16:51:05,302 - requests.packages.urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): api.mycroft.ai > 2017-08-24 16:51:05,763 - requests.packages.urllib3.connectionpool - DEBUG - https://api.mycroft.ai:443 "GET /v1/owm/forecast/daily?q=austin&lang=en HTTP/1.1" 200 1989 > 2017-08-24 16:51:05,775 - skill-weather__init__ - ERROR - Error: 'NoneType' object has no attribute 'to_JSON' So - pre-configured, or explicit locations give the same 'ERROR - Error: 'NoneType' object has no attribute 'to_JSON' in what is probably I don't know if the installed skill code is the latest version in GitHub, but I think the error is thrown in '/opt/mycroft/skills/skill-weather/__init__.py': https://github.com/MycroftAI/skill-weather https://github.com/MycroftAI/skill-weather/tree/cd782186df0cb8bb51bc803f7253a3b3a9f4caaf My thought is to backup the skill files and then start adding LOG.info() calls into the Python code to get a feel for where the error is occurring, but don't see other text in '/var/log/mycroft-skills.log' suggesting there may be a log level setting or another file. Any ideas please?

Interestingly, “will it rain today” gets forecasts, “what is the weather” fails.

Top tip:
watch the skill log whist testing, copy out interesting JSON and dump it into a file, and use jq to format it for human viewing:

tail -f /var/log/mycroft-skills.log
# to understand JSON data
vim json.txt
cat json.txt | jq '.'
# to understand Unix epoch date stamps (10 digits, or if 13 delete the last three)
date -d @1503662400

All the best!

Barney_Woodrow · August 26, 2017, 2:46pm

Has this been solved yet? I get this too.

Mn0491 · August 28, 2017, 3:28pm

I think it may have been solved. If you check out the issue @FloatingBoater filed, there seems to be a merged that could have resolved this issue. https://github.com/MycroftAI/skill-weather/issues/7#ref-pullrequest-253317921

FloatingBoater · August 29, 2017, 6:25pm

Hi,

I can confirm that an update over the weekend fixed this issue (see the detailed log in the GitHub skill-weather linked issue above).

As a bonus, the TTS reply gives Celsius temperatures, presumably as my account is set to Metric units (“systemUnit”: “metric”).

Thanks for all your help!