Debug error on Ubuntu 18.10 desktop

Abdulrahman2 · November 18, 2018, 8:51am

Hi. I am trying to setup Mycroft on an Ubuntu 18.10 desktop. The setup seems OK. However, when I run the debug command I get the following error:

~/mycroft-core$ ./start-mycroft.sh debug
Starting all mycroft-core services
Initializing…
Starting background service bus
CAUTION: The Mycroft bus is an open websocket with no built-in security
measures. You are responsible for protecting the local port
8181 with a firewall as appropriate.
Starting background service skills
Starting background service audio
Starting background service voice
Starting cli
Traceback (most recent call last):
File “/usr/lib/python3.6/runpy.py”, line 183, in _run_module_as_main
mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
File “/usr/lib/python3.6/runpy.py”, line 109, in _get_module_details
import(pkg_name)
File “/home/aia/mycroft-core/mycroft/init.py”, line 17, in
from mycroft.api import Api
File “/home/aia/mycroft-core/mycroft/api/init.py”, line 18, in
import requests
ModuleNotFoundError: No module named ‘requests’

I tried re-setup Mycroft multiple times and I get the same error every time. My network is behind a firewall and I used the export command to setup the http_proxy and other variables. So, I think the problem may not be related to the firewall. Also, the ‘requests’ module is installed in my desktop.

Any suggestion will be appreciated.

Thank,

Abdulrahman

KathyReid · November 19, 2018, 1:45am

Hi there @Abdulrahman2, thanks for posting this error.

There are a couple of suggestions I have;

The requests package in mycroft-core has recently been updated to the newest version - 2.20.0 from memory - after a vulnerability was found in the previous version. Does Ubuntu 18.10 have the newest version of requests? You can find this using the pip list command:

kathyreid@kathyreid-Oryx-Pro:~/mycroft-core$ pip list | grep requests
requests                     2.18.4

You can see here that my system has 2.18.4 installed, but mycroft-core now requires the new version, 2.20.0. We manage mycroft-core using a virtual environment - venv - so we can install requests using the venv.

kathyreid@kathyreid-Oryx-Pro:~/mycroft-core$ source venv-activate.sh 
Entering Mycroft virtual environment.  Run 'mycroft-venv-deactivate' to exit
(.venv) kathyreid@kathyreid-Oryx-Pro:~/mycroft-core$ pip list | grep requests
requests                 2.20.0    
requests-futures         0.9.5     
You are using pip version 18.0, however version 18.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
(.venv) kathyreid@kathyreid-Oryx-Pro:~/mycroft-core$ pip install requests
Requirement already satisfied: requests in ./.venv/lib/python3.6/site-packages (2.20.0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in ./.venv/lib/python3.6/site-packages (from requests) (3.0.4)
Requirement already satisfied: idna<2.8,>=2.5 in ./.venv/lib/python3.6/site-packages (from requests) (2.7)
Requirement already satisfied: urllib3<1.25,>=1.21.1 in ./.venv/lib/python3.6/site-packages (from requests) (1.23)
Requirement already satisfied: certifi>=2017.4.17 in ./.venv/lib/python3.6/site-packages (from requests) (2018.10.15)

Here, you can see that the mycroft-core venv already has the right requests installed.

What I would recommend is first activating the venv - as shown above - then doing pip install requests

Please let us know how you go.

Abdulrahman2 · November 19, 2018, 8:55am

Thank you KathyReid for your suggestions.

It seems that I have already the newest version of the ‘requests’ package installed. I have also followed your instruction to re-install the package.

However, the error remains the same when I run the debug command.

Regards,

Abdulrahman

KathyReid · November 19, 2018, 9:35am

Can you please post the exact commands you used and the exact output?

Abdulrahman2 · November 19, 2018, 12:31pm

Sure.

Command 1:

abdulrahman@abdulrahman-OptiPlex-990:~$ pip list | grep requests
DEPRECATION: The default format will switch to columns in the future. You can use --format=(legacy|columns) (or define a format=(legacy|columns) in your pip.conf under the [list] section) to disable this warning.
requests (2.20.1)
requests-futures (0.9.5)
abdulrahman@abdulrahman-OptiPlex-990:~$

Command 2:

abdulrahman@abdulrahman-OptiPlex-990:~/mycroft-core$ source venv-activate.sh
Entering Mycroft virtual environment. Run ‘mycroft-venv-deactivate’ to exit
(.venv) abdulrahman@abdulrahman-OptiPlex-990:~/mycroft-core$ pip list | grep requests
DEPRECATION: The default format will switch to columns in the future. You can use --format=(legacy|columns) (or define a format=(legacy|columns) in your pip.conf under the [list] section) to disable this warning.
requests (2.20.1)
requests-futures (0.9.5)
(.venv) abdulrahman@abdulrahman-OptiPlex-990:~/mycroft-core$ pip install requests
Collecting requests
Using cached link://files.pythonhosted.org/packages/ff/17/5cbb026005115301a8fb2f9b0e3e8d32313142fe8b617070e7baad20554f/requests-2.20.1-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2 (from requests)
Using cached link://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting certifi>=2017.4.17 (from requests)
Using cached link://files.pythonhosted.org/packages/56/9d/1d02dd80bc4cd955f98980f28c5ee2200e1209292d5f9e9cc8d030d18655/certifi-2018.10.15-py2.py3-none-any.whl
Collecting urllib3<1.25,>=1.21.1 (from requests)
Using cached link://files.pythonhosted.org/packages/62/00/ee1d7de624db8ba7090d1226aebefab96a2c71cd5cfa7629d6ad3f61b79e/urllib3-1.24.1-py2.py3-none-any.whl
Collecting idna<2.8,>=2.5 (from requests)
Using cached link://files.pythonhosted.org/packages/4b/2a/0276479a4b3caeb8a8c1af2f8e4355746a97fab05a372e4a2c6a6b876165/idna-2.7-py2.py3-none-any.whl
Installing collected packages: chardet, certifi, urllib3, idna, requests
Successfully installed certifi-2018.10.15 chardet-3.0.4 idna-2.7 requests-2.20.1 urllib3-1.24.1
(.venv) abdulrahman@abdulrahman-OptiPlex-990:~/mycroft-core$

Command 3:

(.venv) abdulrahman@abdulrahman-OptiPlex-990:~/mycroft-core$ ./start-mycroft.sh debug
Starting all mycroft-core services
Initializing…
Starting background service bus
CAUTION: The Mycroft bus is an open websocket with no built-in security
measures. You are responsible for protecting the local port
8181 with a firewall as appropriate.
Starting background service skills
Starting background service audio
Starting background service voice
Starting cli
Traceback (most recent call last):
File “/usr/lib/python3.6/runpy.py”, line 183, in _run_module_as_main
mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
File “/usr/lib/python3.6/runpy.py”, line 109, in _get_module_details
import(pkg_name)
File “/home/abdulrahman/mycroft-core/mycroft/init.py”, line 17, in
from mycroft.api import Api
File “/home/abdulrahman/mycroft-core/mycroft/api/init.py”, line 18, in
import requests
ModuleNotFoundError: No module named ‘requests’
(.venv) abdulrahman@abdulrahman-OptiPlex-990:~/mycroft-core$

Thanks,

Abdulrahman

KathyReid · November 19, 2018, 12:52pm

Thanks so much @Abdulrahman2 - one more thing to try;

activate the venv
then try pip3 install requests

I think I might have given you incorrect information regarding the pip install command - pip install will install for Python2, but for Python 3 we need to use pip3 install, my apologies.

Abdulrahman2 · November 20, 2018, 4:46am

Thanks again @KathyReid.

I tried pip3 and unfortunately it did not solve the problem.

abdulrahman@abdulrahman-OptiPlex-990:~/mycroft-core$ pip3 list | grep requests
DEPRECATION: The default format will switch to columns in the future. You can use --format=(legacy|columns) (or define a format=(legacy|columns) in your pip.conf under the [list] section) to disable this warning.
requests (2.20.0)
requests-unixsocket (0.1.5)
abdulrahman@abdulrahman-OptiPlex-990:~/mycroft-core$

abdulrahman@abdulrahman-OptiPlex-990:~/mycroft-core$ source venv-activate.sh
Entering Mycroft virtual environment. Run ‘mycroft-venv-deactivate’ to exit
(.venv) abdulrahman@abdulrahman-OptiPlex-990:~/mycroft-core$ pip3 list | grep requests
DEPRECATION: The default format will switch to columns in the future. You can use --format=(legacy|columns) (or define a format=(legacy|columns) in your pip.conf under the [list] section) to disable this warning.
requests (2.20.0)
requests-unixsocket (0.1.5)
(.venv) abdulrahman@abdulrahman-OptiPlex-990:~/mycroft-core$ pip3 install requests
Collecting requests
Using cached https ://files.pythonhosted.org/packages/ff/17/5cbb026005115301a8fb2f9b0e3e8d32313142fe8b617070e7baad20554f/requests-2.20.1-py2.py3-none-any.whl
Collecting certifi>=2017.4.17 (from requests)
Using cached https ://files.pythonhosted.org/packages/56/9d/1d02dd80bc4cd955f98980f28c5ee2200e1209292d5f9e9cc8d030d18655/certifi-2018.10.15-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2 (from requests)
Using cached https ://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting urllib3<1.25,>=1.21.1 (from requests)
Using cached https ://files.pythonhosted.org/packages/62/00/ee1d7de624db8ba7090d1226aebefab96a2c71cd5cfa7629d6ad3f61b79e/urllib3-1.24.1-py2.py3-none-any.whl
Collecting idna<2.8,>=2.5 (from requests)
Using cached https ://files.pythonhosted.org/packages/4b/2a/0276479a4b3caeb8a8c1af2f8e4355746a97fab05a372e4a2c6a6b876165/idna-2.7-py2.py3-none-any.whl
Installing collected packages: certifi, chardet, urllib3, idna, requests
Successfully installed certifi-2018.10.15 chardet-3.0.4 idna-2.7 requests-2.20.1 urllib3-1.24.1
(.venv) abdulrahman@abdulrahman-OptiPlex-990:~/mycroft-core$

(.venv) abdulrahman@abdulrahman-OptiPlex-990:~/mycroft-core$ ./start-mycroft.sh debug
Starting all mycroft-core services
Initializing…
Starting background service bus
CAUTION: The Mycroft bus is an open websocket with no built-in security
measures. You are responsible for protecting the local port
8181 with a firewall as appropriate.
Starting background service skills
Starting background service audio
Starting background service voice
Starting cli
Traceback (most recent call last):
File “/usr/lib/python3.6/runpy.py”, line 183, in _run_module_as_main
mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
File “/usr/lib/python3.6/runpy.py”, line 109, in _get_module_details
import(pkg_name)
File “/home/abdulrahman/mycroft-core/mycroft/init.py”, line 17, in
from mycroft.api import Api
File “/home/abdulrahman/mycroft-core/mycroft/api/init.py”, line 18, in
import requests
ModuleNotFoundError: No module named ‘requests’
(.venv) abdulrahman@abdulrahman-OptiPlex-990:~/mycroft-core$

I also run the ‘dev_setup.sh’ again and the debug error remains the same.

Thanks,

Abdulrahman

j1nx · November 20, 2018, 7:01am

In the bin folder inside the mycroft-core folder is a bash/binary script called; “mycroft-pip”, best practice is to run that one.

cd ~/mycroft-core/bin
./mycroft-pip install requests

This script should bring you into the right virtual environment and install python-requests where it should…

Could you give that one a try?

(the mycroft-core/bin folder could be added to the user it’s $PATH but now sure. If so, you could also just run the command; “mycroft-pip install requests” )

Abdulrahman2 · November 20, 2018, 7:45am

Thanks @j1nx.

I tried mycroft-pip and I still get the ModuleNotFoundError error for the requests package.

Regards,
Abdulrahman

j1nx · November 20, 2018, 9:48am

Can you post the output from the command prompt when you run that command? If it is still not there, there must be an error of some kind that will bring us a bit further.

Abdulrahman2 · November 25, 2018, 5:44am

Thank you @KathyReid and @j1nx.

The probelm has been solved. Mycroft works fine now and no more errors when running “./start-mycroft.sh debug” and the “cli”.

The cause of the problem was related to an incomplete setup. I fixed this by modifying the https_proxy variable from:
export https_proxy=https://proxy_hostname.com:proxy_port
to:
export https_proxy=http://proxy_hostname.com:proxy_port
(i.e., :http without s) and re-running “./dev_setup.sh” again.

Thanks,

Abdulrahman

machinekoder · August 16, 2019, 9:03am

Weird, I encountered the same error, but only inside Docker.

I fixed it by activating the venv before running ./start_mycroft.sh debug