Connecting to other devices (UDP? or other)

Archive - Mycroft Project Skills
1

As I learn and develop skills and discover the power of what we have here I find my self pondering more and more complex systems. My current setup is that I have a picroft running and have created a udp skill that passes commands to a couple other systems in my house. System #1 is a raspberry pi that I have been using as a print server for an old usb printer. This RPI now has a daemon running that intercepts the picroft udp requests and turns on some lights I have in my room that require a Python 3 api that I was unable to get working on the picroft. I also have a debian based PC that we will call system #2. It also has a udp daemon running that intercepts picroft udp request that will pull up web pages and open applications on my workstation (“Mycroft…show me the current weather radar”). I have managed to get this working by passing text based udp messages to these system daemons and it works very well, the only drawback I have is that it is a 1 way system. Picroft sends the request to the appropriate daemon and if all goes as planned the daemon responds. Although this is working I can’t help but wonder if there is not a simpler / more robust way to do this whereas I can directly interface to the Mycroft system by intercepting intents and pushing responses back to the picroft from one of my two systems. Does anyone have ideas and/or examples on how this “should” be done?

2

Great question, and great use of integration, @pcwii.

My colleagues @forslund and perhaps @steve.penrod are best placed to answer here, I don’t have that information to hand. Indeed, I’m not sure we have a “one best way” for implementing this use case.

3

Indeed a great question, one to which I have no great and conclusive answer.

Mycroft core uses a websocket based messagebus so technically any device can connect to that and receive all messages sent from the core or skills and send own messages. This is what the android app (and perhaps iOS app) uses. This has some security implications since messages can be injected to install skills and basically get the system to execute arbitrary code (not with any privilege). For this reason the port is disabled using the firewall on Mark-1 and Picroft.

Anyways, if you open up the port on the firewall your devices can connect to the Mycroft messagebus and communicate with skills and core.

I started working on a simple token authentication for the ws, which together with using secure websockets could be a robust and somewhat secure solution.

This could be either on the entire messagebus or we could have a separate “external” messagebus. Using a separate external messagebus could have the advantage of some types of messages could be filtered out if necessary.

A similar “gateway” service could probably be implemented using simple UDP broadcasts or some TCP server architecture is also possible.

Would be interesting to hear what you would like to see in a service like this.

A related issue issue is automatically finding mycroft devices on the network to connect to. I’ve experimented with creating an avahi service on the mycroft device which allows me to scan the network for mycroft devices using zeroconf

This is the output of my simple test script searching for mycroft devices:

$ python find_mycroft.py 
Press enter to exit...

Mycroft._mycroft._tcp.local. at 10.3.181.82:8181

Code here.

1 Like
4

Thanks for the response @forslund @KathyReid. I will attempt to pull together a list and vision of my needs and wants over the next couple days. In the meantime is there a document or flow diagram on interfacing with the websocket? Ultimately I am looking to develop a daemon that could handle intents or requests without having a full blow mycroft running on every piece of “smart” (computers, phones, cars, IoT devices, etc.). Technology I am interfacing with. Somehow the “smart” end device could report its hardware and software capabilities to mycroft and skills can built or routed from there. But I am getting ahead of myself now. I will try to formalize something soon.

1 Like
5

So I have been thinking a bit more about this lately and I really like the idea of having my computer and/or other IOT equipment separate from mycroft but with the ability to exchange information back and forth. As I mentioned earlier I have been doing this with some success using a simple UDP connection. My disapointment is that it lacks the structure of the mycroft skill format. So I am thinking of undertaking a “remote_device_request_skill”. This skill will simply take a mycroft request and pass the remainder TTS data to the daemon running on the remote device. The daemon on the remote device would utilize the Adapt intent parser to further parse the remainder data on the remote device to execute the daemon_skill on that device.
Eg. “Hey Mycroft…Ask the computer to show me the regional weather radar”

  1. The mycroft remote_device_request_skill has two vocab files called request.voc and device.voc
    1a. request.voc contains “Ask”, "Tell"
    1b. device.voc contains “Computer”,
  2. These become the .required portions of the intentbuilder
  3. The skill then passes along the message.utterance_remainder() portion of the request to the remote device via UDP.
  4. The remote device daemon receives the message.utterance_remainder() and passes it through the adapt intent parser to perform the action on the local device.
  5. The Adapt intent parser would act on “show”, “regional”, “radar” to call up a URL that contains the necessary weather radar (or whatever local function you desire, youtube, e-mail, media, google search, file search).

Does this sound like I am on the right track with this? doable? How difficult would it be to utilize Adapt in an application like this?
Is there interest in something like this?

Thanks

1 Like