Build an open future with us.

Invest in Mycroft and become a community partner.

Running background processes with skills?


#1

I’m working on a skill to have Mycroft accept a command and run a long-running process as a result. I’m curious as to what’s the best practise for this?

  • Using Python threads?
  • Having him execute a shell script that somehow detaches itself?
  • Talking to a must-be-already-running daemon?

What’s the sanctioned method?


#2

This is a great topic that may need some discussion as there are no firm rules or regulations (anything goes). There are however bad and good ways of doing this :slight_smile:

I’ve not run this exact use case myself. I assume you want the process to outlive the skill start/shutdown?

For ease of installation an already running daemon is most likely not practical, since the normal skill install method doesn’t accommodate for installing custom services.

If the process don’t need to outlive the skill I recommend starting the process in the __init__() method and having a custom shutdown method (override shutdown()) that terminates the subprocess in a neat way (I don’t always practice what I preach)

You can start a process and detatch it once and then check if the process is running (either communication or search for running processes)

If starting a separate thread make sure the thread doesn’t share resources with the skill object since this will hinder garbage collection when the skill is reloaded.

Do you have any preferred way you’d like this to be handled? or have you been experimenting with this?

/Åke


#3

Well I have a reasonable amount of experience writing Python (10years) and some experience writing long-running processes in that time. My question is really about how best to frame it within the Mycroft paradigm.

Specifically I want to be able to say “Hey Mycroft, do the thing” and then have it do that thing while also waiting for me to say “Hey Mycroft, stop doing the thing”. If I run everything in an infinite loop, that will block, correct?

If as you suggest, I run the script as a subprocess (that doesn’t wait for a return value obviously) then I just need to develop a way for Mycroft to signal to that subprocess that it’s time to die.

I suppose using subprocess is probably the best option in the absence of any sort of async framework (cellery, or something). I just wanted to be sure that that was likely the best option, or if there was an established norm that I should be following.


#4

For things like periodically polling and on certain result taking action we have an “event scheduler” that can call a method on your skill periodically using the event emitter system. These methods shouldn’t block since they’re running in thread pool.

I sadly have no experience with async-IO and with your 10 years python experience you’re quite a bit more experienced than I am. Any suggestions for a good system for these things is much appreciated.


#5

Alright well I’ll tinker a bit and post something here once I’ve got a working model. Thanks for the pointer to the scheduler, I just wanted to make sure I wasn’t breaking any established norms before I started building.


#6

It might also be worthwhile, @danielquinn, to check out the Mycroft discussions on Mattermost. There was talk back in October last year about this very concept:

https://chat.mycroft.ai/community/pl/iqitiqcaziga5d58jcomugmj3r


#7

This is something that I’ve been thinking about as well, but my Python experience can be counted in days rather than years. There are several tasks for work that I would love to automate with Mycroft, but they can take many minutes, and I wouldn’t want Mycroft to be unavailable during that time. What I would love to see (and have zero idea how to implement) is an ability for Mycroft to kick off external commands/scripts and keep track of what it’s kicked off in a list (like the Cows list). Mycroft would have a verbal acknowledgement when the processes are started. After that, provide a mechanism for that process to notify Mycroft that it’s complete so it can give a verbal queue to me. After that, remove it from the list of running tasks. As I said, I have no idea how this would be implemented, but it would be very handy to have.