I’ve been studying and trying to learn/understand the flow of execution as mycroft starts up and as it runs; exactly what commands are being executed, line by line, of any code that wasn’t written outside of mycroft. As I continue, I’m seeing a lot of the websockets module. How in-depth should I go in learning about this module? I have some experience in web architecture and development, but I’d say I’m on the low side of intermediate.
I’ve managed to get all the way through start-mycroft.sh, and I’ve looked at a fair amount of the modules in the mycroft-core/mycroft folder. I’ve also looked at some of the mycroft-core/skills folder; I’m stuck on finding where the code moves from starting up to running, where the code is that controls taking in the text (whether from mycroft-say-to or from stt) and running the right skill in return, and then goes back to listening for the next input.
Interested to hear what others think about this.
First I’d say it always depends on why you are learning, and what your goals are? Is it so you can audit what’s happening on your device, to improve your technical skills for your career, or something else?
Fully understanding the technologies behind every part of Mycroft would take (at least) years, if not your whole lifetime. Depending on what you consider a “full understanding”. I’d take a pretty strong bet that no one at Mycroft fully understands everything. Everyone specialises in something, whether that’s backend, core, machine learning, etc. For most things I would say learn them as you need them.
A high level understand of what websockets are and how they differ from other protocols seems quite useful. This will help you to understand how messages are being shared and the nature of those messages. The wikipedia page for websockets looks like a fairly good yet brief overview that probably covers enough. At least until you run into a problem that provides a reason to go into more depth.
Thank you so much for your reply.
My end goal is making a system that can carry out what is essentially an interview, with different paths based on responses to certain questions. It has a script to follow, starting with a greeting, and then asking about the users barrier for healthy eating. To start with, it will just ask “is ___ a barrier for you”, but eventually I’d like it to just ask “what are some barriers” and be able to pick out key words. It has a different set of questions for each potential barrier. For this kind of intent parsing I’d like to integrate the Stanford NLP package, which I assume would have to alter/build on the functionality of Adapt.
I’ve been starting with just finding/understanding the commands being executed as mycroft runs. I’ll need to know how Adapt is called and used if I’m going to alter/build on it, and at the very least it will be much easier for me (and the silly way my mind works) to write a skill if I understand the context in which it’s being called. I’m imagining some code somewhere that says something along the lines of
while (utterance) {
skill_info= adapt_function(utterance)
python3 -m skill_info[‘name’] skill_info[‘params’]
}
But I’m not sure where that is, or where that functionality happens, if it’s not written in that way. And I can’t find the code that runs between “python3 -m ${_module} ${_params}” in start-mycroft.sh, and the pseudo code I just mentioned. When I start for the first time, it runs the pairing skill. Why? Where does it check if it needs to do that? Once it has checked, where does the code jump to?
By full understanding I guess I mean vaguely knowing the path it takes, if you were to run it in debug mode and step through every line that runs that’s in the mycroft files, down to mycroft-core/mycroft/util (which I have looked at a lot). So, not the normal non-mycroft python modules, just the stuff that’s been written for mycroft.
I will definitely take a look at that wikipedia article. Thank you!
Taking a step back then, it seems like you can create the functionality you’re describing within a Skill.
A Skill already receives the utterance that triggered it. You can then use the get_response method to ask the questions. You will receive the responses back directly and can then process those to determine what to do next.
Conversational context can also be used to keep track of where the user is up to, and which intents should trigger based on previous answers.
I wonder if you could even use the python package for Stanford NLP in the skill? I haven’t used it myself, so unsure what would be required there.
Ah okay, that get_response method will be super useful. If I’m able to use the python package for the stanford NLP inside the skill, would I be able to bypass the adapt intent parser altogether? How do I get it to use my skill on startup, instead of triggering it with an utterance?
That’s a much bigger question if you do really need the Stanford NLP package to be parsing all intents you’re looking at building your own intent parser. In that case I’d be trying to understand how Adapt and Padatious operate and interact with Mycroft-core.
Everything centres around the messagebus though. A message placed on the bus would be received by all the connected processes. It is up to each process to decide what it wants to listen for and react to.
Okay, good to know. Yeah I got some more info from chat about the architecture of mycroft and stuff about the message bus. I think I have enough now to start working with, until I run into other things. I’ll make a new post if I run into something that the internet and chat haven’t been able to explain or resolve for me.
Thank you so much for your help!
Okay one last question. How, if at all, is the websocket related to the messagebus? Is the websocket the connection to the messagebus? Or is it a separate thing for communicating with Mimic stuff that isn’t on the machine?