[Programming] Online GUI Skills Builder


#1

Skill name: Skill-Builder

User story:

A skill for those users who want to rapidly prototype a new skill using graphical building blocks in the Mycroft web based configuration / settings page, rather than cli development - this would rapidly broaden the skillsbase - perhaps even a moderated “Skills-store” rather than gitbhub ?

Similar to ‘scratch’ to drag n’ drop and build skills online.

What third party services, data sets or platforms will the Skill interact with?

There could be a pre-defined set of services available from drop-downs or text entry.

Are there similar Mycroft Skills already?

There are the github templates, but this can be off-putting for those who are new to coding.

What will the user Speak to trigger the Skill?

Whatever the user defines, to activate the new skill they are creating.

What phrases will Mycroft Speak?

Defined by the user creating the base skill.

What Skill Settings will this Skill need to store?

Defined by the user creating the base skill.

Other comments?

Maybe use the pairing key / uid to lock the dev skill to your Mycroft until released and approved in the skill-store >


#2

slightly related, i made a node-red fallback skill, so you can use visual programming to add functionality to mycroft


#3

As a follow-up to this, I’ve been building a new Robot (actually a fully functioning Wall-E for my daughter - a surprise for her so shhh), and as Mycroft struggles on a Pi-Zero, I tried out Watson.

Based on the TJBot, it needed a bit of hacking for the zero, but it works well - the best bit being the Watson Assistant Website - which is very similar to my suggestion above, and the point of this follow-up.

Basically it’s a point n’ click, type in what you want to be recognised and enter the responses or further dialog, not a scrap of coding required. This would be a godsend for us Mycroft fans - any chance of something similar in the future ?


#4

We’ve had a lot of discussion about a visual Skills builder for Mycroft - the idea has very strong internal support. The challenge is the technical approach and implementation. It would probably need some sort of modular approach, where functions are exposed to the visual Skills builder. How do we expose several Python functions visually? We could perhaps get to a very basic building-block level, but how complex could we get?


#5

I’m not sure on the technical aspects of such a proposal… But a WYSIWYG skills creator would - in my opinion - be essential for the eventual mass adoption of Mycroft.

The Everyday Joe/Everyday Sally doesn’t want to learn to code, they want to get their “smart assistant”, drag-and-drop to create their own skills on their computer/smartphone/phablet/tablet and then go on their merry way… One of the “big players” (Google or Amazon if I remember correctly) recently introduced something similar, and I think Mycroft will miss a big piece of the pie if it does not follow the same path.

Yes it’s going to be limited in what it can do, but at the end of the day, it’s no different to a WYSIWYG website builder - you can do a lot of what you want to achieve without learning coding, but if there’s something you can’t do with a WYSIWYG wizard, then you’re just going to need to suck it up and learn to code…


#6

I agree 100%. When I want to create a prototype of a voice interaction / skill I would use Adobe XD where I can draw a flowchart and add voice interactions to it, that are operated by Amazon Alexa. I would love to use open source for designing and for voice interaction.

I am keen to help design / build such a WYSIWYG builder for mycroft. How can we start?


#7

Could we use Dia (https://sourceforge.net/projects/dia-installer/ ) or similar tool as the visual editor?


#8

Seems like double work to me. Right now, a skill is the integration of an API or important function into mycroft.
Visual Programming would require to the same thing again, but this time the “skills” would be building blocks for a Visual Language.

That would double the amount of work required to integrate an API.


#9

I have been working in chatterbox, a mycroft based build it yourself assistant for kids, we have been testing in schools with great feedback! if a kid can use it, anyone from this community can also do it

We use a block based approach to build skills, our core is a fork of mycroft with more features, this means not every skill built with chatterbox will work on mycroft, but all mycroft skills will work on chatterbox, we are exploring how to better handle this

our kickstarter launch is imminent, stay tuned

http://hellochatterbox.com


#10

I LOOOOOVE THAT. And I am sure my daughter would to…I WANT SOME OF THOSE :slight_smile:

but it has to be translated to my language……how can that be done? Sme way as mycroft or how ? I would like to help out as much as posible.

P:S. Is there a grownup box that is bigger and could do some house work as well :wink:


#11

you will be making your own skills, in minutes! while possible to download skills, the main idea is that you build everything yourself, we are working hard to make that super easy! this means skills will support your language by default, you just write them in that language

on the interface side we are also keeping language support in mind, all blocks and interface will support multiple languages

development is being made in english, but as a portuguese speaker i’m making sure it also supports my own language, and therefore any other language :wink:

like with mycroft, the big hurdle will be the parsing/formatting functions that are language specific, but it is safe to assume that everything mycroft supports, so do we


#12

Det lyder fantastisk. :slight_smile: