I have backed on the new Mycroft and want to use it ideally completely offline.
I have a central home automation server, that fetches all important data from sensors or apis that should be controled by mycroft.
But I want to avoid that my speech is sent to a server outside my controlled home-subnet.
So the question is, what is the easiest way to make mycroft work offline (without an extra server or with a self-hosted one.)
Hi there @skeltob, this is one of our most requested features - completely offline use. Unfortunately we don’t have good documentation on how to do this at the moment. I do know that some of our Community members have done this in the past, including @Jarbas_Ai and may have some guidance.
It’s definitely something we want to do @gregory.opera, however it requires a bit of work on our side. Our existing home.mycroft.ai platform is scaled to support tens of thousands of users, and runs across several virtual hosts - probably not all that usable as a local / personal backend. So we need to work on scaling that down.
The other layers to this problem are;
Speech to text - really this is the biggest blocker at the moment. Until we can get DeepSpeech to a point where it can run (or at least a vocabulary subset can run) on an embedded device, then we’re going to be stuck with cloud-based STT, irrespective of which cloud that runs on. There have been some substantive efforts by the DeepSpeech community toward this objective.
Skill support - most Skills need some form of internet connectivity as they’re connecting to third party APIs.
Configuration settings - at the moment, configuration of Devices is done via Skill Settings at home.mycroft.ai so we would need to find a way to do configuration locally.
I have been running deepspeech locally with the ‘pretrained’ model on a separate computer in my house recently.
It was fairly easy to set up and to point Mycroft at it. The server does not have a GPU, so it’s not as fast as it could be, but I think the gain in local network speed makes it not that different from the cloud service, which is kind of slow too, in my opinion. I will probably get a GPU based server at some point, but don’t expect a huge improvement in speed, because non-GPU is already usable for the short commands I use.
The big hit I’m taking is with accuracy. I have to speak slowly, right in front of the mycroft, and leave gaps of silence between words.
I’m currently starting to research ways to better train the local service. I have not gotten very far
My pipe dream would be for the mycroft community to be able to share and asimmilate incremental training gains without sharing any audio. That’s way over my head at this point, though
If you manage to release something on Windows any time soon, I’m fairly certain that it’d be really easy to do something with C# or VB.NET that uses the System.Speech.Recognition namespace in the Common Language Runtime. Its accuracy isn’t the best, but it is definitely a functional baseline. And it doesn’t seem to need pauses between each word like DeepSpeech apparently does.
yeah, an offline version would be great. My thought is, even if its a “server” program that runs on a GPU server in my house, then I link my imbeded devices to that server, and skills that need to reach out to the internet would, but most of the stuff would happen within my network would be great!
it running on the device itself would be cool, but at least for me, running a central server that those devices connect to within my house instead of the cloud would be awesome!
Just curious, https://home.mycroft.ai/#/deepspeech is just english, isn’t it? Because I’ve found some sentences in spanish and in german… and they were recognized not only correctly, but written in a perfect spanish (e.g: cómo estás, even with the proper accents!)
It would be interesting to tag or filter somehow those new languages, so people who can understand them can flag them out as correct and don’t create false positives… I mean, english people who doesn’t speak any other language, most probably will flag any other language as “No” because they will think is not english. Even me, who speaks spanish, doubted if mark it out or not, because I didn’t see any language filter on the site. It would be a shame to have so many few sentences on those new languages marked as non-valid when the are really good, don’t you think?