The Mycroft Personal Server - Starting the Conversation

I am very much interested in a personal server from both a personal as well as corporate standpoint. I am a researcher for a large company and see value in the potential to integrate this technology into some of our projects. However, for both personal as well as corporate, privacy (as well as the perception of privacy) is critical. In my opinion, a private server would radically improve your offering and value to both the personal as well as the business marketplace.

After reading a good bit of your posts I feel I can trust you. However, that is not enough. Things change, accidents happen, and if the audio is allowed to leave my protected area (home or work) then I must assume it may be compromised. For me personally, I am ok with that. I can review your code and documentation and calculate the odds of sensitive audio getting out. However, I would not be able to convince my family or my company to take such risks as they can not fully understand them. This service is not yet a necessity in our culture (like cell phones or facebook) and as such the level of acceptable risk is much lower. I expect that to change someday soon. Meanwhile, a private server for TTS would make the digital assistant service available to all. Please offer it and offer it soon. I am happy to help in any way I can.
Ron

1 Like

How Powerful a gpu are we talking here?

Id be interested to know what the performace would be on something like UDOO Bolt
that has a Radeon Vega 8 in it, claims “a mobile GPU on par with GTX 950M”

Could you build a “Mycroft Plus” based on one of them and slave others to it?

or are we talking Desktop class/Gaming GPU

I will paraphrase @steve.penrod :wink:

Another plus for using Docker to distribute the platform is that some NAS vendors (primarily QNAP) are now offering NAS models that both have the CPU & GPU capabilities to run AI type workloads (TensorFlow,etc) in box.

I happen to have a Ryzen based QNAP box at home that also has a Nvidia 1060 6GB GPU in it. Something like this might be an alternative option that is a little more “user friendly”, while still having the horse power to handle the load.

Hey there,
I think its a great idea. I would even say its necessary in order for Mycroft being truely cloudless and private.

So, what about the Software that is running on Home now? Is it open source?

My idea would be to release it as a virtual appliance, this could be run on any desktop computer or notebook that had a dedicated gpu for the home user or on an ESXi instance for the pro users.

I have been using linux for years and have to admit that getting nv and ati gpus to work with the appropriate drivers can be frustrating, so a completely preconfigured vm would be great for the average user as well as for power users who are running vms anyway.
I also like the idea of releasing it as a qnap compatible format, as this would be relatively easy to use for home users as well.

And of course i think distributing it over all the mycrofts in the household is a great idea, this doesnt seem to be the right choice when it comes to optimizing latency and also it seems like a much bigger project than just running it on one server.

I think this is a great idea. Not everyone wants the put their info in the cloud or like you said they have internet/bandwidth issues. I also see this expanding personal assistant use to non traditional places such as cars, boats, off grid locations.

I am personally interested for the non traditional and privacy concerns.

I personally thing this will work better if it can be optimized for lower hardware specs. it will be able to go more places at that point. If you could run the whole thing on 2 Raspberry Pis or on some outdated home server someone built years ago would be awesome.

I have some old crappy PC hardware and a raspberry pi ready for testing if needed!

Great Idea! I’ve been actually thinking of it for a while since I built my own “personal smart assistant” including mycroft and a few other technologies, such as vision and so. Everything was stuffed into a raspberry PI 3 but STT… which had to be outsourced to google. As a consequence I started playing with deepspeech, so according to me you are proposing an… almost perfect solution.
It is almost perfect because, as you stated, it requires a quite expensive and geeky hardware to run the server.
I would rather go for deepspeech and tensorflow lite running on an ARM platform with GPU (sounds familiar, doesn’t it? maybe a smartphone?). The point here would be to build tensorflow lite and Deepspeech and provide the needed info to build them for most “hacker-friendly” smartphone platform such as OnePlus and Google Pixel.

1 Like

I think this is an amazing idea! I’m a geek who enjoys creating and hosting his (or her, I’ll never tell :wink: ) own resources either for fun, or to educate myself. Though mostly, it’s out of concern for my privacy. And while I must commend you, because I haven’t come across any issues with any of you, I’m still a little overly cautious and like to host my own resources (such as mail, web pages, databases and such). I think the idea would be rather neat.

https://www.xilinx.com/applications/megatrends/machine-learning/machine-learning-developer-lab-form.html

It would be interesting to utilize FPGA’s. For $15-20, I am hoping that quite a boost can be obtained. So running complex voice recognition modules, and thus having better speech recognition for speech control. That is what is so awesome about this project. You are essentially controlling skills. Outside of needing different pricing model, I think that that mycroft personal could absolutely target the commercial market.

Count me in.
This will definitely be the make or break feature for me to use MyCroft to add smart Voice Control to my home automation.
Seeing this happen in combination with a solution for having distributed access devices (sound in/out) throughout the house would definitely come close to my ideal setup for this.

Thanks for your feedback @Martin_Corino - we agree - having a self-hosted solution is important for those who value their privacy, which is a key reason that people come to us instead of say Alexa, Siri or Google Home.

Distributed device access is harder to achieve but definitely something we want to work on.

Check out our Roadmaps for more information.

Thanks for the reply Kathy.
I will be eagerly awaiting test versions :grinning:
In the meantime I’ll pitch in with the Dutch translation because that’s the third “must have” for me.

1 Like

Hi,
Just found Mycroft.ai, I am experimenting with a Rock64 running Deb. installed Mycroft for Linux working OK.

Regarding computing power for a local server. I would be very unhappy to see vast power hungary X86 Gpu pcs employed.

I cannot see anyone mentioning using an arm based SBC cluster… Good processing power, a lot less power consumption and cheaper to buy.
Have you quantified how much processing horsepower you need for a domestic server?
Here is a brief article :- https://www.linux.com/blog/2018/7/sbc-clusters-beyond-raspberry-pi
Here is a company building and selling various multi SBC clusters:-https://www.picocluster.com/
It wold be interesting to see what compute power these clusters could deliver!
regards
Patrick

When something can replace the compute TOPS and per watt of a gpu at comparable pricing, I’ll be happy to look at it. At last check, the best ARM GPU’s were a factor or more below what a desktop GPU could. the power draw on a small gpu is minimal when not actively engaged in processing. Clusters tend to be less efficient than a larger single machine, and cheaper in the long run.

But they’re fun to setup and play with.

Hi,

Lets not run away from reality here!

A slightly confusing reply, from you, but I believe you are referencing the Intel TOPS project:-

A fantastic project, but I see no pricing for practical iron yet??

A medium size arm sbc cluster doesn’t measure up to that or indeed a top of the range gaming GPU. Here is a recent ARM project including benchmarks:-

https://climbers.net/sbc/nanopi-fire3-arm-supercomputer/

This was measured using Linpack TPP v1.4.1 at 60,000 Mflops BUT only 55w total power at full load. The cost was around £545 complete!

Gaming Gpus have many orders of magnitude more horsepower but the total cost is greater, the cost has to include the pc the GPU is installed in.

How many Mflops (approximately) power is necessary to produce a working local compute mycroft server with a reasonable performance?

There is no point specifying a server that will need a second mortgage to purchase it and a power station in the back garden to run it!

After all you are competing with a $25 product from Amazon or Google, (Ignoring the privacy aspect). You have to put a very high value on your privacy to fork out for a PC based GPU server to replace commercial unit.

cheers

Patrick

trillion operations per second.
Used desktop, $50 (lenovo m73). Nvidia 1030, $75 (x2). This runs mycroft, local copy of wikipedia, and mimic2 easily. I actually have 2 1030’s, so it can do kaldi or deepspeech as well. Kaldi works better than DS right now, but it’s a bit slower. Performance for everything but STT is quite reasonable. Mimic and Deepspeech are both working towards lowering the requirements necessary for usefulness as well. *edited to add, also runs the personal backend and front end bits easily.

I have a Neural compute stick; their limited model compatibility makes them moot for use on mycroft currently.

Im loving the idea of my croft local server… I have 2 recently decommissioned ProLiant and a few quadro cards lying around. I would be happy to lend them to this project if it proceeds… Keep me posted.

New to Mycroft - but VERY excited about the project and personal server. Like others, I run a DL380 G7 @ home with vmware. I also have a fairly powerful linux desktop with 32g ram, 9 core AMD processor, and lower powered GPU. Looking forward to learning and growing with you guys!

For your holiday free time…

How I got local wikipedia working: https://github.com/el-tocino/mycroft-local-wiki

A quick and dirty TTS backend for local mimic2 instances: https://github.com/el-tocino/localcroft
This one basically just pulls the WAV from mimic and returns it, skipping visimes and some other (useful) safety checks, but it does work locally. It keeps local copies of responses, which speeds up repeat responses quite a bit. This will work…poorly…with very long responses, odd numbers, etc. Also be aware that the first response from a newly-started mimic2 demo_server instance will probably trip the request time out. Subsequent responses should work well. After restarting, load the web page and verify so you can avoid this.

If you’re running a local copy of Deepspeech, this can be pointed to your own instance already using local config*. I am not yet enamored of this option, though I’m going to try making a custom language model to try and make it work better for me. This repo is the script I use to start that up, adjust to your local settings as needed: https://github.com/el-tocino/DSSS

Between these, you’re 95% of the way to running local. The remainder is currently left as an exercise for the reader, and will get addressed at some point in the future by the core folks. Also skills will still do their thing, of course.

2 Likes

This looks awesome!

Can you give a hint on machine requirements (cpu, ram, etc.) for the deepspeech-server?
Is it feasible for desktop-cpus or should i stay away when i don’t have a gpu?
What is necessary to have for example a 1:1 performance (1 second of speech requires 1 second of deepspeech processing)?