How to handle 1000 Mycroft users accessing one backend?

General Discussion
1

I am working on a project that wants to manage 1000 users with one Mycroft backend (for one company).

We thought about using a containerized server that spins up a Mycroft-core container/pod for each user. Another idea is using (and probably modifying) HiveMind. Any ideas how Mycroft could support 1000 users? What are the bottlenecks?

2

How would you handle STT and TTS? Are you trying to run all the skills on one backend? Are you allowing each user to have personalized skills? Are you trying to run offline or in the cloud or…?

3

Would a core shared between client devices be an option? NeonAI (forked from Mycroft with skills inter-compatibility) could do this with one core and per-user settings for language, location, etc. NeonCore on Github. We have a demo of this up at Neon Nano for Website Development Applications | Neon AI.

4

Thanks for the feedback. Our setup has these requirements:

  • A user must be able to use one or more devices to access their personal assistant (mobile device, Smartspeaker).
  • A user can customize their assistant (skills, in-skill configuration, language).
  • A user must be able to access Internet/Intranet-based skills (query information from other systems).
  • A user must be able to access only their own conversations.
  • The assistant should not share user’s data without consent.

These requirements already constrains the options, e.g. an only-offline solution is not useful, STT and TTS should work on-device (no sending of data to a third-parties server).

5

Main obstacle I see here is per-user skills (not skill config, but actual available skills)… A possible solution could be one core per user, though it just seems like a lot of overhead to spin up ~1000 instances. Is there a fixed set of skills available for users to add? Could you just install all available skills and then use per-user skill whitelists/blacklists; I think that would be easier to scale to a large number of users.

1 Like
6

Yeah, presently the “fast” approach I’d take to do this would be 1 core per user but what sort of processing that requires would take some testing. Given you don’t need to run any wake word or audio services on the server that would reduce the required resources. The Skills service would probably be the most resource intensive then.

For your use case I’m guessing that most of the users would be online at the same time? Eg standard work hours would be extremely busy, then basically no traffic overnight? Or is it more distributed across a 24/7 cycle?

7

You are right that there will be peak times over the day. However, in my case, user registration is controlled and once someone is registered, their assistant should be available 24/7. A container per user was my first thougt too, but it is more like a crutch rather than a proper solution because it does not scale well. We are investigating what we could do to solve the issue long-term.

8

We could limit the skills to a set of accepted skills and then allow users to customize that list only.

9

I think with this, the missing piece to implement is tracking skills blacklist/whitelist per-user and then using that to filter intents in the IntentService. You will have to work out some logic to limit which intents/skills are matched in Adapt/Padatious, but I think that is an easier problem to solve than scaling with multiple cores

10

Yeah, I think the proper solution will require some pretty significant changes in core. It’s an area that’s on our longer term roadmap but the focus for now is very much on the Mark II.

In terms of why this is the case - Mycroft development so far has been based on the premise of mycroft-core being run on a single device. It makes a lot of sense in your use case, however for the majority of Mycroft users, a big selling point is not sending everything to the cloud.

If it’s something you want our help with then please shout out, but it sounds like your team are working on a solution already.

11

We are investigating solutions indeed. The whole “multi-user multi-device” topic requires quite some effort to address properly (means mainly secure, scalable, and compatible). I hope we manage to build a small prototype before Christmas though :slight_smile: