Anybody who could confirm that all the WiFi stuff actually works with the onboard WiFi chip as well?
Time for me to invest in a rpi4.
Rpi3 version is there as well😉
Sorry, I was not home or did not have time since you rls this. I will write you, hopefully tomorrow.
No worries! I just carried on developing. The next version is already around the corner.
Would be great to have some feedback on the whole WiFi system though, so hope you mange to find some time to test.
I would like to push another test release out into the world. I really, REALLY could use some feedback/testing/bug hunting/etc. from other users then myself. There is only X amount of test work you can do, and even then… I will always do thing the way I have them intended to. Other people might do things in a way I haven’t thought about and it is just those things I am after.
So by all means, if you have some spare time, please download this version and give it a spin. I am open for any type of contributions.
Alright…
MycroftOS - Version 0.1.0 - alpha 7
This 7th version comes with some fixes and additional features. I would also like to visualize the whole “First run / setup - wifi system” by showing some pictures of it. Hopefully I am able to create a nifty little video show casing it as well this weekend, but then again; Said the same to myself last week…
Changelog:
- Started developing the Python enclosure code. Although in the end when finished, this should be handled as an enclosure, for now I migrated it all into a Skill of which a very first intial version is added.
- Mycroft visuals on the HDMI if used for “listening”, “thnking” and “speaking” (see screenshot below).
- Prioritising the loading of soundcard drivers, giving external sound cards more priority over the broadcom onboard chip without blacklisting it as module or diabling the overlay within config.txt
- Couple of filesystem - permission errors/fixes.
- WIFI fixes when using the onboard Broadcom wifi chip.
- Bluetooth fixes when using the onboard broadcom bluetooth chip.
- Fixing the respeaker HAT Led skill, hanging the whole device. The respeakers skill can now be installed and use if wanted (All drivers are already included)
- Latest & Greatest Mycroft 19.8.6 version
Added smart speaker functionality and features:
- Added Spotify support. When enabled, you have a “MycroftOS” spotify connect device within your Spotify app. By default for Spotify connect on the same network you do not need to configure any credentials. If you want to control spotify by voice, you need to istall the Spotify skill and configure it. More on that later…
- Added Airplay support. When enabled, you have a “MycroftOS” airplay device within your iOS music player.
- Added Bluetooth speaker support. When enabled, you have a “MycroftOS” bluetooth audio device you can pair with. No PIN required at the moment, will look into setting a PIN for it some time later on.
- Added Snapcast Server support. When enabled you can make your MycroftOS device a Snapcast server device, forcing all audio output being pushed to Snapcast clients. (Note: You still need to configure this via SSH)
- Added Snapcast Client support. When enabled you can make your MycroftOS device a Snapcast client device, receiving multiroom audio from a Snapcast Server. Yes this could be the Snapcast server on the same MycroftOS device. (Note: You still need to configure this via SSH)
- Added MPD support. When enabled you can make your MycroftOS a MPD server/player. (Note: You still need to configure this via SSH)
First run and Wifi configuration
After you have flashed the image onto a SD card and booted your machine, the following visuals are present when you also attach an HDMI screen.
Depending on how fast your screen switches, a boot message screen is shown;
At first boot, it will first expand the filesystem over the whole size of the SD card and check if the wifi is configured. If not it will show below screen;
At the moment above text is not being spoken out loud. Need to look into that. How to cache Mimic2, because Minic1 sounds to robotic as it will be the very first impression people will get. It has to sound beter than Mimic1
On you phone or tablet, go to your wifi settings and connect to the open MycroftOS-WiFiSetup network.
When it connects it will auto detect, and open a browser to configure your MycroftOS wifi settings. It acts as a captive portal. iOS opens the browser itself;
On Android devices you need to “click to login”;
Anyhow, select your WiFi from the select box. Insert your credentials for it (don’t make a typo mistake !) and hit the connect/send button.
Your wifi will be configured and the device will reboot and continue the process of setting up your MycroftOS device. A few notes on above system;
- If you want to use LAN, click the “cancel link” at the bottom of that page to configure a blank wpa_supplicant file and therefor not configure the wifi, but make it look like it is configured for the system.
- At the moment, I have not yet figured out how to check for a LAN cable connection while still having no wifi configuration. In that regards, you HAVE TO cancel the wifi via above or accept the open wifi network being created.
- If you have a wpa_supplicant.conf file at hand that you would like to use instead of going through the above wizard. Just save the wap_supplicant.conf at the boot FAT partion before you start up the device. The file will be moved to the right folders, skipping the above wizard because of it.
Second boot after the WiFi wizard
When the device reboots, again booting visual is shown. After a while and depending on how fast the network setup at your place is you will see (or not at all);
After that, when internet is established;
Which means the Mycroft A.I. software stack is being started. After some time the logo will disappear when every thing is done and dusted. That means, pairing your device (not yet being shown on the HDMI screen) and installing and loading all default skills.
First boot will take a bit of time as it will download all the default skills including their pip packages. Have a little bit of patience. Mycroft will speak out, when it is fully done.
When everything is done and dusted and Mycroft is ready to rock and roll, the following HDMI visuals are used to communicate it different stages.
Listening / Wake word being heard.
Thinking
Speaking
One more remard to make about the whole HDMI section. At boot it will decide where to output it’s sound. If it sees an HDMI screen attached, it uses the HDMI for audio autput. If not, it uses the 3.5 speaker jack. This system is not hot swapable however later on in time I will make it configurable to be also forced to one or the other.
MycroftOS Configuration Skill
Above features and services can be enabled via the same enclosure skill that handles the Framebuffer visuals. At the moment, starting and disabling these services via voice is disabled. Still figuring out the best way forward for that.
When you have paired your device, you will have a MycroftOS Skill settings section available at your mycroft account page;
The smart speaker functionality just works as it is after enabling them (Snapcast client being the exemption). At a later stage I will explain how to setup the Spotify skill, so you could also voice control your Spotify devices.
All the other system services can be enabled and they will be started with default configuration files. But for them to work properly, you will need to SSH into your box and configure them.
Talking about SSH. ALL services are disabled by default. Different users different use cases, however I dislike everything being started. I take the other way approach. If you want it you need to enable it. For now, while still developing the system SSH is the only one that is enabled by default. However as soon as some sort of final release it there, SSH will by default disabled as well.
If you enable and log into your device. The credentials are still; mycroft / mycroft
CHANGE THE PASSWORD !!!
The firmware / images are currently being uploaded. You can download them at my downloads section;
Happy testing. PLEASE let me know your thoughts, both positive and negative.
Thx
3 Likes
Configuring the Spotify Connect daemon
To make use of the Spotify Mycroft Skill, you will need to configure MycroftOS to use your Spotify premium credentials. It uses a different daemon called “Spotifyd” instead of “raspotify”. Spotifyd is based on raspotify, with some extra things. (Checkout their github repo for more information)
Log into your MycroftOS device via SSH (see above post for more info).
When logging in the mycroft-cli-client will been shown.
Cancel out of it with; CTRL-C
No onto the linux CLI issue the following command.
sudo nano /etc/spotifyd.conf
A text editor will open, showing the following content;
[global]
# The port `spotifyd` uses to announce its service over the network.
zeroconf_port = 57621
# Cleartext username and password for remote Spotify access.
# Make sure the file is only accessible by spotifyd.
# username = USER
# password = PASS
# alsa, pulseaudio or pipe
backend = pulseaudio
# Name in Spotify Connect, cannot contain spaces:
device_name = MycroftOS
Go to the username / password section and fill in both your Spotify premium username and password and remove the HASH sign (#) infront of it.
# Cleartext username and password for remote Spotify access.
# Make sure the file is only accessible by spotifyd.
username = my-spotify-premium-username
password = my-spotify-premium-password
Exit out of the text editor by; CTRL-X It will ask if you want to save, answer by; Y
Now we need to restart the spotify connect daemon with the following command.
sudo systemctl restart spotifyd.service
Spotify on the MycroftOS device is nwo properly configured.
Just install the Spotify Mycroft Skill as ussual and follow the installation instruction.
First OAUTH Mycroft via the button. Then fill in the username and pasword at the mycroft skill section. You do not have to use the real user/passwd combo, just fill something in as there is a check for empty fields. They will not be used, our username/password setup above will be used instead.
Enjoy Spotify connect with voice control.
I will see if i can get some tinkertime this weekend for this. I am sure i have an sd card lying around here somewhere.
Perfect! All I can ask for.
SD card doesn’t have to be big. Had to increase the default starting size from 1 GB to 1.5 GB because I left the MPD depency for Samba there nbecause at some point I will start using sharing some folder over the network anyway (config files, local music, update partition, etc)
All with all with all default skills installed after the first run, you end up with; 1.2 GB in use.
mycroft@MycroftOS:~ $ df -h
Filesystem Size Used Available Use% Mounted on
/dev/root 14.3G 1.1G 12.6G 8% /
devtmpfs 925.1M 0 925.1M 0% /dev
tmpfs 958.1M 0 958.1M 0% /dev/shm
tmpfs 958.1M 24.4M 933.7M 3% /run
tmpfs 958.1M 0 958.1M 0% /sys/fs/cgroup
tmpfs 958.1M 4.0K 958.1M 0% /var/tmp
tmpfs 958.1M 128.0K 958.0M 0% /tmp
tmpfs 958.1M 268.0K 957.8M 0% /var/log
tmpfs 20.0M 8.0K 20.0M 0% /ramdisk
/dev/mmcblk0p1 31.9M 9.4M 22.5M 30% /boot
So an old 4 GB card is more than enough. (Not takingh the speed into account ofcourse)
This looks very promising. Excellent work.
1 Like
Lets see if i can make use of this new toy for some testing…
4 Likes
(more) pics or it didn’t happen…
6 Likes
Preliminary testing results and recommendations.
Tested on…
- new out of the box rpi4 (4gb)
- 8 gb sd-card (class 4, not great, all I had lying around)
- no microphone
- hdmi connection to samsung tv
- Download from the provided link - No issues
- Flashed image to sd-card utilizing balena etcher (1.5.53) - No issues
- Booted the rpi4 - Noticed the “Safe to turn off screen” showed up momentarily, a bit confusing since I just turned it on.
- Wifi - setup instructed to connect to “MycroftOS-Setup” when actually the SSID was “MycroftOS-WiFiSetup”
- Wifi - setup showed some wifi networks as “ID List” and showed both my access points “Kelsey.AI” It did not show any 5Ghz networks.
- Wifi - recommend a “show password” option when entering SSID password.
- Wifi - After setting up the wifi the web browser indicates the device will reboot but the TV showed “safe to reboot”, I was unsure if I should cycle power or wait. I waited.
- Booting - I think a boot progress bar or spinning mycroft logo
would be helpful to know if it is actually booting or frozen. - When connecting mycroft to the mycroft.ai website showing the pairing code on the hdmi splash screen would also simplify the connection process.
- Showing the device IP address on the hdmi splash screen would also be helpful. I had to check my router.
- Paring to mycroft.ai occured without issue once I figured out i had to turn up my tv to hear the pairing code.
- Network Hostname - i noticed my router reported a MycroftOS hostname, I like this, but if I have more than one MycroftOS device what will happen when trying to identify the hostname.
- SSH - Attempted to connect but could not figure out the username / password as there are several listed in the forum topic. I did figure it out eventually.
- SSH - the mycroft-cli-client worked as expected
- SSH - when attempting to install a skill an error was reported with package manager.
- SSH - attempted “msm update” and received merg conflicts (this could be mycroft.ai issue).
- Mycroft.ai websettings skills page - I really like this and wanted to try the snapcast client but was not certain on the instructions required to get this working, maybe a link to snapcast configuration?
- I attempted to install the snapcast client but “dpkg” was not found in the OS.
- At this point my device began to become slugish. I was unable to determine if this was MycroftOS related or a thermal throttling of my rpi4.
- I suspect thermal throttling, and eventually my rpi locked up completely.
- Does the MycroftOS include the latest rpi4 firmware (reduces thermal issues) or should this be installed separately?
- Will the MycroftOS stay up to date with the latest Mycroft.AI builds?
I will resume testing later when I have some time and my rpi4 cools down a bit.
My initial impressions on MycroftOS were very good, and I will definitely consider using this as my “go-to distro” for Mycroft.AI.
I hope these notes are helpful, keep up the good work!
3 Likes
PERFECT FEEDBACK
Thanks for that. At a bar at the moment, but will go through the list soon. First impression is that most of it are “nice to haves”. So very good, as no fundamental errors occured.
Get back to you with a full review a bit later.
Thx again.
1 Like
Have one in celebration of a job well done! 
Alrighty then, let me go throught the different points and comment on them.
-
Good
-
Perfect
-
Right, not sure why you seen the “Safe to turn off” screen as that screen is being pushed to the framebuffer via systemd within the “halt.target” and “poweroff.target” runlevels. So for some reason it did a quick reboot. Perhaps the resizing of the OS over the whole SD card failed in some way. If you have the SD card still at hand, perhaps you can validate this by checking if all space has been utilized.
-
Hmm, yes, see that now. Will update the screen. Made that screen way before the wifi setup system.
5a) I also saw a few funny SSID’s. I am just using the “iw” command to show all networks and then use text line magic to search for all the SSID’s. See code below;def scan_wifi_networks():
iwlist_raw = subprocess.Popen([‘iw’, ‘dev’, ‘ap0’, ‘scan’, ‘ap-force’], stdout=subprocess.PIPE)
ap_list, err = iwlist_raw.communicate()
ap_array = []for line in ap_list.decode('utf-8').rsplit('\n'): if 'SSID' in line: ap_ssid = line[7:] if ap_ssid != '': ap_array.append(ap_ssid) return ap_array
5b)Perhaps I need to dig in a bit deeper and check for type of network as well. About the 5G networks not being shown. At my test bench I only have 2.4G so stupidly didn’t look further. Will have a look, but think I already know what the problem is. I have no country specified within the default wpa_supplicant files. I believe 5G needs to know the country becaquse of the channel / power settings of the wifi chip. Will look at it.
6) Thanks, indeed a simple thing that helps preventing typo’s. I have on the list (way at the bottom though) to check the crendentials before the reboot, but for now that is a easy compromise.
7) Yeah, perhaps the wording should be changed a bit. I thing I will change it to “Safe to reboot / Rebootig now”. While at it, thik I will change the poweroff screen to “Safe to poweroff / Powering off”. It depend a bit on the hardware if it can actually really reboot/poweroff the system. But as we are still only looking at the RPi’s it is not yet a valid point. (I know some Amlogic boards, can’t really power off)
8) Sure, that is already on the nice to have list. At the moment, I just implemented a very, VERY simple splash approach utilizing systemd. At some point I will change this to something more advaced, but still not as advanced as plymounth. Will probably end up with psplash or something.
9) Indeed, on the list but awaiting the way forward from the Mycroft Dev team in combination of the GUI. At the moment there are multiple ging going on. They write the WIFI code raw to the framebuffer. The pairing code is handled by a Skill, but there is still an open ongoing discussion about enclosures/code. I am trying to use as much as possible from the Mycroft Dev team as possible. This makes is look the same, avoids confusion. However the visual part of Mycroft is still a bit early bird.
10) Great idea. For the rest, see point 9. But perhaps we should communicate this one to the Dev team as well for their Mark-2 development.
11) Good.
12) Doh! Never thought about that. Gave it some thoughts at the bar Could be easily solved by adding the last bits of the MAC address to the hostname. Again systemd will help us out and makes it robust.
[Unit]
Description=Set hostname to MycroftOS-[MAC]
Before=systemd-networkd.service
Before=avahi-daemon.service
[Service]
Type=oneshot
ExecStart=/bin/sh -c "hostnamectl set-hostname MycroftOS-`sed 's/://g' /sys/class/net/eth0/address | tail -c 7`"
[Install]
WantedBy=multi-user.target
-
Hmmm, thought the mycroft/mycroft combo was clear. Anyhow, on the list is to have the user force a password if the enable the SSH at the settings page. Then the underlying Skill/Enclosure code will initiate as passwd command before starting the SSH daemon. Open network protocols with default credentials are a no go if you ask me. Anyhow, will look at it when I make SSH disabled by default.
-
Nice
-
Indeed, I need to have a look at MSM code and see what package managers it is looking for. Than either need to patch the code or mimic an empty package manager. Buildroot does not have a package manager. Need to give this some thoughts to find the right decision.
-
That is done intentionally. Still working on the mycroftos enclosure skill. Only added the skill in this release for the Framebuffer visuals. When the whole Skill is done, I will make it install properly so it can be updated etc. For the volume skill, I included a patched version where I added “MycroftOS” to the ALSA_PLATFORMS array. This together with the pushing e erything from ALSO to pulse give me a quick fix for handling the volume. The mycrofos-enclosure skill it the place where I will and need to integarte the PulseAudio volume control code. This should inlcude; Increase/Decrease volume, muting, but also switching sinks, perhapos seperate sink volume control, seperate sink muting. This figuring that stuff out in my head. Most likely also need some changes to CPS or Audio service. All audio is forced into pulseaudio and as this is a embedded system only used for mycroft, we should allow Mycroft to control all the sinks of pulseaudio. So when we start playing something via spotify connect on the device, we should be able to say “Hey MYcroft, stop” and it should just stopp all playback, including the ones that aren’t even initiated by Mycroft itself.
-
Yeah, the snapcast and mpd needs to be added to the settings page. Not 100% sure how though…
-
Snapcast client is already installed. Just enable it via the settings page and it will be started and started at boot because I use systemd for it.
[Unit] Description=Snapcast client Documentation=man:snapclient(1) Wants=avahi-daemon.service After=network.target time-sync.target sound.target avahi-daemon.service [Service] EnvironmentFile=-/etc/snapcast/snapclient ExecStart=/usr/bin/snapclient $SNAPCLIENT_OPTS StandardOutput=null Restart=on-failure [Install] WantedBy=multi-user.target -
I will investigate and think I know what it is. Indeed overheating and downthrotling. I have set the CPU govenor to performance. Precise is asking quite a lot of the RPI3, so the CPU goes up and down. I could have changed the on-demand threshold, but in combination with precise that is about the same as going ful lspeed all the time anyway. BUT, we might not need this for the RPI4 as it is a bit faster. Will see if I can make it automaticall; performance for rpi3, on-demand for rpi4.
-
Agree, see 19.
-
Not sure, need to read into it. Will get back to this.
-
In the end; YES. At the moment NO, not without flashing again. I have OTA firmware upgrade system on the list. Either RAUC (what HassOS uses) or SWupdate. Both systems can be use to upgrade the system via USB, a webinterface or could poll a remote server for updates. All user data will be moved to /opt so it will survive upgrades. Anyhow, will only startdeveloping this when I reach some sort of “final” first release as things keeps changing a a regular basis at the moment. But already doing my homework for it
Once again, perfect feedback. You really approached it with a (dummy) kind of consumer eye. This type of reports, makes something going from 90% done to 100% That last 10% are always the most difficult in a project.
Thanks again!
1 Like
It has been a while since I was able to really work on the project, but as Corona hits us all and we have to work from home, I can multitask a bit in between the daytime job task.
Gathered all my different (sticky) notes into a proper task list again. For the people that are interested or are or want to be involved, see below (Let me know if I missed something)
MycroftOS - Version 0.1.0 - alpha 8
ToDo:
- Ondemand governor for both RPI3 and 4 (instead of permormance), most likely need some tweaking in the threshold
- Fix: MPV not compiling because of missing LD stuff (disabled for now) [WIP]
- Update to latest LTS kernel and drivers / firmwares [WIP]
- Update to latest LTS buildroot (20.02.x) [WIP]
- Update to latest Mycroft version and dependencies [WIP]
- Addition of additional packages for the near future [WIP]
- OpenCV
- Python-Numpy
-
Motion[DONE] -
Mosquitto[DONE]
- Change WifiSetup splash; MycroftOS-Setup -> MycroftOS-WiFiSetup
- Have a look at the wifi scan to exclude non real SSID (“ID List”)
- Add 5G support to the wifi system (Country setting most likely)
- Figure out how to make the wifi password box to be able to show password (eye icon)
- Change the wording on the splash screens;
- Safe to reboot -> Safe to reboot / Rebootig now
- Safe to poweroff -> Safe to poweroff / Powering off
- Look into animating the boot splash screen, showing progress during boot
- Implement initial framebuffer drawing GUI
- Look into showing the pairing code on the HDMI as well (look at mycroft’s system)
- Look into showing the IP address on the HDMI as well (look at mycroft’s system)
- Look into combining the wifi code, the framebuffer code and the enclosure code into one system.
- Add the last bits of the MAC address to the hostname
- Include changing the hostname from the MycroftOS settings skill without messing up above MAC addition.
- Look into the msm error/warning “no package manager found” (there is none, but just handle the warning)
- Included volume skill can’t be updated
- Should be changed to pulseaudio anyway, but…
- Check if the included volume-skill can be updated, via offical git commits
- Avahi zero config publish to be set up (pulseaudio, mpd, snapcast, etc.)
- MycroftOS settings skill to be able to setup smart speaker services / software
- VLC Framebuffer video playing support (if not accelerated, check omxplayer but needs to be created as audio backend within mycroft)
- Add and make configurable, where sound output has to go;
- BT Speaker (A2DP) normal (Use a bluetooth speaker as output)
- Bluetooth mic/speaker combo support using HSP (use headsets as input/output for Mycroft)
- Airplay (example: SONOS) using module-raop-discover and module-raop-sink of PulseAudio
- Autoconnect trusted BT devices as soon as it sees them using: module-switch-on-connect from PulseAudio
- Combined: Create a virtual pulseaudio sink to send sound to multiple output sinks/devices
- Change README with all new changes
- Update / Complete “documentation” folder and docs
MycroftOS - Version 0.1.0 - beta 1
ToDo:
- Build precise from source at build time (0.3.0 dev version)
- create buildroot packages for all dependencies.
- create buildroot package for precise-engine & runner
- create buildroot package for wake word modules (including the commodity ones)
- figure out how to give back to the communicty by allowing recording of wake words easily
- Fix / Check all Buildroot package dependencies (SELECT and DEPENDS references within Config.in)
- Cleanup buildroot config file
- remove / disable any unused packages that might have slipped in while testing / debugging / etc.
- Make sure all Skills from the market can install (python dependencies either available or pre-installed)
- Enclosure code for enabling services, halt, reboot, etc
- Enclosure code for pulseaudio control (volume mostly)
- More hardware support
- At least the Google-AIY
- PS3 Eye [WIP (ordered one)]
- Systemd notify and Watchdog support to make starting/stopping/restarting and failures way more robust.
- ReSpeaker Mycroft LED pattern which is blue-isch and looks like the logo of Mycroft
MycroftOS - Version 1.0.0 - alpha 1
ToDo:
- 64bit support for RPI3 and RPI4
- Cleanup buildroot:
- rootfs default overlay seperate
- device specific overlays
- Cleanup the kernel config
- Remove unused drivers and stuff (DVB and such)
- Separate default config into systeemwide .config
- Board specific configs as overlays
- Support of OVA (Virtual system - Virtualbox)
- Implement easy (OTA) update system (RAUC or SWUpdate, not sure yet)
- Add an first initial GUI system using Kivy, just as Mycroft.ai
- Implement first initial WEB frontend/backend system for configuration and all
- Backend for settings, configuration and update
- Backend pulseaudio control (Already available project as quick fix, but needs to integrate with MycroftOS system)
- Frontend for possibly accelerated browser hooking into the Mycroft GUI
- Support for more devices
- Rockchip boards
- ReSpeaker Pro V2
- Odroid
- X86 / 64
- etc.
2 Likes
Quick double post to let you know, the above list has been updated with more stuff from saved bookmarks, notes, investigations.
I know a long list and it keeps getting longer. But it kind of show cases how I see an end-consumer type of Mycroft device
1 Like
Looking forward to doing some more testing, let me know if you need anything specific tested, I may have some time on my hands in the next week or so.
1 Like