Mycroft Community Forum

Mycroft and ducking pulseaudio

so I have enabled two features in the config:

“pulse_duck”: true
“duck_while_listening” : 1.0

I somehow don’t understand what happens. So when I say Hy Mycroft it does not duck the currently played music. Even while listening to my question or order it does not duck the music. However once mycroft gives the answers it does duck the music.

How can I duck the whole process from the beginning of recognizing the wake word till the answer has been given?

It must be a Mycroft problem since the pulseaudio stream seems correctly configured (see music and phone Icon):

EDIT:
When I edit the https://github.com/MycroftAI/mycroft-core/blob/bcb426ab6980a5c221a963ca44bb3c1c88791771/mycroft/util/audio_utils.py#L64
File and change it to “phone” instead of “music” it will at least also duck when the “i’m listening now” sound from mycroft is played. but still does not duck while listening to my question…

Those looks like the old ducking controls. Sadly those didn’t work reliably (streams that were ducked could stay ducked if they exited while ducking). What should work is TTS ducking, while mycroft is speaking.

"tts": {
  "pulse_duck": true
} 

Should enable that part. Ducking while listening I think was removed back in 19.08.

The environment you changed to phone should be music so the TTS can duck media played by Mycroft skills.

I had a hack that implemented ducking while listening but it was a bit dodgy so it was never included.

hmm ok… It would be a great feature… So I need to do it by myself by controlling the audio player that mostly just plays through this audio sink…

can report a ducking problem, too. (with only ```“pulse_duck”: true`)

Not only he stays ducked despite

 18:56:54.971 | INFO     |   790 | __main__:handle_wakeword:67 | Wakeword Detected: hey-mycroft
 18:56:55.443 | DEBUG    |   784 | VolumeSkill | MUTING!
 18:56:55.444 | DEBUG    |   784 | VolumeSkill | Finding Alsa Mixer for control...
 18:56:55.429 | INFO     |   790 | __main__:handle_record_begin:37 | Begin Recording...
 18:56:55.477 | DEBUG    |   784 | VolumeSkill | Volume before mute: 75
 18:56:55.478 | DEBUG    |   784 | VolumeSkill | 75
 18:56:55.478 | DEBUG    |   784 | VolumeSkill | 0
 18:56:57.797 | DEBUG    |   784 | mycroft.skills.settings:download:337 | No skill settings changes since last download
 18:56:59.515 | DEBUG    |   784 | VolumeSkill | Finding Alsa Mixer for control...
 18:56:59.555 | DEBUG    |   784 | VolumeSkill | 75

VLC (used by the skill tunein in this example) throwing an error

18:57:00.922 | DEBUG    |   784 | mycroft-skill-tunein_johnbartkiw:find_station:122 | aliased search_term: antenne bayern lounge
 18:57:01.177 | INFO     |   784 | mycroft.audio.services.vlc:stop:79 | VLCService Stop
[ade4c958] http stream error: local stream 1 error: Cancellation (0x8)
[adf0c290] http stream error: local stream 1 error: Cancellation (0x8)
[adf94738] prefetch stream error: unimplemented query (264) in control

I can definitely say that this has to do with pulse_duck, because i’m switching back and forth trying to find the culprit.

Does this have something to do with this? Where is the vlc configuration stored?
Or is this just an aftereffect?

I get the impression that he’s not muting after hearing the wakeword, he is silenced to begin with (Beep should’t be muted)

cough Or maybe not. Made the error to add high-priority = yes
to the daemon.conf. module documentation
(“The decision whether a stream has high or low priority is made based on the stream role”)

A comment on the feature: The ducking (if you listen to a radio stream for instance) is a little late and transitions into the text read (the first second or so)

1 Like

Still had problems with the ducking… sometimes. News, Emby, much less tunein not recovering after some tts action. And i can’t make a rhyme out of it since the whole audio thing is a big black box to me.

So, desperate times need desperate measures, i thought to myself… The Volume skill has an option “ducking while listening”. Since i turned that off things works smoothly (not tested long-term), still ducking due to conf settings when i want him to duck (tts, speech input; given that the backend option syncronized properly. Although i don’t get why he is ducking at speech input since ducking is only mentioned in the {“tts”:…} section )

Is he performing some double duck here :duck: :duck: that kicks s*** south?

There is a “pause” feature that is always active with the “simple backend” so when Mycroft is listening the audio playback is paused.

I’ve seen it stick muted when using module-role-cork, if you’ve got that module loaded try replacing it with module-role-ducking

I’ve unloaded module-role-cork as i set up the ducking module, as suggested.

Is there a difference between loading and then unloading a module on the one hand and commenting out said module on the other? I’ve experienced some unexpected silence as i commented out 2 and 3 instead of

load-module module-role-ducking
unload-module module-role-cork
unload-module module-suspend-on-idle

That should be the same result :thinking:

Gonna setup ducking here and see if I can replicate it…

This might be interesting in that regard:

He’s not able to restore the volume and stays dark after this

2020-09-08 11:56:37.388 | INFO     |   801 | audioservice_vlc:stop:79 | VLCService Stop
2020-09-08 11:56:43.634 | ERROR    |   801 | concurrent.futures | exception calling callback for <Future at 0xa07f2f90 sta                                                                                     te=finished raised AttributeError>
Traceback (most recent call last):
  File "/usr/lib/python3.7/concurrent/futures/_base.py", line 324, in _invoke_callbacks
    callback(self)
  File "/home/pi/mycroft-core/.venv/lib/python3.7/site-packages/pyee/_executor.py", line 60, in _callback
    self.emit('error', exc)
  File "/home/pi/mycroft-core/.venv/lib/python3.7/site-packages/pyee/_base.py", line 111, in emit
    self._emit_handle_potential_error(event, args[0] if args else None)
  File "/home/pi/mycroft-core/.venv/lib/python3.7/site-packages/pyee/_base.py", line 83, in _emit_handle_potential_error
    raise error
  File "/usr/lib/python3.7/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/home/pi/mycroft-core/mycroft/audio/audioservice.py", line 338, in _restore_volume_after_record
    restore_volume()
  File "/home/pi/mycroft-core/mycroft/audio/audioservice.py", line 330, in restore_volume
    self.current.restore_volume()
AttributeError: 'NoneType' object has no attribute 'restore_volume'
2020-09-08 11:57:06.742 | INFO     |   801 | mycroft.audio.speech:mute_and_speak:127 | Speak: Spiele jetzt Station ANTENNE                                                                                      BAYERN Lounge von Tune in

Bit of context: This happened after stopping the stream from emby and calling for a stream “ANTENNE BAYERN LOUNGE” from tunein

I got from the code that this shouldn’t be an issue since logically there shouldn’t be a stream present at that time - in my case - so nothing to restore. BUT is the follow up stream “lounge” not present at that time?

But since this is only a Volume thing, i should be able to set this via speach. Which i can’t.

Another thing that’s bothering me is this line here. Is he mixing things up here?

BTW, if someone wants to reproduce this, this is with the audio backend pushed to “vlc”

  "Audio": {
    "backends": {
      "local": {
        "type": "simple",
        "active": true
      },
      "vlc": {
        "type": "vlc",
        "active": true,
        "duck": true
      }
    },
    "default-backend": "vlc"
  },

Thanks for pointing to the vlc backend. I’ve been trying with the “simple” backend so far. Have you tried to disable the vlc backend ducking (and just using the pulse ducking for tts)?

I’ve gone the MycroftOS way of dealing with streams: Pulses’ module combine_sinks.

That setup seems stable in regards of my broken (or misguided) stream problem. And the streams seem to duck just right, instead of warping into each other with the ducking one respond too late (most of the time).

I still consider this as a kind of bandaid, yet remedied my problem that i dragged over several weeks with no conclusion.

Disableing vlc ducking wouldn’t have been an option since i want my node-red reporting my sensory stuff as they trigger (with music playing most of the time)

Another thing: After applying the MycroftOS approach (combine_sinks) i realized that due to vlc pre-flags cork i’ve ended up with a corked (Filter START_CORKED; to 40% vol) stream (despite unload cork on my end), which was overcome by commenting out load-module module-stream-restore. Also, the stream is sent with a video role applied.
Maybe this is throwing things off, so maybe a useful lead

You could be on to something;

I am not a Python dev, but somethings I read as, the stream should be “music” to be able to be ducked.

This should pass the role:music per the deepcopied _ENVIRONMENT which is chosen (_ENVIRONMENT or os.environ) if the config holds pulse_duck: True
and got called by _play_cmd (play_wav/play_ogg/play_mp3)

Either this (the call) is not the case in regard of vlc processing or isn’t working at all (processing with simple suggest otherwise), since the role remains video.

That said, i wouldn’t know why this should be overcome by a combined sink. Makes no sense to me. The bigger lead in my estimation is that vlc sets cork=true (which means asks for being muted and paused) and the stream arriving at a muted state from tts processing resulting in a paused stream.

From https://www.videolan.org/developers/vlc/modules/audio_output/pulse.c

pa_stream_flags_t flags = sys->flags_force
                            | PA_STREAM_START_CORKED
                            | PA_STREAM_INTERPOLATE_TIMING
                            | PA_STREAM_NOT_MONOTONIC
                            | PA_STREAM_AUTO_TIMING_UPDATE
                            | PA_STREAM_FIX_RATE;

BTW the whole shebang seems not necessary with MycroftOS (rpi4 64-Alpha9). I suspect the config entry "listener": { "mute_during_output": false, ...
to circumvent the vlc problem. (besides the pulse setup)

1 Like

As I recall from way back the Pulse audio role ducking by default only ducks the music stream. That’s why it’s set there so skills and audioservice using those commands can be ducked. VLC should play using the “music” pulse role (atleast on my setup).

There is some pulseaudio config for ducking all roles but I have not been able to get that to work.

Would be interesting to see your config. I would like to revisit this at some point and see if a good simple (and stable) setup can be achieved.

I might have a proper look at PulseAudio as well. Considering the GUI and therefor video playback being present, only ducking audio roles doesn’t cut it all aymore.

Will try to reproduce this using VLC and the Youtube (video) skill and report back.