Introducing Mimic 3

Mimic
#11

That’s what I would have meant if I’d been paying attention :stuck_out_tongue:

So it seems the Docker image may be the only option unless you want to run Mimic 3 remotely on a different machine. If you do that, you can just use Mycroft’s MaryTTS plugin to connect to it.

#12

I have the same error. I set a speaker in the user config as suggested, but the only difference is that mycroft falls back to mimic1 instead of staying silent

#13

Hello,
i can comfirm this. I set the speaker in config like described in the documentation.

#14

Hi I have setup mimic3 using the guide, It all setup. The speaker works the microphone works but it wont speak to me. I set it up using the plugin and have a raspberry pi 4 setup using the raspbian os with mycroft installed. I don’t know where to get logs from. Thanks in advanced

#15

Any errors should be in /var/log/mycroft/audio.log

A common problem is using an outdated pip before installing the plugin. There is some dependency-of-a-dependency problem with the dateparser and regex package. After upgrading pip, you may need to do pip install --upgrade regex as well.

#16

Thanks for all the hints, guys! Using the plugin, I could make my Mycroft Linux Mint test install work with Mimic3—works a treat!

Here’s my mycroft.conf:

{
  "max_allowed_core_version": 21.2,

  "listener": {
    "wake_word": "hey computer"
  },

  "hotwords": {
    "hey computer": {
        "module": "pocketsphinx",
        "phonemes": "HH EY . K AH M P Y UW T ER .",
        "threshold": 1e-90,
        "lang": "en-us"
    }
  },

  "tts": {
    "module": "mimic3_tts_plug",
    "mimic3_tts_plug": {
        "voice": "en_US/cmu-arctic_low",
        "speaker": "rms",
        "length_scale": 0.8,
        "noise_scale": 0.667,
        "noise_w": 0.8
    }
  }
}
2 Likes
#17

I’d really love to see Brazilian Portuguese supported.

#18

If you know of a good dataset, I’d happily train a voice :slight_smile:

#19

I’ve not even a clue where or how I’d search for that. Any ideas?

#20

https://edresson.github.io/TTS-Portuguese-Corpus/

According to this comment from the GitHub repo it is Brazilian Portugese.

1 Like
#21

@synesthesiam said previously in a Rhasspy forum thread he tried that one before and the quality isn’t good…

It was, but people told me that the voice I trained wasn’t understandable. I used this dataset: https://github.com/Edresson/TTS-Portuguese-Corpus

Do you know of any other TTS Portuguese datasets?

edit:
Listening to the recordings resulting from the TTS experments on this page TTS-Portuguese Corpus In my opinion the audio file results from of the Portuguese TTS is perfectly fine and understandable - the results of experiment #1 and #3 on that page. Experiment #2 was also understandable but had added noise distortion.
For example this wav file result of the longest phrase from Experiment #3 is very good and highly understandablef:
Hoje é fundamental encontrar a razão da existência humana
So @synesthesiam I wonder why when you used this data set your results were not understandable? The above results are pretty much the same quality as you get when using the Google Translate page to generate Portuguese TTS audio. It’s good!

  • Experiment 1 uses the DCTTS model, trained in the TTS-Portuguese Corpus, and vocoder RTISI-LA (Good).
  • Experiment 2 uses the Tacotron 1 model, trained in the TTS-Portuguese Corpus (Bad)
  • Experiment 3 this experiment explores the use of the TTS Mozilla model, trained in the TTS-Portuguese Corpus (Very Good)
#22

Interesting results. Edresson is a knowledgable person when it comes to TTS, e.g. see his work on YourTTS.

@synesthesiam Does it make sense if I apply my audio preprocessing chain (that I have used for Thorsten-DE) to the TTS-Portugese-Corpus?

#23

Looks like he’s contributing his TTS knowledge to the Coqui.ai project.

#25

I wonder if I need to just train a model directly on characters rather than trying to use a phonemizer. My most recent attempt in Mimic 3 used the pt-br voice from espeak-ng.

Looking at Edresson’s model config, his audio settings are a bit different from mine. For example, his sample rate is 20000 instead of 22050, and he’s using “preemphasis” which appears to filter the audio before training. So maybe the problem is my naive use of the data directly without enough preprocessing?

#26

Hi, sorry for bothering you. Do you have the plan for releasing the training source code yet? Thank you, looking forward to your information.

#27

I’ve published a video on my Youtube channel showing all ways to install/run Mimic 3 and first steps to synthesize audio by CLI oder local WebUI :slight_smile:.

Just in case it’s interesting for you.

2 Likes
#28

Hi @rostom132, no bother :slight_smile:
I do have a plan, but not definite release date yet. The training software as it is right now is the result of a year and a half of experimentation, including a lot of dead-ends. I’m cleaning it up now and removing a lot of the unused code. My hope is to make it work closely with Mimic Studio.

1 Like
#29

Thanks for the awesome video, @Thorsten! I was very happy to see all the installation methods worked out well :slight_smile:

For anyone curious about the SSML volume, it currently just goes from 0-100%. Something I need definitely need to fix :+1:

1 Like
#30

You’re welcome @synesthesiam :slight_smile:

In general a volume value between 0 and 100 makes sense, but just in case you want your TTS voice “yelling” at you a higher value could make sense too :wink:.

1 Like
#31

Hello,
My final goal would be to record enough audio for slovak voice and then later on use espeak-ng phonemizer with that recording to train the voice.
Slovak is somewhat similar to czech and given there are some czech recordings available, perhaps I can first get to understand the whole process withese czech data and then move on to record slovak recordings.
There are some czech voices for festival with available recordings.
For example here is the list of words: voice-czech-ph/words at master · brailcom/voice-czech-ph · GitHub
And here are the actual recordings: voice-czech-machac/wav at master · brailcom/voice-czech-machac · GitHub
These are so called diphone voices for festival so these recordings include list of words where each word features concrete preselected syllable used for training.
Would it be doable and does it make sense to use these recorded words for training mimic3 czech voice? Is likelly to provide better results than festival?

If I manage to record or otherwise source reasonable number of recordings for slovak voice let’s say a few hours will I be able to train the voice on my own using my laptop or my computer or does it need much more power?
I see how LJ speech or thorsten german recordings are structured. Should I move on to recording or am I supposed to know how it all works before recording? In other words does my experiment with existing czech recordings make sense?

Greetings

Peter

1 Like