The main issue is that there was no sniffling symbol in the transcript. And the generated text wouldn't contain it either, because (thankfully) they are pruned out of written interviews that I used to train the model.
Thanks for the explanation. I had some assumptions but wasn’t totally sure how this was trained.
How would you make it sniffle in a natural way, too? It’s not a usual speech mannerism, and the way he does it is distinct. I wouldn’t know how to efficiently represent it with text. Maybe it’s easier than I’m imagining.
The TTS model is trained on two things: speech samples and their transcript. If you add enough sniffle-symbols every time a sniffle appears in the speech, I am confident the model would pick up on that. And then you would be able to replicate a sniffle in the generation part. The more time-consuming bit would be to add in the training data for the language model those sniffle-symbols, so that they would be organically added in the text in the text-generation phase.
But seriously, it's not worth it. I think he's a brilliant man with an idiosyncratic speech, let's leave it to that.
I agree, I personally don't hear his sniffles when I'm listening to him intently. It's irrelevant. I was mostly curious if and how, generally speaking, a model could be trained to sniffle. Now that you describe it though it seems fairly clear, so thanks!
