So, I've been experimenting with AI voice tracking, using the Kokoro engine. I have to admit that the generated voice sounds more natural and expressive than others. However, without a lot of context, the engine had trouble with proper names and short phrases. And, it didn't have a dictionary, so you had to manually correct each and every flub, which was painful.
Although it wasn't readily apparent during my testing through speakers, when listened to with headphones, the generated voice track tended to speed up at the end, sound a little bit chipmunk'ish. Almost as if it wanted the track to end quickly. Some were worse than others, but if you listened closely, the effect was always there.
Needless to say, I went to the old way of doing things (i.e., a non neural-network approach). Luckily, most of the work done was determining the public domain status of a song. I'm not going to go through and attempt to redo the tracks using Kokoro, as there were just too many fixes required.
Next, I did some research on denoising audio tracks, specifically OTR, spurred on by @richpowers talk of the work done on Quiet Please. I attempted to use various online AI engines to fix up some of those episodes. They did too good a job - the voices were left there, but the background music mostly disappeared. I know there are tools 'out there' that will work, but the work would certainly not be fully automated. I need to do a bunch of audio tracks automatically and quickly to improve the sound, not work for hours on a single track to get it as perfect as you can get.
To put it bluntly, I'm not impressed. Maybe I didn't use the right engine(s), but there appears to be no magic solution to this without a lot of painstaking and time consuming work.
Finally, I tried out some of the AI engines in an attempt to get a few radio jingles. Suno.ai seemed to be the best, at least for what I wanted to do. They give you 10 free radio 'songs' per day. And I needed all 10 for the past week to generate something approaching what I envisioned.
Again, there were problems. It the AI didn't recognize a word (i.e., "synchopated", which describes the music of the early 20th century), it just skipped over it. Lots of time, it would cut off words at the end - that appeared to be random. The process is certainly not perfect.
But I have to admit, I got a few jingles that were acceptable.
These AI engines are best when you confine the domain in which they work very narrowly (speech is difficult, particularly English, as there are a myriad of definitions and corresponding pronunciations for many of the same words, depending on context and meaning).
I had one thought at the end of all this. The music in the jingles I created are supposedly royalty free, but it sounds very similar to other, perhaps copyrighted, music of that era. The AI engine has likely been trained with a lot of that music. I'm wondering when someone who has the rights to a song sues an AI engine developer for infringing on their copyright. Maybe it's already happening or happened.
Addendum. Well, I guess I'm out of touch with what's happening. There are tons of lawsuits going on against AI engine developers alleging mass copyright infringement of their output, never mind using copyrighted material for training. And in the U.S., at least currently, you cannot copyright the output of an AI. Of course, that's only if you admit that what you attempt to copyright was the output of an AI. What a mess.
... Next, I did some research on denoising audio tracks, specifically OTR, spurred on by @richpowers talk of the work done on Quiet Please. I attempted to use various online AI engines to fix up some of those episodes. They did too good a job - the voices were left there, but the background music mostly disappeared. I know there are tools 'out there' that will work, but the work would certainly not be fully automated. I need to do a bunch of audio tracks automatically and quickly to improve the sound, not work for hours on a single track to get it as perfect as you can get.
To put it bluntly, I'm not impressed. Maybe I didn't use the right engine(s), but there appears to be no magic solution to this without a lot of painstaking and time consuming work. ...
It's been years since I messed about with trying to "restore" heavily damaged OTR (specifically the QP episodes), no there is definitely no "magic solutions" out there, the most effective option you have - is exactly as you say; "a lot of painstaking and time consuming work.". I found Isotopes (Izotropes?) software was a bit more capable than Audacity, but not particularly so much better that it's worth the cost of their software. I found that with clicks and artifact removal, the most effective method is by manual second by second work, closely selecting each individual flaw and "pasting in" clean audio "into" each individual flaw selection from a fraction of a second of an area in audio that's directly next to it that is clean - that way absolutely no time shift occurs and it doesn't much matter if it's voice, music, or a silent portion of the audio that's being cleaned, it's effective in all respects and you lose nothing whatsoever of it's audio quality the files has already. That's what gives you the best results - the drawback is that it takes about an hour to clean about 10 seconds of audio! But it provides the best final outcome. Whereas if you try automated click removable it's often going to deteriorate the audio it's trying to "fix" even further.
Hiss and roar cleaning is vastly more problematic than clicks and pops -- I found no effective method of cleaning it without muffling the crap out of the audio quality at the same time.. And that's where that guy on YouTube Quiet Please episodes comes in..
Because I was already specifically familiar with numerous individual QP episode files and what parts of them had the most difficult audio problem, I instantly recognized where he had achieved major improvement in some of QPs most damaged audio that I had tried for months, repeatably, in the past attempting to correct with out success -- And I was like "how the hell did he do that?". So I asked and he said it was mostly "CapCap" which I had never heard of. -- No, it's not magic fix, his versions still need lots of work imo, but there better than most any other versions available out there. Still it's obvious to me that he just kind of pressed a CapCap button and it did most of the work for him, but a using combination of minute manual work and CapCap together could probably work wonders. --
Finally, I tried out some of the AI engines in an attempt to get a few radio jingles. Suno.ai seemed to be the best, at least for what I wanted to do. They give you 10 free radio 'songs' per day. And I needed all 10 for the past week to generate something approaching what I envisioned. ...
Suno probably is the best, but to clarify; you get 10 free 'generations' per day, not necessarily 'songs' - it's very common for me to use 8, 9, 10, or a dozen generations on a single song, occasionally even more when it just isn't giving me what I want.. For example you mentioned when it mispronounces words, or you called for a male voice to say whatever but it spits out a female, or you wanted the tempo to ease up right here and it instead intensifies .. so you regenerate that particular area .. Yeah, for me one song most often requires at least 5 or 6 generations to accomplish whatever the objective is, and even then won't really get exactly what you were aiming for, but rather you get something close to it.
That's why I upgraded to the $10 (or is it $8?) a month plan, not only does it provide ample credits, but it also gives you the legal rights to do whatever you want with your creations. With the free plan you can only use the outputs for "personal use" and you don't own any of your creations, Suno does.
I had one thought at the end of all this. The music in the jingles I created are supposedly royalty free, but it sounds very similar to other, perhaps copyrighted, music of that era. The AI engine has likely been trained with a lot of that music. I'm wondering when someone who has the rights to a song sues an AI engine developer for infringing on their copyright. Maybe it's already happening or happened.
.... And in the U.S., at least currently, you cannot copyright the output of an AI. Of course, that's only if you admit that what you attempt to copyright was the output of an AI. What a mess.
That's not quite accurate, you can and people do and have copyrighted their AI works.. but yeah it's a mess because apparently.. well, it's not uncommon that for example YouTube engines often flag AI works with copyright infringement - it's all a malformed mess.
I've watched several of Krystle Delgado YouTube videos on her "Top Music Attorney" channel. In one of her videos she takes you step by step on how to copyright your ai creation through the governments copyright office website. She's specializes on the legalities and hoops of AI Music creation. https://www.topmusicattorney.com/ I find here videos very well done and she explains and spells out thing very well and clearly illustrates just how insane, complicated and convoluted AI music legalities can be.
I was going to talk about how when Udio and Suno both got sued for the same thing and the same time and while Udio kind of lost in their deal yet Suno got an entirely different deal and came out way up on top. I originally liked Udio better, but when they lost their suit its ultimate result was that you can still create on their platform but you can't download your own creations! - What's the point in paying for that? That's why I went back to Suno.
@artisan-radio Have you checked this out? Below some copy and pasted key points:
https://techcrunch.com/2026/03/26/mistral-releases-a-new-open-source-model-for-speech-generation/
French AI company Mistral released a new open source text-to-speech model on Thursday that can be used by voice AI assistants or in enterprise use cases like customer support. ... the company wanted the model to sound human and not robotic... The model has been built for real-time performance, .... an end-to-end agentic system that supports audio as an input or output,” Stock said. ... Mistral’s positioning is that its open source and customization bit will help enterprises adopt its voice models over competitors, as they can tune it the way they want. ....
To get Mistral’s new Voxtral TTS model, you can choose between running it locally using open weights or accessing it through Mistral's hosted platforms.
1. Download Open Weights (Local Deployment)
The model is available as "open-weight," meaning you can download and run it on your own hardware. .. under a CC BY-NC 4.0 license.
https://huggingface.co/mistralai/Voxtral-4B-TTS-2603
Requirements: The 4B model is lightweight and can run on most modern consumer GPUs ... requiring approximately 3GB of RAM when quantized.
To actually generate speech locally, you'll need a compatible inference engine like
vLLM ...
If you prefer not to manage hardware, you can use Mistral’s managed services, you can test the model in the Mistral Studio playground. It is also available via API for integration into your own apps at a rate of $0.016 per 1k characters.
Mistral is rolling out voice capabilities directly in Le Chat, allowing you to interact with the model via a browser or phone.
For developers building complex agents, Voxtral TTS is already being integrated into open-source frameworks like Vision Agents and Open WebUI, which provide pre-built interfaces for the model.
...
I'll check it out, but I'm happy with what I'm using right now