52 AI Experiments

Mon Mar 17 - Written by: Brendan McNulty

Automation Creativity

Week 11: Using AI to Clone My Voice

AI Voice Cloning Illustration

The Experiment

After weeks of creating AI content, I had an idea: why not turn it all into a podcast? But being allergic to extra work (and recording my own voice), I wondered if AI could handle the heavy lifting. The goal was simple: train an AI on my voice, feed it my written content, and have it automatically create and publish podcast episodes. A hands-free content empire! What could go wrong?

The Process

Here’s what I tried:

  1. Setting Up the Infrastructure
  • Created a podcast page on Podbean (free hosting and monetization)
  • Got the URL ready for my future audio empire
  • Planned a workflow: Write content → AI voice creation → Automatic posting

AI Voice Cloning Illustration

2.Training My Digital Twin

  • Chose 11labs (I’d used it before for some quick and dirty YouTube voiceovers for Now Novel)

  • Went with the $5 package (keeping it lean!)

  • Spent 25 minutes reading an article to train the AI on my voice

  1. The Moment of Truth
  • Listened to my AI twin
  • Discovered it sounded like me… if I’d grown up in America
  • Found the voice wasn’t me, despite what I thought was my energetic training data

The Outcome

Well, this was OK… My AI voice clone had two major issues:

  • An inexplicable American accent (despite my decidedly non-American voice)
  • The delivery was OK, but didn’t sound like me.

There is a more expensive “Creator Pack” that might yield better results, but after hearing my American doppelganger chat, I lost enthusiasm for the whole project. I’ve got one lonely test episode (of this newsletter) sitting on Podbean, a testament to my brief flirtation with automated podcasting. Listen to my weird twin here.

Overall, I don’t think the voice is too bad, it just doesn’t sound exactly like me, so the mimicry is a bit off…

Key Takeaway

While AI voice cloning has potential, it’s not quite ready for prime time—at least not at the budget level. 11labs doesn’t seem to have improved much in the last year since I last used it. The technology needs to better capture not just the voice, but the personality and energy that makes human speech engaging (or maybe I’m more of a damp squib than I thought ).

I listened to someone else’s podcast who used the pricier 11labs package (which TBH is not expensive) with a much better correlation to their voice, so maybe I am a damp squib :)

Pro Tips for Beginners:

  1. Start Small: Test with the basic package before investing in premium options
  2. Consider Alternatives: Tools like Descript might be worth exploring if you’re serious about podcasting
  3. Train correctly: Use a good microphone, get a good selection of training data

Want to Try It Yourself?

  • 11labs offers a $5 package to test the waters
  • Podbean provides free podcast hosting to get started
  • Record at least 20-25 minutes of clear, energetic speech for training
  • Be prepared for some discrepancy

What’s Next?

I might give Descript a try in the future, or wait for the technology to improve. For now, my podcast empire will have to wait. Maybe I’ll stick to writing and let the robots handle the simpler tasks.