ElevenLabs and ethics of genAI for music
Summary of ElevenLabs voice cloning and music generation. Includes some product announcements, key features, and a few insights on ethics of genAI for music. Supports the Unfair Use? series, PART 3.
This post on ElevenLabs supports PART 3 in our 8-part series on ethics of generative AI for music, announced in this INTRODUCTION post on 6 ‘P's in AI Pods. Subscribe to be notified when new articles are published (it’s FREE!)
This article is not a substitute for legal advice and is meant for general information only.
ElevenLabs
Description from site: “We're building AI to shape the future of digital communication, ensuring content is understood by everyone, everywhere.”
Description from PitchBook: “Developer of an artificial intelligence voice dubbing tool designed to automatically dub videos and podcasts into other languages. The company's tool allows users to automatically convert speech from one language to another while preserving the original voice and emotions, enabling businesses to automatically publish their videos in other languages.”
ElevenLabs was founded in 2022, headquartered in NYC. They are privately held (VC-backed), with ~115 employees. Latest financing round in Jan. 2024 was $2.6m. Andreessen Horowitz holds a board seat, along with the two co-founders.
According to this LinkedIn post by Isha Marathe, ElevenLabs is “part of the Disney Accelerator”. A company blog post on Feb. 22, 2024 confirms this. ElevenLabs announced a partnership with Audacy on April 12, 2024.
Earlier this year, I had included ElevenLabs in two articles as an example of an ethical voice cloning provider (voice cloning tool providers, voiceover tool evaluation). I even selected ElevenLabs as my preferred ‘shoestring’ tool for adding voiceovers to my posts in future.
Around that time, ElevenLabs expanded into using generative AI for sound (site, blog) and music (YouTube, Tom’s Guide ref). The quality of their 3-minute songs was touted as better than Suno or Udio (e.g., Alex Banks on LinkedIn). However, ElevenLabs’ reticence about data sources implies likely unethical use of AI. VentureBeat asked them about data sourcing for the May 4, 2024 article:
“As with all generative AI applications, the question is what ElevenLabs trained this model on and if it included copyrighted materials. And if so, whether it obtained permission from the rights holder or if it believes training without permission is protected by fair use. … ElevenLabs declined to comment beyond what was shared on social media.”
No further information has been shared about their data sources in the 2+ months since that article came out. Their AI Safety page does not address this concern. This makes it seem likely their music model wasn't ethically trained with properly-sourced songs 😞.
Ethicality of the ElevenLabs music features cannot be assumed. It needs to be assessed separately from their voice cloning (which may need to be re-assessed).
For now, ElevenLabs is off my ‘ethical shoestring’ list.
What’s Next?
This post supports PART 3 on WHO, WHAT, and WHEN. PART 4 is coming up next. Subscribe for FREE to be notified automatically of new posts and support our work:
REFERENCES
See this “AI for Music” page for links to all posts related to this series, as well as bonus articles mentioned above on voice cloning and other music-related topics.
Voice cloning tool provider overview:
