PART 3: Unfair Use? Ethics of genAI music - WHO, WHAT, WHEN
Who's who & who is doing (or not doing) what: an overview of 85 companies working on generative AI for music, with initial triage findings on ethics (3/4Cs: consent/control, credit, and compensation)
This article overviews 85 companies working on generative AI for music, product announcements, key features, and initial findings on ethics. It is PART 3 in our 8-part series on ethics of generative AI for music in . This report includes multiple bonus articles and pages (links below).
PARTS 4-7 will dive deeply into 4 pillars of ethical concerns for the top uncertified companies. In PART 8, we’ll summarize and share some recommendations. Subscribe to be notified automatically when new articles are published (it’s FREE!):
This article is not a substitute for legal advice and is meant for general information only.
Post Contents (these will become hyperlinks after this article is published):
Acronyms
AI Ethics in Music
Who, What, When: 85 companies
Landscape of GenAI Music Product Announcements
Certified as Fairly Trained - Music and Voice Cloning (9 + 2)
Not Certified as Fairly Trained - Voice Cloning (13)
Not Certified as Fairly Trained - Top companies in Asia (4)
Not Certified as Fairly Trained - Top companies outside Asia (7)
Not Certified as Fairly Trained - all other (11 summarized + 39 others)
What’s Next?
References
Acronyms
3Cs / 4Cs - Consent, [Control, ]Credit, and Compensation to contributing artists (for use of their content)1
DAW - Digital Audio Workstation
AI Ethics in Music
The field of competitors offering products using generative AI to create musical melodies is vast and growing. Our goal is to help shed light on the ethical positions and risks of these companies and their products. For some, their AI tactics clearly show multiple bright red flags. For others, we just don’t know enough yet. We’re on a mission to find out and share the info with you, so we can all make better-informed decisions.
What’s ETHICAL in use of generative AI for music? At a minimum: avoiding use of unlicensed content (music and voices) and ensuring the “3Cs/4Cs” (consent / control, credit, and compensation) for contributing artists. See PART 2 and this post on scope and the 4 pillars of ethics for more specifics.
Landscape of GenAI Music Providers
In the area of generative AI for music, recent major events in 2024 have included Adobe’s product preview announcement in February (link), Microsoft’s announcement of the Suno plugin for Copilot (link), and startup Udio emerging from stealth on April 11 (link).
They are hardly the first or only enterprises to announce AI-based tools for creating music from text prompts with generative AI, though. Many more companies have announced previews and products, and many websites have posted pages citing or comparing the “best music generators”. (It’s interesting to see which tools are, and aren’t, covered or mentioned where.) Links are in the References.
11 Certified as Fairly Trained - Music and Voice Cloning
9 certified companies for music
Page “9 ethical genAI music tools [Unfair Use? series, PART 3]” summarizes the 9 companies in generative AI for music who are certified as Fairly Trained.
Profile pages are available for: Beatoven.ai, Boomy, Endel, Infinite Album, Lemonaide, LifeScore, SOMMS.ai (now MusicalAI), Soundful, Tuney.
2 certified companies for voice cloning
Two other voice cloning tool providers were also certified as ethical by Fairly Trained, Voicemod and Kits.ai. They have not been profiled.
Is your main goal for reading this post to identify an ethical genAI music or voice tool you can use without concern about adverse impact on musicians? Simply check out those listed above, and you can safely skip the rest of this article. 😊
At this writing, none of the companies listed below (including the big, well-known firms) are certified as Fairly Trained. That doesn’t necessarily mean they are unethical. Let’s see what we can find out.
Profile pages summarize what’s known on whether companies give musicians the 3Cs or 4Cs (“consent, [control,] credit, and compensation”) for use of their creations. The profiles cover:
History & Partnerships
Key Features
Training Data & Technologies
Ownership, Usage Rights, & Pricing
13 Not Certified as Fairly Trained - Voice Cloning
We analyzed voice cloning companies in our April 29 article “AI Ventriloquism: Send in the clones”. At the time, we classified four uncertified companies as likely to be ethical for voice cloning: ElevenLabs**, Revocalize, Speechify, and Voice-Swap**.
** The scope of the April 29 analysis was limited to voice cloning and vocal covers. Three updates are worth noting:
Several Substack writers shared comments on using Wondercraft.ai for audio voiceovers. Because of this, we included Wondercraft in our May 9 evaluation “Say what? Adding audio voiceovers with AI, ethically”. We have not fully vetted the ethics of their models and tool, though.
On May 9, ElevenLabs announced previews of new features for generating music with AI (ref: MSN). Their training dataset wasn't clear at first, and the company declined to comment on it to VentureBeat. With no further information forthcoming since then about their data sources, we see a risk that their music model wasn't ethically trained with properly-sourced songs 😞. Ethicality of the ElevenLabs music features cannot be assumed. It needs to be assessed separately from their voice cloning (which may need to be re-assessed). See this ElevenLabs profile page for the latest details.
On May 21, Voice-Swap announced their partnership with BMAT on using technology to verify music sources (ref: Billboard).
We also identified 8 uncertified companies with red flags on their products’ ethics (not a complete list). They are Covers.ai, FakeYou, Fineshare’s Singify and FineVoice, Lalals, Musicfy, Vocloner, Voicify (now Jammable), and Uberduck. We have profiled Musicfy to share some insights into our concerns about the companies in this group.
Wondercraft.ai is now #13 in our voice cloning list. It will be profiled in the near future if I resume exploring use of AI-generated voiceovers for my posts and podcasts. (At present, they are not big in the music space, only in voice cloning.)
4 Uncertified Major Companies in Asia
In this July 1 post, we covered what’s known about four leading Asian companies. Two have over 100 million users each, and all are partnering with major music labels. Each company has a profile page:
BandLab Technologies, in Singapore
ByteDance (TikTok parent), in China
HYBE/Supertone, in South Korea
Tencent Media Entertainment (TME), in China
Top 7 Not Certified as Fairly Trained - Rest of World
These 7 companies have individual profile pages that will be ongoing works in progress. The profiles cover History & Partnerships, Key Features, Training Data & Technologies, and Ownership, Usage Rights, & Pricing.
These 7 were selected as the top major uncertified companies outside of Asia.
The obvious question: Why only these 7?
What about Amazon, Apple, and other major players?
In general, the reasons fall into one of these 4 categories:
They are not active in their own R&D for developing genAI products for music. (Example: Amazon - they are partnering with Endel.)
Their offerings are outside of the space of genAI for musical melodies. (Examples: Brain.fm is in functional music; Spotify is focused on music recommendation AI.)
They don’t have high customer mindshare or visibility at this point in genAI for musical melodies. (Examples: Apple, IBM)
Information on their technologies and training data is too constrained to allow a deeper dive.
We may research and write additional profiles in future if there’s sufficient interest, or as new offerings are announced.
Spoiler alert: All 7 have at least one red flag. Some have many red flags.
You’re not surprised, right, given what’s been in the news lately? Two bonus articles touched on 2024 events relating to all 7:
May 24: “ScarJo, pizza glue, Recall spyware? AI ethics & safety news” - Adobe, Google, Meta, Microsoft, OpenAI
June 24: “Breaking news: Big music labels suing Suno and Udio over AI-based copyright infringement” - Suno and Udio
Ethical considerations relating to these 7 companies are addressed on their WIP profile pages. The four ethics pillars will be explored even further for these 7 companies in PART 4, PART 5, PART 6, and PART 7. These 7 profiles will also be updated over time as new significant events and product announcements occur.
Not Certified as Fairly Trained - All Others (50)
The research for PART 3 unearthed over 85 companies active in genAI for musical melodies. This set of 85 company names is not exhaustive, though. There are likely hundreds or even thousands of companies worldwide in this active space!
The sections above describe 35 of the 85 companies. This next section summarizes 11 more in some detail. The remainder of this post lists the 39 not yet covered.
11 more uncertified tool providers - summarized
Both Amper Music (Shutterstock) and Beatbot (Splash Pro) are now defunct. However, their names popped up in many places during my research, so I’ve briefly summarized both on this page. The other 9 companies are: AIVA (Artificial Intelligence Virtual Artist), Amazon Music, Ecrett Music, Melobytes, Mubert, Riffusion, Soundraw, Stability AI (HarmonAI, Stable Audio), and Wavtool [gone ‘offline’ as of Dec. 30].
The profiles on the 9 active companies cover what has been discovered so far about their ethics, the training data they use, and whether they give musicians “consent, credit, and compensation” for use of their creations. Updates may follow as new tool providers emerge or as major events occur.
3 major companies without customer-facing offerings for musical melodies (yet)
Apple, IBM, and Spotify will be profiled in future if they make meaningful moves towards offering tools using generative AI for musical melodies. They haven’t yet. Apple’s 2022 acquisition of “AI Music” aims at personalized functional music. Spotify’s closest so far is Basic Pitch. The Github repo for IBM’s 2016-vintage Watson Beat looks stale, but the tech is still in use as of January! and We’ll be watching them.
3 DAWs with AI plugins or features
Like AIVA, BandLab Song Starter, and Wavtool, Audacity and Audio Design Desk accept plugins which may use AI. (Example: Audacity “OpenVINO AI effects”.) Some, such as Hit’n’Mix RipX, combine AI plugins with built-in AI features. Statements on ethics of the AI in these plugins and features are generally sparse.
13 uncertified tool providers making statements on their AI ethics
Of the 33 remaining, 13 uncertified tool providers assert that their AI use is ethical. These claims have not been vetted, and are provided for context only.
ACE Studio (AI DAW): This Audiocipher article reports that “Each of the AI vocal models have been licensed and cleared for commercial use”.
AIbeatz: Their home page claims you can “generate unheard music in seconds, customize it in minutes and own it forever using ethical AI” and that “Our AI composition technology is fed with our proprietary urban music dataset crafted by Pro producers & musicians.”
Aimi.fm: This MusicTech report quotes the founder and CEO as saying that “The machine learning models in charge of generating those tracks have also been “trained ethically and legally””.
Audialab (Emergent Drums 2): Their site claims “No source recordings are used to generate the samples, so each one is truly original.”
Controlla Voice: Their ethics page says they enforce ethics in the voices that can be trained in their tool. “We don't allow user-uploaded voices, model imports, or any features that enable users to skirt regulations and steal voices from artists.” This attention to user-provided content is positive. However, no insights wre found on any underlying AI models they use or how they were trained.
DataMind Audio: Their site claims “instead of producing large models that can make any sound, we create small models that only imitate one artist at a time … we are currently only producing boutique Artist Brains that are ethically trained on the select Artists we affiliate with”.
HookSounds: Their site claims that “our 100% in-house team of musicians compose and produce assets” for their tool.
Hooktheory Hookpad: This Audiocipher article says of the Aria plugin “The Anticipatory Music Transformer was originally trained on the Lakh MIDI dataset. That's a collection of ~176k MIDI files under a creative commons license.” Hooktheory co-founder Ryan Miyakawa has stated that Aria is not trained on customer projects.
Larnii: Their site claims “We don't use LLMs to copy from artists, our super smart and fast algorithms are trained by humans!’)
Murf.ai: Their “ethical AI” page claims “We team up only with professional voice artists”.
MusicLang: Their site claims “We are committed to respecting artists' rights. Data Relevance: Using symbolic music means our model requires less data to perform at a high level, relying solely on CC0 data.” [NB: This is a reference to the CC0 license of Creative Commons.]
Splice: Their site claims their AI is trained on “responsibly sourced sounds” from “the world’s best sample library”.
Synthesizer V (Dreamtonics): Their site claims they are “Using AI the Ethical Way” and that “All our voice databases are officially licensed, each one sung by a professional musician.”
20 uncertified tool providers without credible AI ethics statements
20 other tools which surfaced during the research do not make claims of ethicality, rely on overly-broad views of “Fair Use”, or are reported to follow QDPs (questionable data practices).
Amadeus Code (Evoke Music)
Computoser (site no longer active?)
Humtap (part of Abbey Road’s Red incubator)
Jukin Composer (Jukin Media)
Orb Producer Suite 3 and Plugins (the FAQ link in the footer of their page is broken; the one in the top menu works, but doesn’t address AI ethics)
Vocaloid (by Yamaha)
Waveformer (by Replicate) - uses Meta MusicGen, but also recommends Riffusion and provides tools based on Harmonai Dance Diffusion
Based on known information as of July 17, 2024, these 50 companies will not be analyzed further in this article series.
If you’d like us to profile one or two specific companies from this list, or if you know of some genAI music companies not in the list that you’d like to see covered, please suggest them 🙂 And if you have insights on ethicality of any of these tools, please add them here!
What’s Next?
In PART 3, we identified 85 companies involved in generative AI for music, analyzed 31 with in-depth profile pages, and selected 7 of these for deeper analysis. In PART 4, we’ll dive into the ethical concerns regarding Pillar 1: Training and Reference Data. See you there!
If you enjoy my work, I’d love to have your support via a heart, share, restack, Note, one-time tip, or voluntary donation via paid subscription!
REFERENCES
Articles and bonus pages in this 6 ‘P's in AI Pods series
See this “Ethical AI for Music” page for links to all posts related to this series, as well as bonus articles mentioned above on voice cloning and other music-related topics.
NOTE: All company profile pages are now in new section “profiles”. Posts in this section will not be sent to current or future 6P subscribers by default. Subscribe to that section (free) if you’d like to be notified when new company profiles are published. Help on managing individual section subscriptions is here.
Specific articles, in the order they are referenced in this post
![4 pillars of ethical genAI for music [Unfair use? series, Part 2] 🗣️](https://substackcdn.com/image/fetch/w_140,h_140,c_fill,f_webp,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F69ebdded-e557-42fc-87a8-fb959b309e2e_1730x1170.png)
![4 AI music companies in Asia: quick looks [Unfair Use? series, PART 3]](https://images.unsplash.com/photo-1612383401582-e96cc0bc83d2?crop=entropy&cs=tinysrgb&fit=max&fm=jpg&ixid=M3wzMDAzMzh8MHwxfHNlYXJjaHwxfHxhc2lhJTIwbWFwfGVufDB8fHx8MTcxOTYyNDczNnww&ixlib=rb-4.0.3&q=80&w=1080)
![9 ethical genAI music tools [Unfair Use? series, PART 3]](https://substackcdn.com/image/fetch/w_140,h_140,c_fill,f_webp,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F91c8b468-57fd-4baa-b87f-dc36aff28175_1920x1275.jpeg)
![11 uncertified companies [Unfair Use? series, PART 3]](https://substackcdn.com/image/fetch/w_140,h_140,c_fill,f_webp,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc6c3ea78-efc6-4631-90d6-f48e2d266489_3360x2400.jpeg)
Selected reference links on genAI music tools
2022-06-22, “Composing with artificial intelligence: how AI can help you write music”. Identifies: AIVA, Amper Music, Ecrett Music, Humtap, Orb Producer Suite 3, Soundraw
2023-01-28, “Google’s new AI turns text into music”, by Mitchell Clark (The Verge), on Google’s MusicLM
2023-03-23, “There’s an AI for that now: Music”, by Ishika / Dubverse. Identifies:
Amper Music, AIVA, Jukin Media, Jukebox by OpenAI, Melodrive, MusicLM by Google, and Larnii
2023-06-22, “Battle of the Bands: MusicLM vs MusicGen vs Riffusion”, by
2023-08-02, “Meta’s AI music generator could be the new synthesizer — or just muzak”, by Emilia David, on AudioCraft/MusicLM
2023-08-21, “YouTube starts Music AI incubator with Universal Music as partner”, by Yuvraj Malik (Reuters)
2023-08: “Top 12 AI Music Generators in 2024”, Analytics Vidhya. Identifies these 12: AIVA, Amadeus Code, Amper Music, Chrome Music Lab’s Song Maker, Computoser, Ecrett Music, Generative.FM, Google’s MusicLM and Magenta (listed separately), Humtap, OpenAI’s MuseNet, and Soundraw.io
2023-12-07, “How Does AI Actually Work? 28 Learning Resources”, by @whytryai, on Microsoft Copilot plugin for “Suno”
2023-12-14, “Muzic: Microsoft AI Music Team Builds Text-to-MIDI & More”, by Ezra Sandzer-Bell / audiocipher
2024-02-19, “A Complete Guide to AI Music Generators”, by Ezra Sandzer-Bell / audiocipher: covers:
TTM (Text To Music): AudioCipher, Google MusicLM, Mubert, Riffusion 2.0, Splash Music, SoundGen, Suno AI
AI DAWs for musicians: AIVA, Audacity OpenVINO AI effects, Audio Design Desk, Bandlab SongStarter, RipX DAW Pro, WavTool
AI music generators for beginners: Amper, Boomy, Musenet, Soundful, Soundraw
2024-03-01, “9 Best AI Music Generators (March 2024)”, by Unite AI/Alex McFarland: covers AIVA, Amadeus Code, Amper Music, Boomy, Ecrett Music, Loudly, Soundful, Soundraw, WavTool.
2024-04-12, “10 Best AI Singing Voice Generators for Making Music in 2024”, Ezra Sandzer-Bell. Covers ACE Studio, Controlla Voice, Emvoice, Kits.AI, Murf.ai, Revocalize, Synthesizer V, Uberduck, Vocaloid, and the Google Colab Underground.
2024-04-17 , “Top AI Music Generation Companies in India”
2024-04-26, “Udio Vs Suno: Most Popular AI Text To Song Generator Apps”, Ezra Sandzer-Bell: covers Udio, Suno, Riffusion.
2024-05-23, “8 Best AI Music Generators You Need to Try Today”, Ramesh Reddy: covers AIVA, Boomy, Google Magenta Studio, Google MusicFX, Melobytes, OpenAI Musenet, Soundful, Soundraw
End Notes
Credit for the 4Cs (consent, control, credit, compensation) phrasing goes to the Algorithmic Justice League (led by Dr. Joy Buolamwini).
Credit for the original 3Cs (consent, credit, and compensation) belongs to CIPRI (Cultural Intellectual Property Rights Initiative) for their “3Cs' Rule: Consent. Credit. Compensation©.”