Text to Music Generators: Meta’s AudioCraft, Google’s MusicLM, and Shift to Algorithmic Compositions

August 6, 2023

Imagine a world where words effortlessly transform into melodies, where the power of technology and creativity merge to create a symphony of innovation. Meta’s AudioCraft and Google’s MusicLM bring this dream to life, but their impact on the music industry also raises important questions about the future of musical creation. Let’s take a delightful stroll through these two captivating creations while considering the potential threats and fears they might evoke.

Meta’s AudioCraft: A Symphony of AI Soundscapes

Meta, the ingenious minds behind AudioCraft, have given us a glimpse of what’s possible when AI meets music. Forget about complicated jargon and technicalities – AudioCraft is your friendly companion that understands the magic of words. It’s like having a musical genie that turns your text prompts into melodies that resonate with your imagination.

This open-source codebase consists of three models – MusicGen, AudioGen, and EnCodec – each exploring the possibilities of generative audio. Picture MusicGen as your virtual composer, creating music from scratch based on your whimsical descriptions. It’s like having a conversation with a musical muse that translates your thoughts into harmonious tunes. MusicGen’s adaptability extends to track length, allowing the generation of extended sequences through a simple windowing approach. It was trained on an extensive collection of music owned by Meta or licensed exclusively for this purpose.

And then there’s AudioGen, your sonic storyteller, conjuring up barking dogs, footsteps, and other real-life sounds that whisk you away to different worlds. It’s like having your personal sound effects wizard who can transport you anywhere your imagination desires.

At the core of AudioCraft lies EnCodec, a neural audio codec that ensures that the music retains its essence even after undergoing its AI transformation. Think of it as a guardian of sound quality, making sure that every note and tone remains true to its origin. The diffusion-based approach to EnCodec further refines audio compression, minimizing artifacts and ensuring a seamless audio generation process, outperforming the MP3 format.

AudioCraft is not the first foray into the fusion of AI and music. Google introduced MusicLM to the public over six months earlier.

Imagine sitting down with a brilliant composer who can take your text and weave it into musical compositions that tug at your heartstrings. MusicLM brings your words to life, turning them into melodies that dance through the air. It’s like having a conversation with a musical virtuoso who understands the language of your soul.

While MusicLM remains inaccessible for direct experimentation, Google has shared an array of sample compositions, offering a glimpse into its capabilities. The examples presented are impressive, ranging from 30-second musical snippets inspired by genre and vibe descriptions, which they refer to as rich captions, to five-minute-long pieces generated from text prompts like “melodic techno.” Even more impressive and promising is the Story Mode, where MusicLM transitions between prompts, effectively crafting a narrative through musical expression.

The model’s versatility extends to generating 10-second instrument clips, interpreting specific genres, crafting music for thematic scenarios like a prison escape, and even differentiating between a beginner and an advanced piano player’s style. The explorations continue with MusicLM simulating human vocals, although there remains a grainy or staticky quality to the voices it produces. You can even transform a painting’s description such as “His melting-clock imagery mocks the rigidity of chronometric time. The watches themselves look like soft cheese—indeed, by Dali s own account they were inspired by hallucinations after eating Camembert cheese. In the center of the picture, under one of the watches, is a distorted human face in profile. The ants on the plate represent decay.” into a musical composition that perfectly matches its essence.

Back in May, Google rolled out MusicLM’s trial in the AI Test Kitchen on the web, Android, or iOS. You can signup for MusicLM’s waitlist. In a blog post, Google said it was working with musicians and hosting workshops to gather early feedback and see how this can empower creative process. It’s thrilling to imagine its improvement within a year, given that the primary hurdle for text-to-music or audio generation remains centered around maintaining exceptional quality and adherence to the caption.

The Promise and Perils of AI Music

Generative AI’s potential to create music from scratch challenges the traditional roles of composers and artists. While it offers a fresh canvas for creativity, there’s a fear that it might diminish the role of human composers, altering the dynamics of the music creation process. The prospect of AI-generated music flooding the market raises questions about originality and artistic identity. Will the music industry lose its soul to the allure of convenience?

As we explore the realm of AI-generated music, we’re met with a duality of promise and apprehension. While these AI companions hold the potential to enhance artistic endeavors, they also challenge the very essence of human creativity and expression. The threats of AI-generated music displacing human artists and homogenizing the musical landscape are real, and they demand a careful consideration of ethical and creative boundaries.

Meta and Google, in their pursuit of innovation, understand the importance of treading this path with caution. Their responsible approaches reflect a commitment to nurturing creativity while safeguarding against potential pitfalls. It’s a reminder that while AI can be a powerful tool, it should complement and amplify human ingenuity rather than overshadow it. So, whether you’re dreaming of composing your own symphony or grappling with the impact of AI on the music industry, AudioCraft and MusicLM serve as catalysts for a larger conversation.