“The Combobulator represents a
pioneering approach to sound
generation through neural audio
synthesis, which has significant potential
for the music production industry. We
hope that our contributions and
innovations in this field inspire further
research and development in AI sound
design, but more than anything, we
hope our neural network marketplace
will model the concept of ethical
collaboration with artists in building
training datasets for generative AI
algorithms.”

– Ben Cantil, DataMind Audio 

Introduction

DataMind Audio is a pioneering company that operates at the intersection of music and artificial intelligence (AI). Created by musicians for musicians, their mission is to produce innovative electronic instruments that leverage the power of AI to expand and augment human creativity and capability in the realm of sound design.

Organisational Context

In 2022, DataMind were accepted onto the final round of Creative Informatics’ Resident Entrepreneur programme, proposing to use the £12,000 grant to support their five-strong core team in developing a groundbreaking new AI plug-in, the Combobulator.

Using AI-models trained by Stability AI servers on the audio from world-class music producers (such as Tipper, Max Cooper, and Mr. Bill) with their explicit permission, the Combobulator transforms a live audio signal to sound like the selected music producer’s unique style. Whilst this programming an AI to “hallucinate” an interpretation of an audio signal itself already represents a completely new paradigm for creative sound design, the plugin comes with many modular controls that will be familiar to synthesizer users.

Image: Ben Cantil introduces the Combobulator. Credit: Creative Informatics/Chris Scott.

Challenges and opportunities

During the development of the Combobulator MVP, DataMind came to understand how innovative their online marketplace for ethically-sourced neural networks, which will create a new economic opportunity for artists in the AI space. By curating the world’s finest collection of neural networks and establishing a global marketplace for AI-generative implementations of their work, DataMind Audio is providing a new income stream for music producers to earn royalties, addressing the financial insecurity faced by artists This represents a significant market innovation and initial sharing of the product suggest it will garner positive attention in the music software industry.

Furthermore, the idea of a neural network marketplace, where trained networks can be sold based on specific artist datasets, has broader implications beyond sound design. This concept could potentially be applied to other forms of media in various creative industries. As the technology rapidly expands and accelerates, DataMind aim to actively participate in the conversation surrounding its applications in all creative industries.

Open innovation ecosystems

Participating in the Resident Entrepreneur programme, a funding stream of the Creative Informatics R&D programme which ran from 2018-2024 in Edinburgh as a component of the UK’s industrial strategy for the creative industries, helped DataMind’s team develop as researchers, audio specialists, artists, and community members. Internally, they have adopted a rolling work plan and a private Discord server to help streamline production. This approach allows them to easily track each other’s progress and make updates in real-time, saving time and effort in coordinating tasks and ensuring everyone is on the same page. 

Mentorship was provided by Tinderbox Orchestra’s Luci Holland, whose guidance and expertise in the UK music industry proved to be an invaluable resource, providing astute insights which have helped position DataMind for success, and which is continuing on an ongoing basis. DataMind have produced two working plugins, both of which are in Alpha testing at the time of writing.

In addition to the production of these tools, this project has also helped the company increase their knowledge and awareness of ethical considerations in this emerging field of AI creativity. Working with AI technologies raises ethical considerations around data privacy and ownership of generated content, and DataMind have striven to ask the hard questions and find their place to rise to them. The project has deepened the team’s awareness and understanding of AI ethical issues and helped them craft ethical policies for the company’s use of artist content for data training.  DataMind are committed to paying out royalties of 50% of the sales price of each model, after the initial expense of training the model (£800-£1000) has been paid. On a wider scale, they hope to become a model company for generating ethical training data, working with artists in the AI music space to become the UK’s premier ethically-sourced AI neural audio software company.

Measurable outcomes

Since their participation in Resident Entrepreneur, the company has been successful in gaining a further £5,000 of R&D funding through Creative Informatics’ Creative AI Music & Audio Pilot Project to retrain the Combobulator using an improved version of the current algorithm to enable the tool to scale more quickly and efficiently, and reduce the likelihood of it being swamped by larger scale competitors. They have also recently been awarded £50,000 by Innovate UK to train a number of Model Reliability Engineers, who will specialise in training the tool’s neural networks in close collaboration with the artists whose styles they will be applying, and with ethical considerations at their forefront.

For more information on DataMind Audio, see here.