A pipeline for music generation and modification driven by procedural and/or AI systems.

The AI-Driven Generative Music Pipeline project, a collaboration between SIRT and Spawnify, aimed to create a system for the automated modification of existing music tracks. Driven by the potential of AI, the project involved an environmental scan, prototype design, and iterative development based on user testing. The prototype incorporates audio operations like stem splitting/recombining, tempo and mode changes, key transposition, and audio effects.

The Motivation Behind Automated Music Modification

The collaboration between SIRT and Spawnify was motivated by the desire to explore and prototype a system that could automatically modify existing music tracks. This project was driven by the potential to streamline music modification processes using AI-based approaches. An initial environmental scan of possible approaches was conducted. The primary goal was to enable specific modifications to existing music tracks automatically.

The Approach to Developing the Prototype

SIRT and Spawnify collaborated closely to design and prototype a system that would allow automated modification of music tracks. The approach included:

• Environmental Scanning
  Investigating potential methods, including AI-based techniques.
  
• Prototype Design
  Creating an initial prototype focused on automatically performing specific modifications to music tracks.
  
• Implementation and Iteration
  Developing the prototype and refining it based on internal user testing throughout the project.
  
• System Components
  Developing components leveraging specific tools and libraries. For example, using Demucs for splitting tracks into stems and Pydub for recombining them. Zplane reTune was used for changing mode and transposing keys. The system architecture features a client-server model, with a web interface built using Streamlit, a Python backend for handling requests, and Python code for executing audio operations.

The Prototype System for Music Modification

The project has produced a prototype system designed for the automated modification of music tracks. Key aspects of the prototype include:

• Audio Operations
  Implementation of audio operations such as splitting and recombining stems, changing tempo and mode, transposing key, and applying audio effects.
  
• Web Interface
  A user-friendly web interface developed using Streamlit, enabling users to interact with the system.
  
• Client-Server Architecture
  A system architecture supporting multiple concurrent clients, with the Streamlit server handling requests and executing audio operations.
  
• Class Structure
  Defined classes such as operation_context, operation_controller, and base_operation to manage the flow and parameters of audio processing