AI DAW is a small, AI-native music studio for your desktop — write, arrange, mix, and master in one lightweight app, and let an AI agent work in the session alongside you.
It is a real DAW: a multi-track timeline, a piano roll, a mixer with sends and buses, automation lanes, a built-in synth, and VST3 / CLAP / AU plugin hosting. It is also built to be operated by a model — a bundled Skill teaches any AI harness to drive the running app over MCP, so an agent can add tracks, write notes, balance a mix, and analyse the result while you watch it happen in the UI.
Everything runs on your own machine. No account, no API key, no audio leaving your computer.
Models got good at writing code because they could run it through a coding-agent framework; models got good at 3D modelling because they could drive real modelling software through Blender MCP. In both cases the model gets a real tool, a real result, and a real feedback loop — and that is what makes fast iteration possible.
Music has mostly been missing that loop. A model can generate audio, but it cannot work on a session the way a person does: no arrangement to edit, no mixer to ride, no undo history, no way to notice that its last change made things worse.
AI DAW is a small, deliberate attempt to close that gap. It is a real DAW an AI agent can operate through a Skill and MCP, which makes it useful for:
- Testing what a model actually knows about songwriting — not whether its audio sounds plausible, but whether it can build an arrangement, choose parts, and fix a mix.
- Collecting real songwriting traces — every agent edit lands in a normal project with a normal undo history, so a session becomes data about how a model reasons about music.
- A step toward models that can write music the way they write code — with a tool that talks back.
Being small is a feature, not a compromise.
- A lightweight footprint — no Electron, no cloud, no bundled model zoo. The macOS app is about 38 MB — for reference, mainstream DAWs such as Ableton Live, Logic Pro, and FL Studio typically install at 1–3 GB, making AI DAW 95%+ smaller.
- Fast to start — nothing heavyweight loads at boot; AI models are pulled in only when you use them.
- Big models on demand, never in the installer — heavy work such as music generation is not integrated by default. Singing voice (SoulX-Singer, 1.4 GB) and voice swap (SoulX-Singer-SVC, 2.4 GB) are opt-in models in Manager → Models: download them when you want them, and they cost no disk until then.
- Built to be extended — interfaces for plugging external AI models into the workflow are on the roadmap.
AI DAW ships a Skill (skills/aidaw) that teaches an AI agent how to operate the running app — reading and writing the project, tracks, clips and notes; analysing a style, looking up music theory and drum patterns, and turning a genre recipe into MIDI; mixing (gain, pan, sends, automation); transport and recording; loudness and spectrum analysis; WAV export; plugin scanning. It works over the built-in MCP server, so any harness that supports Agent Skills can pick it up.
The app also ships 18 sampled instruments — acoustic piano, nylon-string and clean electric guitars, finger bass, an acoustic drum kit, a string ensemble, slow strings, solo violin/viola/cello/contrabass, pizzicato strings, trumpet, French horns, and synth strings/brass 1 & 2 (SF3/SF2, about 13.5 MB total) — ready to play out of the box.
Option 1 · Install from the command line
npx skills add heshengtao/ai-dawOption 2 · Ask your agent to install it
Install this Skill: https://github.com/heshengtao/ai-daw
Every edit an agent makes goes through the same command pipeline as the UI: it lands in the undo history, shows up in the interface, and is attributed to the agent — so you can always see what it did and undo it.
Lay out a song the way you hear it. Drag clips across tracks, split them where the music changes, and let the ruler and grid follow a tempo and time signature you can change anywhere in the song. MIDI clips draw their own notes and audio clips draw their own waveform, so you can read the arrangement at a glance.
- Multi-track arranging — MIDI and audio clips side by side, cross-track drag & drop, marquee select
- A piano roll that reads well — clip previews show notes; the editor gives you velocity, zoom, and a keyboard
- Dynamic tempo & meter — change tempo or time signature at any point; the grid follows
- Transport & loop — play, stop, seek, loop a region, metronome
- Automation lanes — draw curves for anything a plugin or the built-in synth exposes
- A real editing toolkit — split, move, copy/paste, undo/redo, plus a keyboard-shortcut cheat sheet
Balance the song, then make it loud. Channel strips give you gain, pan, sends, buses, mute/solo and metering; a mastering chain on the master bus handles loudness, peaks, and stereo width so you can render something release-ready without leaving the app.
- Mixer — gain, pan, sends, buses, mute/solo, and level metering
- Built-in effects — dynamics, EQ, and time-based effects, ready to drop on any track
- Mastering chain — loudness normalization, true-peak limiting, and stereo width
- Built-in synth — oscillators, envelopes, and a filter with a live oscilloscope
- Export — lossless WAV or LAME-encoded MP3 (sample rate and bitrate your pick), a standard MIDI file, or a MusicXML score
AI DAW hosts VST3 and CLAP plugins everywhere, plus Audio Units (AU / AUv3) on macOS, so the tools you already own come with you. A plugin manager searches the open registry, filters by type and OS compatibility, and installs, updates, or removes plugins in one click — or point it at something you already have on disk.
- VST3 / CLAP / AU hosting — load third-party instruments and effects on any track (AU and AUv3 on macOS)
- Plugin manager — search the registry, filter by instrument/effect and platform, install / update / uninstall
- Local import — bring in a plugin you have installed elsewhere
- Editors & parameters — open a plugin's own UI, or drive its parameters from the inspector and automation lanes; AU plugins without their own view get a generic parameter panel
The AI features are built for real sessions, not demos. Point them at a track and they hand back something you can keep editing: MIDI notes you can move, a vocal take with less room noise, stems you can rebalance. Everything runs on your own hardware — CPU by default, with optional CoreML (macOS) and DirectML (Windows) acceleration.
- Tempo detection — Beat This! (ISMIR 2024) infers a global BPM and time signature in one pass
- Audio-to-MIDI — Basic Pitch turns a recorded performance into editable notes; switch the model to MuScriptor (via the audio.cpp engine) for multi-instrument transcription, which lands one MIDI track per instrument. 393 MB–5.1 GB, downloaded on demand (CC BY-NC 4.0 — non-commercial)
- Voice cleanup — LocalVQE reduces noise and reverb on vocal takes
- Stem separation — HTDemucs v4 splits a track into 4 or 6 stems, ready to remix
- Singing voice — a MIDI melody, lyrics and a reference voice become an editable vocal take (SoulX-Singer zero-shot timbre cloning). Lyrics can be edited per note in the piano roll (
-slurs), and reference voices live in a voice library shared across projects — changing the timbre is just picking another one - Voice swap — take an existing vocal, keep its melody/lyrics/timing and replace only the timbre with a reference voice (SoulX-Singer-SVC, zero-shot). The result lands on a new track below the source and is a single undo away; separating the vocals from a full mix first keeps accompaniment out of the pitch tracker
- Offline by design — no account, no API key, no uploads
Flip one switch and your project becomes something an AI assistant can operate. The built-in MCP (Model Context Protocol) server exposes the session over HTTP bound to 127.0.0.1 — connect your agent, hand it a task, and watch it add clips, fix notes, or change levels while the UI updates in real time. The aidaw command-line tool is a thin client for the same server.
- MCP over HTTP — a loopback-only endpoint your agent connects to
- Live edits — agent actions show up immediately as ordinary project changes
aidawCLI — installable onto yourPATHfrom Settings
Pre-built binaries will be published on the GitHub Releases page. Until then, build from source.
- Rust (stable) with Cargo
- Node.js ≥ 18 and pnpm
- Tauri v2 system prerequisites — see https://tauri.app/start/prerequisites/
git clone https://github.com/heshengtao/ai-daw.git
cd ai-daw
pnpm install
pnpm tauri dev # run the app in development
pnpm tauri build # produce a release buildOther useful commands:
pnpm build # type-check the frontend and build it
pnpm build:aidaw # build the aidaw CLI (see tools/build-aidaw.sh)| Platform | Status |
|---|---|
| Windows (x86_64) | ✅ Supported |
| macOS (Apple Silicon) | ✅ Supported (see the “app is damaged” note below) |
| Linux (x86_64 / arm64) | ✅ Supported |
macOS says “App is damaged”? The app is ad-hoc signed (not notarized), so macOS quarantines downloads. Run this in Terminal, then open the app normally:
xattr -cr "/Applications/AI DAW.app"
Intel Macs are not supported yet. ONNX Runtime publishes no prebuilt binaries for
x86_64-apple-darwin, so the local AI models cannot be linked for Intel macOS.
Yes. AI DAW is open source under Apache 2.0. There are no paid tiers and no feature gates.
Yes. The app and every AI model run locally. The only network access is downloading plugin packages and AI models (which you start), and fetching the plugin registry.
No. AI models run on CPU by default. macOS can use CoreML and Windows can use DirectML when the hardware supports it; both are optional.
Two small ones do: Basic Pitch (audio-to-MIDI) and LocalVQE (voice cleanup), so those work the moment you install. The larger models — Beat This! (tempo detection) and HTDemucs (stem separation, 4- or 6-stem) — download on demand from the in-app model manager the first time you use them, and are stored outside the source tree.
No model weights are ever part of the repository or the installers, and each model keeps its publisher's license: see MODELS.md. That includes the optional CC BY-NC 4.0 (non-commercial) models — YuE2 for song generation and SheetSage2 for cover scores.
VST3 and CLAP on Windows, macOS and Linux; Audio Units (AU / AUv3) as well on macOS.
Yes. The Skill and the aidaw CLI talk to a running AI DAW over MCP, so start the app first (aidaw status will tell you if it is up).
The application code is Apache-2.0 — see LICENSE.
Model weights are a separate layer: they are never part of this repository or of the
installers, they keep the license their publisher gave them, and the app downloads them
on demand (and verifies each downloaded file's sha256). See MODELS.md for the full
table of models, sources and licenses.
Some optional models are CC BY-NC 4.0 (non-commercial) — YuE2 for song generation and SheetSage2 for cover scores. They are opt-in downloads: do not use those weights for commercial work unless you hold a commercial license from their publishers. Generated audio/MIDI belongs to you, under the upstream terms and subject to the rights you hold in the input audio.
This project builds on third-party open source software. Files adapted from other projects, and dependencies whose licenses are not in the permissive default set, are listed in NOTICE.