MIDI vs MP3: On-Device 125M Piano Models
That MIDI file you’ve been treating like any other audio file? It’s not. It’s a spreadsheet of intentions—a list of who pressed what, when, and how hard—nothing more. No sound, just data. A musician’s left hand finds a chord, the right answers with a melody, but the notes disappear the moment your fingers lift. Unless, of course, the sustain pedal is down. Then those notes linger, ghostly, until you finally release it. That’s the trick MIDI files exploit: they don’t store music, they store the ghost of music, the barely-there sequence of events waiting to be brought back to life.
Simon Edwardsson turned that ghost into an autocomplete engine. Almost a year ago he rigged his MIDI piano to an iPhone and taught it to finish his phrases mid-play. Like GitHub Copilot for piano, it watches your hands and suggests where your next note might go. The tool, RollTab, now lives in the App Store for anyone with a MIDI keyboard and an iPad. It’s not magic—it’s a 125-million-parameter model humming on your phone, translating MIDI events into plausible continuations faster than you can lift your foot off the pedal.
Technical Overview
The system’s core is a note scheduler that sequences musical events with precise timing. Each note carries delta (offset in ticks), duration, and velocity, and they're played in the order they're received. The example below shows two bars of music: a simple C major scale followed by a C major arpeggio with slight velocity variations.
NOTE(C4, delta=0, duration=12, velocity=80)
NOTE(D4, delta=24, duration=12, velocity=80)
NOTE(C4, delta=24, duration=24, velocity=80)
NOTE(E4, delta=0, duration=24, velocity=78)
NOTE(G4, delta=0, duration=24, velocity=82)The first two notes form a two-note ascending pattern with a quarter-note gap between them. The second set plays a sustained C major chord with E and G notes entering immediately but held longer, creating a rolled arpeggio effect. The velocity differences are subtle but noticeable—78 versus 82 on the high notes, which is enough to make the arpeggio feel dynamic rather than mechanical.
Under the hood, the scheduler uses a priority queue to order events by their absolute tick position. When the scheduler starts, it inserts all notes with their delta values converted to absolute time. This means the sequence is deterministic: if you replay the same list, you get the same output. The actual timing depends on the tick rate, which is usually 480 ticks per quarter note, but the scheduler doesn’t enforce that—it’s just a unit of measurement.
The velocity values are 7-bit, ranging from 0 to 127, but in this example, they’re clamped to 80 and nearby values. A velocity of 0 would technically turn the note off, but in practice, most synthesizers treat it as a "note-on with zero volume" message. The scheduler doesn’t care about the semantics—it just passes the numbers through. If you want to mute a note, you’d set velocity to 0 explicitly rather than omitting it.
One thing that’s genuinely confusing here is how the delta values interact with the note durations. The first bar has two 12-tick notes with a 24-tick gap between them, which is exactly one quarter note in a 4/4 bar at 120 BPM. But the second bar starts with a 24-tick note (a half note) and then two 24-tick notes with zero delta, which means they overlap. The scheduler plays them simultaneously, but the underlying synthesizer might handle overlaps differently depending on its polyphony settings. If you’re generating music programmatically, you have to account for this—otherwise, you’ll get unintended chords or dropped notes.
Industry Impact
What strikes me about this approach isn't the technical novelty—it's how it sidesteps the entire question of audio fidelity that dominates most AI music discussions. MIDI generation operates on a fundamentally different axis than raw waveform synthesis. You're not trying to fool anyone's ears; you're encoding musical intent in a format that musicians have been working with for decades.
This distinction matters more than I think the initial demos communicate. A musician receiving a MIDI file knows exactly what they're getting: performance data they can route through any virtual instrument, modify note-by-note, or arrange however they want. It's the difference between getting a finished painting versus getting the artist's sketch notes. The feedback about rhythm and composition improvements makes sense here—those are precisely the kinds of issues that emerge when you're working at the symbolic level rather than the audio level.
But I'm genuinely uncertain about the commercial trajectory. Musicians already have sophisticated tools for generating and manipulating MIDI—DAWs with built-in arpeggiators, chord generators, and algorithmic composition plugins. Convincing them to adopt an AI system means demonstrating clear advantages over tools they've already invested years learning. The requests for multi-part baroque accompaniment and file export suggest users want integration with existing workflows, not another standalone application.
Conclusion
The model only knows about 12.5 tokens per note, so it's compressing everything—every key press, every pedal change—into that narrow window. That's why the autocomplete works: it's treating music like a language model treats text, one event at a time. But unlike text, where a wrong word is just wrong, a wrong note in a piano piece can derail the whole thing. The model doesn't know what sounds good. It just knows what comes next.
RollTab is free and runs on-device, which means no cloud processing, no latency, no privacy concerns. But it also means the 125M parameter model has to do everything itself, with no help from a server farm. That tradeoff—local inference at the cost of model size—feels increasingly common. We're seeing it in mobile AI apps, in edge ML, in developer tools that promise to run anywhere. The question isn't whether on-device models will work, but what we're willing to give up for them to work at all.
I'm still not sure what to make of AI-generated music that starts with human input. Is it collaboration or replacement? Augmentation or automation? RollTab doesn't claim to answer that. It just plays the next notes.