Play Music with Your Hands
You do not need a keyboard, a MIDI controller or any training. With a webcam and a browser tab, your hands can pick chords, shape the sound and trigger drums — and this guide takes you from a first chord to a finished track.
How hand-gesture music works
Gesture music is simple in principle: a camera watches your hands, software turns what it sees into numbers, and those numbers drive a synthesiser. The difficulty is in the middle step. Raw hand data is noisy, so a naive mapping sounds like a random note generator.
Tunevate uses Google's open-source MediaPipe hand tracker, which finds 21 landmarks on each hand directly in your browser. It then counts extended fingers, measures wrist tilt and hand height, and passes them to a small music-theory engine that works in a key you pick. Because the engine only ever offers the chords that belong to that key, whatever your hands do lands on something musical. That is what lets a complete beginner sound good in the first minute.
Everything happens on your device. The video frames are processed and discarded locally, with nothing uploaded — read the privacy page for the details.
The gesture cheat sheet
Your left hand chooses what to play; your right hand chooses how it sounds.
| Left hand · 1–5 fingers up | Chord degree I–V in your key |
| Left hand · tilt the wrist | Flip between major and minor |
| Right hand · 1–4 fingers up | Voicing: root, 1st inversion, 7th, extended |
| Right hand · thumb out wide | Drop the chord down an octave |
| Right hand · tilt the wrist | Sweep the lowpass filter, bright to warm |
| Right hand · raise or lower | Volume |
| Right hand · fist, then open flat | Strum (guitar and bass voices) |
| Right hand in Drum Mode · finger combinations | Trigger one of 31 drum patterns |
Your first chord progression
- Open the live studio, allow the camera, and keep the key on C.
- Hold up one finger on your left hand — that is the I chord. Raise your right hand with one finger for the plain triad.
- Change to five fingers on your left hand (V), then four (IV), then back to one (I). I–V–IV–I is the backbone of countless songs, and it will already sound like music.
- Add a second right-hand finger for an inversion, and tilt that wrist to sweep the filter open and closed.
- Press record, wait for the count-in, and loop the progression. Then play over it, or add a drum pattern.
From a loop to a finished track
Gestures are the input, not the ceiling. Once you have a part you like you can layer it as a loop on the online looper, program a beat on the virtual drums, switch the voice to piano, guitar or a synth, and record your own voice or an instrument straight from your microphone onto its own track.
Arrange the tracks on a timeline, mix them with gain, mute, solo and a per-track filter, then export the result as WAV or MP3 — or save the whole project and pick it up later. The online music maker page walks through that workflow end to end, and the virtual instruments hub lists everything you can play.
Tips for reliable tracking
- Light your hands from the front; a bright window behind you hides them.
- Keep both hands fully in frame, palms toward the camera, fingers spread.
- Use headphones so the speakers do not leak into a microphone take.
- Pick the key first — every gesture stays in tune with it.
- On a phone, prop it up at chest height; the studio has a mobile layout with a collapsible control panel.
Frequently Asked Questions
Common questions about making music with your hands.
Can you really play music with just your hands and a webcam?
Yes. A hand-tracking model reads 21 points on each hand from your webcam, and Tunevate maps finger counts, wrist tilt and hand height onto chords, voicings, filter sweeps and volume in a key you choose — so the notes you land on are always musical.
Do I need any musical training?
No. Your left hand picks one of five chords in the key (I to V) and your right hand picks how it is voiced, so a beginner cannot hit a wrong note. Learning the key's chords by ear is a side effect, not a prerequisite.
What equipment do I need?
A computer or phone with a webcam, a modern browser (Chrome, Edge, Firefox or Safari) and ideally headphones. There is nothing to install and no account to make. No webcam? The no-camera studio runs the same chords, drums and loops by clicking.
Is my camera feed recorded or uploaded?
No. Hand tracking runs entirely inside your browser tab; the video is never uploaded, streamed or saved. See the privacy page for exactly what, if anything, leaves your device.
Can I make a whole song, or is it just a toy?
You can make a whole track: layer quantised loops, program drum patterns, record your microphone onto its own track, arrange everything on a timeline, then export WAV or MP3 or save the project to reopen later.
Why is my hand not being detected?
Almost always lighting or framing: face a light source rather than a window behind you, keep both hands inside the frame with the palms toward the camera, and avoid a busy or skin-coloured background. Moving the camera so it sees your hands at chest height helps more than anything else.