What

GestureSynth.me is a free, browser‑based musical synthesizer that uses webcam hand‑tracking to translate gestures into real‑time audio. It targets musicians, educators, and hobbyists who want to create music without external hardware or software installations, positioning itself as an accessible, privacy‑first alternative to traditional MIDI controllers.

Features

Two‑hand chord selection – The left hand’s raised fingers choose chord degrees (I–V, VI, VII) while wrist tilt switches between major and minor qualities.
Expression and harmony control – The right hand’s height adjusts volume, finger count selects triads, inversions or seventh‑chord variations, wrist tilt modifies a low‑pass filter, and thumb extension shifts the octave.
Dual performance modes – Gesture mode supports chord‑based playing; Theremin mode maps right‑hand vertical position to continuous pitch for melodic expression.
Four‑track loop sequencer – Allows tempo (30–300 BPM), time‑signature, quantized recording, per‑track mixing, and local export to WAV (or WebM fallback).
Local privacy processing – Webcam frames are analyzed in‑browser via MediaPipe; no video, audio, or user data is uploaded or stored on servers, and no microphone permission is required.

Helpful Tips

Use soft, front‑facing lighting and a background that contrasts with your hands to improve tracking reliability.
Position yourself about one arm’s length from the webcam, keeping both wrists, palms, and fingertips fully within the frame.
Run the synth in recent desktop Chrome or Edge for optimal camera, WebAssembly, and Web Audio support; enable low‑performance mode on slower devices.
In Theremin mode, begin with slow hand movements to map the pitch range before attempting rapid melodic passages.

OpenClaw Skills

Within the OpenClaw ecosystem, GestureSynth.me could be accessed through a headless‑browser skill that launches the web app, captures the generated audio stream, and forwards it to downstream agents for analysis, remixing, or transcription. A likely workflow would involve an OpenClaw “gesture‑to‑MIDI” skill that interprets the hand‑tracking data (e.g., finger count, wrist angle) into symbolic MIDI events, enabling integration with composition assistants, AI‑driven arrangement tools, or collaborative performance bots. Such a combination would expand the capabilities of music‑creation agents by adding a natural, camera‑based input modality, potentially reducing reliance on physical controllers for remote or educational settings.