AirAim

Hand tracking used as mouse input.

PythonMediaPipeOpenCVNumPyPyDirectInput

Why I built this

I've always wanted to play video games with gesture controls. An earlier webcam keyboard had too much latency and complexity, so I scoped down to a mouse and picked Aim Labs.

How it works

webcam → mediapipe handlandmarker → gesture classifier → cursor motion → os input

Open palm moves the cursor. Thumb-index pinch clicks. Fist pauses movement so you can reposition your hand. Sandbox, cursor, and live modes separate testing from real PyDirectInput control.

Decisions

Relative motion over screen-position mapping

Relative motion with acceleration, a deadzone, and EMA smoothing feels closer to a real mouse.

Hysteresis on pinch detection

Separate on/off thresholds prevent repeated clicks near the pinch boundary.

Coast through tracking dropouts

The cursor keeps its last velocity briefly before decaying to zero when tracking drops.