Atelier

active

A vision-based agent that paints on browser canvases.

PythonOpenCVNumPyPillowscikit-imagemssPyDirectInputFireworks VLMReactVite

Why I built this

I wanted to build an agent that paints by looking at a canvas, not by calling an API. The long-term goal is Meccha Chameleon, a game where you paint to hide from seekers. I have not integrated it with the game yet, but it works on browser canvases.

How it works

screen capture → perception → planner → executor → verifier → loop

perception.py computes regional color and edge error. planner.py chooses the worst region, executor.py turns intents into brush strokes, and verifier.py checks whether error decreased. orchestrator.py coordinates the loop. easels/browser_canvas.py handles DPI-aware capture, fiducial localization, and input injection. A React + Vite dashboard shows events in realtime.

Decisions

Vision-only, no canvas API access

If the agent can see the canvas and move a mouse, it can paint anywhere, without DOM access or game mods.

OpenCV over VLM for perception

OpenCV is deterministic and cheap. VLM planning is optional in planners/vlm_planner.py.

Closed-loop verification

It checks every stroke batch and retries when the error does not decrease.