Built for real work.

A Chrome extension, a relay on your machine, and a command-line client. Nothing leaves your computer.

You are the password

The relay only listens on localhost. A Chrome profile shows up when the Pilot extension connects. Turn auto-connect off for a manual handshake every time.

# one relay, many profiles
node cli.js '{"action":"shot"}' --profile work

Works in the background

Pilot clicks, types and takes screenshots in inactive tabs while you keep working. Driven tabs go in a yellow Pilot group, so you can see what the agent touches.

# no tab switch, no stolen focus
node cli.js '{"action":"snap"}'

Reads screens with no vision key

An on-device OCR tool built on macOS Vision reads the screenshots. No API key and no upload.

./ocr /tmp/page.jpg        # text
./ocr /tmp/page.jpg --json # boxes

One command, one action.

Your agent sends JSON to the CLI. Pilot does it in the page and answers.

Relay status: unknown

node cli.js --statusWho is connected

{"action":"snap"}Title, URL, text and clickable items

{"action":"click","sel":"button.x"}Click by CSS selector

{"action":"clickText","text":"Save"}Click by visible text

{"action":"clickXY","x":100,"y":200}Click at a point

{"action":"type","text":"hi"}Type into the focused box

{"action":"key","key":"Enter"}Press a key

{"action":"fill","sel":"#name","value":"Ada"}Set an input

{"action":"form"} · {"action":"dialog"}Read the open form or dialog

{"action":"tabs"} · {"action":"navigate"}Tabs and navigation

{"action":"shot"}Small screenshot to ~/.pilot/shots/

Up in two minutes.

Node.js and Chrome. The OCR tool needs macOS.

Start the relay

node server.js

Load the extension

chrome://extensions → Developer mode → Load unpacked → extension/

Click Connect in the Pilot popupThat click is the handshake.

Drive it

node cli.js '{"action":"snap"}'

Your agent drives Chrome. You keep your window.