OpenAdapt

Give your agent a program, not a mouse.

The open-source, verified execution layer for computer-use agents.

What it can do · Documentation · Quickstart · Live demo · Website · Discord

A computer-use agent looks at the screen and chooses every click again, on every run. OpenAdapt lets your agent, or a person, do a task once in any app: web, Windows, macOS, Linux, RDP or Citrix. It compiles that run into a program, and your agent calls the program as one tool. The program runs with no model calls and sends no screenshots anywhere. It confirms the right record before each save, reads the save back from the system of record after it, and returns a plain outcome. When anything doesn't match, it stops and tells your agent instead of guessing.

Measured on OpenEMR

One new-patient referral in a test copy of OpenEMR, timed from a fresh browser through sign-in, the entry and the saved record read back (median):

The API step replaces the program's 41 screen steps with the one form submission the app itself sends, then reads the record back. It covers single HTML form saves like this one; other apps and request types are not yet measured. In both program rows a possible-duplicate referral stopped without saving, as it should. Both program rows ran on 19 Oct on one copy of the app, interleaved, without the approval gate and without resumable runs; the API step was admitted by the benchmark harness, not by a person. The agent rows are the 14 to 15 Oct timing study, on its own copy of the app. SCOREBOARD.md has every run.

A screen can't show a save that went wrong behind it. On a test copy of OpenEMR, with faults injected in its database (a field blanked, a value changed, a duplicate chart, a chart removed, another chart deleted or edited), the agent claimed success on all 30 bad saves. Reading the record back caught 25 of the 30 and confirmed all 5 clean saves, with no false alarms. It missed the 5 where a different chart was edited. On MockMed, a test app that shows a save as done before its server answers, the agent also claimed all 30 bad saves, and the read-back caught all 30 (details below).

One harness, three modes

Why the read-back matters

A success screen is not proof of a save. We injected 8 kinds of bad save into a test app (a save that never landed, landed partly, landed twice, went to the wrong record, or changed another record), 9 runs each, alongside clean runs. Each check below was asked whether those 72 bad saves had succeeded:

This is a coverage test over a small, hand-written set of faults, not a real-world error rate. EFFECT_E2E.md has the method and raw results.

An agent working alone can't see these faults either. Claude Sonnet 5.5, using computer use, did a task on a test app whose server broke the save in 6 ways, 5 times each. The agent reported all 30 bad saves as done. Given the agent's run, openadapt-flow verify-run read each record back: it flagged all 30 and confirmed all 10 clean saves, with no false alarms. The test app shows "saved" before the server answers, so the screen gave the agent nothing to notice. That gap is what the read-back closes. SCOREBOARD.md has the method, the raw runs and the cost.

Try it in one command

uvx --python 3.12 openadapt quickstart --break-it

No account or API key. uv fetches Python 3.12 if you need it; with Python 3.10 to 3.12 already installed, python -m pip install --upgrade openadapt then openadapt quickstart --break-it does the same.

What it shows

The command runs a synthetic clinic task twice on your computer. The first run saves an entry and confirms it through the app's read-only API. For the second run, the test server is set up to reject the save, so it never reaches the record. This is the output from openadapt 1.17.0, with the break-it lines in the current wording:

[1/5] Record the demonstration against a real persistence boundary
[2/5] Compile and propose a fixture-specific effect contract
[3/5] Certify against the clinical-write policy
[4/5] Admit and execute under the standard profile
      VERIFIED in 5.6s; 0 model calls; the system of record holds 1 record(s)
[5/5] Emit the local run receipt
...
--- break-it: the same certified bundle, with the save set to fail ---

  Injected fault:      'optimistic' -- the server rejects the write
  On screen:           every on-screen check passed, including the saved message
  The verifier found:  1/2 declared effect(s) REFUTED by an independent
                       read of the system of record, which holds 0 record(s)
  The engine did:      HALTED at the consequential step instead of claiming
                       success (transaction: RECONCILIATION_REQUIRED, billable: no)
  Result:              Check the record. A save may have gone through. A person checks the record before anything is retried.

The quickstart tutorial walks through every line.

Build with it

Compile your agent's run

Your agent (Claude or OpenAI computer use, or your own loop, on your own model key) does the task once while OpenAdapt records it. A person reviews and admits the draft program, or a registered agent admits it under a policy a person signed. After that, the program runs instead of the agent.

from openadapt_flow.jit import compile_agent

jit = compile_agent(my_agent, task=task, app=app, verifier=verifier,
                    policy="clinical-write", cache_dir="jit-cache", model_key=key)
jit.run(note="Follow up in two weeks")    # the agent does it; a DRAFT is saved
# a person reviews, runs the canary, and admits the draft
jit.run(note="Recheck knee in ten days")  # the program runs: 0 model calls, save read back

Compile a computer-use agent's run has the full example. The same path compiles an agent's run in web apps and in Windows, macOS, Linux, RDP and Citrix apps.

Call programs from your agent

claude mcp add openadapt -- uvx --python 3.12 openadapt-agent serve

Each compiled workflow becomes one tool call. The agent decides what to enter; OpenAdapt makes the entry, checks it, and returns an outcome instead of the agent driving the screen. With no bundles, the server offers a synthetic sandbox workflow that returns every outcome. To serve your own workflows, run openadapt agent serve --mode production --bundles ./bundles; without --mode production the agent can list them but not run them. openadapt-agent emit-skill writes an Agent Skill for one workflow, and openadapt flow verify-run judges a write made by any agent or script against the system of record. Connect an agent covers other clients.

Record a task by hand

python -m pip install --upgrade openadapt
openadapt flow record --url https://staging.your-app.example --out rec
openadapt flow compile rec --out bundle --name my-first-workflow
openadapt flow replay bundle --url https://staging.your-app.example --run-dir run

replay drives the app at --url and performs every recorded action, including saves you can't undo. It doesn't ask first and doesn't check the saved record. Point it at a test or staging copy, never at production data.

A program that writes to a real system also needs record checks, a deployment file, certification, qualification, and an admission; then openadapt flow run runs it. By default each save waits for a person to approve it in OpenAdapt Desktop or on a paired phone. Your first workflow walks through it. Several recordings of the same task can become one program with branches and loops (openadapt flow induce).

To record a desktop app, install the recorder and your operating system's adapter, for example python -m pip install "openadapt[capture,windows]", or use OpenAdapt Desktop.

Judge or train any agent

openadapt flow verify-run judges a write made by any agent or script against the system of record. openadapt-flow serve-reward turns the same check into a reward for training or evaluating computer-use agents (REWARD_WORKER.md).

An agent alone, and an OpenAdapt program

The same referral task on a local OpenEMR 8.0.0.3 test copy with synthetic patients, 62 timed runs each (14 and 15 October 2026), judged by the same database read-back:

Both were equally correct here, and the agent matched or beat OpenAdapt in three other tests. Every result is published. That program was only about a fifth faster because it drove every page, as the agent does; the API step compiled from the recording's network traffic removes the page steps for single-form saves. OpenAdapt isn't affiliated with the OpenEMR project.

Every backend

The same record, compile, run and repair steps work on every surface. Where the app exposes its structure, OpenAdapt uses it; on remote desktops it works from pixels.

Each application on each surface needs its own qualification. Surfaces and Limits have the details.

How it works

flowchart LR
    S["Record, or let an agent do it once"] --> C["Compile"] --> Q["Certify and qualify"] --> A["Admit"] --> R["Run with checks"]
    R -.->|"stopped: approve a fix"| C

For each step the compiler uses the most reliable layer the app offers: page structure first (DOM, Windows UI Automation, macOS Accessibility, Linux AT-SPI), a direct API call where one is bound, and pixels (template, text and layout matching) only where nothing else exists, such as RDP and Citrix. Qualification runs the program against normal and fault cases before an admission lets that exact version run. At each save the program checks that the right record is on screen, makes the entry, and reads the record back.

How it compares

What it can do

• Learns from any successful run, a person's demonstration or a computer-use agent's verified run, and compiles it into a program that runs with no model calls.
• Uses the most reliable layer the app allows: page structure (DOM, UI Automation, Accessibility), direct API calls where one is bound, and pixels only for remote desktops.
• Builds branching programs from several demonstrations, repeated halts and background runs, and routes each item to the right verified variant.
• Works across surfaces: web, Windows, macOS, RDP and Citrix, in one workflow.
• Confirms the right record before every write and reads the saved record back from the system of record, never just the screen.
• Stops instead of guessing, never retries an uncertain save, and says what changed before a stop.
• Binds each approval to exactly what will be saved, with one-tap answers on a paired phone.
• Repairs itself when the app changes, probing a fix up to the save and asking a person to approve it.
• Works as a safety layer under any agent: MCP tools, a guarded save, Agent Skills and a CLI.
• Runs locally: no model calls on healthy runs, patient data redacted, signed receipts.

CAPABILITIES.md has the evidence and code for each.

What your agent gets back

The console's last line can read HALTED for both the second and third rows. transaction_outcome in report.json tells them apart, so base every rerun on it. Run outcomes lists every value.

When to use something else

• The app has an API for the task. Call the API.
• Every task is new. Use a computer-use agent. OpenAdapt pays off when the same entry repeats.
• There's no way to read the record back. The record check needs read access to the system of record: an API, the database, FHIR, or a separate signed-in session. Without one, a run can only end as finished but not checked.

Packages

pip install openadapt installs the openadapt command, the engine, and the MCP server. The product packages live in this repository:

Each package installs only what it imports. For example, pip install openadapt-privacy doesn't install the engine or the desktop app. CI checks each package's install size and dependency list (install weight).

Research on GUI agents lives in separate repositories: openadapt-evals (benchmarks) and openadapt-ml (model training). The packages map has the full list.

Security and procurement

Screens, typed values, and recordings stay on the computer or runner that runs OpenAdapt. The shareable receipt and the Cloud dashboard hold counts, digests, and closed values. A sanitized copy leaves only after a person approves it, and screenshots go to a model host only if you turn on model grounding. Security & procurement covers data handling, deployment boundaries, evidence, subprocessors, and release integrity. Plans are on the pricing page.

Earlier versions

The pre-1.0 version (v0.46.0) is frozen under legacy/. docs/LEGACY_FREEZE.md explains how to install it and how to migrate.

Contributing

Launcher, engine, MCP server, and contract changes all belong in this repository; engine changes go in packages/flow/. Contribution guide · Issues · Discussions · Report a vulnerability · Code signing policy

Maintained by OpenAdaptAI under the MIT License.