Magnitude is an open-source, vision-first browser agent written in TypeScript that lets developers control a real browser with natural language, and it is aimed at engineers who need to automate repetitive web work, move data between applications that lack APIs, or test web interfaces rather than click through them by hand.

What it is

Magnitude is an Apache-2.0 licensed browser agent that uses a visually grounded language model to see and understand any interface, plan actions, and then execute them with precise mouse and keyboard input. It is published in the Node and TypeScript ecosystem through npm, with a scaffolding package for new projects and a separate test-runner package for existing web applications. Its topic list places it in the browser automation space alongside Playwright and Selenium, and the README presents vision as the architectural difference from both.

The concrete problem it replaces is element-addressing by DOM inspection. Most browser agents wrap numbered boxes around page elements, which the README argues generalises poorly on complex modern sites, because the agent's behaviour is tied to the page's structure. Magnitude instead asks the model to specify pixel coordinates, which keeps the agent independent of DOM shape and, per the README, positions it for future use against desktop apps and VMs. It also targets a second failure mode: the "high-level prompt plus tools, work until done" pattern that demos well but does not hold up in production.

Key capabilities

• Navigate, Interact, Extract, and Verify are the four stated capabilities: seeing and understanding an interface, executing mouse and keyboard actions, pulling structured data out, and asserting on results.
• agent.act() accepts high-level tasks, such as 'Create a task', with an optional data payload the agent uses where appropriate.
• The same API handles low-level instructions, for example dragging an item to the top of a specific column.
• agent.extract() returns data matched against a supplied zod schema, and can derive new insights such as a rated difficulty field, not only copy existing text.
• A built-in test runner installs as magnitude-test and generates tests/magnitude, containing magnitude.config.ts and an example test file named example.mag.ts.
• Custom actions and prompts are configurable at both agent and action level, giving a choice between granular actions and larger flows.
• A native caching system is intended to make runs deterministic, and is documented as in progress.

Who uses it and how

• Teams automating routine web tasks that have no API surface, using agent.act() as the substitute for manual clicking.
• Data engineers extracting structured records by describing the target list in natural language and a zod schema.
• Web application teams running visual assertions in a test suite, from the terminal on a developer machine and, per the linked documentation, inside CI/CD pipelines.
• Developers building their own browser agents who want Magnitude as a lower-level building block rather than an end-user tool.
• Enterprises that need additional features or support contact the maintainers directly via [email protected] or a scheduled call.

Getting started

For a new project, run npx create-magnitude-app, which scaffolds a project and an example script. To add the test runner to an existing app, run npm i --save-dev magnitude-test && npx magnitude init. Both paths require a large visually grounded model: Claude Sonnet 4 is recommended, and Qwen-2.5VL 72B is also compatible.

How it compares

The project's topic list places it next to Playwright and Selenium, but Magnitude's stated distinction is that it does not depend on DOM structure, addressing elements by pixel coordinate instead, which is also the basis for its desktop and VM ambitions. Against the general class of demo-oriented prompt-and-tools agents, the README draws the line at controllable and repeatable automation through configurable abstraction levels and caching.

When to use it — and when not to

Adopting Magnitude means taking on a dependency on a large visually grounded model, so planning should include the cost, latency, and configuration of that model, along with the npm packages and the generated tests/magnitude directory. Teams that need guaranteed deterministic replay today should be cautious, because the caching system that provides it is still marked in progress. The README is short and defers most operational detail to docs.magnitude.run, so anyone unwilling to read the external documentation, or unable to call a strong vision model, is a poor fit.

// Magnitude can handle high-level tasks
await agent.act('Create a task', {
    // Optionally pass data that the agent will use where appropriate
    data: {
        title: 'Use Magnitude',
        description: 'Run "npx create-magnitude-app" and follow the instructions',
    },
});

// It can also handle low-level actions
await agent.act('Drag "Use Magnitude" to the top of the in progress column');

// Intelligently extract data based on the DOM content matching a provided zod schema
const tasks = await agent.extract(
    'List in progress tasks',
    z.array(z.object({
        title: z.string(),
        description: z.string(),
        // Agent can extract existing data or new insights
        difficulty: z.number().describe('Rate the difficulty between 1-5')
    })),
);

npx create-magnitude-app

npm i --save-dev magnitude-test && npx magnitude init