agentql is a free, open source data extraction & web scraping project written in Python and released under MIT. It has 1,463 GitHub stars, 171 forks and 10 open issues, and was last pushed 4 days ago. On this registry it ranks #67 of 105 tracked projects in Data Extraction & Web Scraping, with 5 head-to-head comparisons available.

What is agentql?

AgentQL is an MIT-licensed suite of tools — an AI-powered query language plus Python and JavaScript SDKs, a REST API, and a browser debugger — that connects LLMs and AI agents to live web pages for data extraction and workflow automation, aimed at developers building scrapers, RPA scripts, and agentic automations on top of Playwright.

What it is

AgentQL is a query language and tooling suite for pulling data out of, and driving interactions on, live websites. It ships as a Python SDK, a JavaScript SDK, a REST API endpoint for running queries without an SDK, a Chrome debugger extension for refining queries in real time, and a hosted Playground, with documented integrations into agent frameworks and automation tools such as Langchain, Zapier, and an MCP server.

The concrete problem it addresses is selector fragility in browser automation. Scripts written against CSS or XPath selectors break whenever a site's markup changes, so teams maintaining Playwright scraping and RPA workflows spend their time repairing selectors rather than collecting data. AgentQL replaces those selectors with natural language queries that locate elements by page content, work across similar sites, and self-heal as a UI evolves over time.

Key capabilities

  • The AgentQL Query Language defines extraction and interaction in natural language, with structured output determined by the shape of the query itself.
  • Transforms and extraction happen inside the query, so derived values are computed without post-processing in application code.
  • Official Python SDK and JavaScript SDK integrate directly with Playwright for advanced automation and testing.
  • A REST API endpoint runs queries in languages or environments where neither SDK is practical.
  • Cross-site compatibility allows the same query to run against different sites that hold similar content.
  • Queries work on public and private pages, on any URL, including content behind authentication and content generated dynamically.
  • A Debugger Browser Extension and the Playground let queries be tested on live sites and exported as Python scripts.

Who uses it and how

  • Web scraping teams collecting structured records at scale, where cross-site compatibility lets one query cover several comparable sources.
  • RPA and automation groups replacing brittle selector-based browser steps, using the SDKs to close cookie dialogs, dismiss popups, and drive forms.
  • AI agent builders wiring page access into LLM workflows through the MCP server, Langchain, or Zapier rather than writing bespoke scrape code.
  • QA and test engineers already on Playwright who want content-based element targeting alongside their existing test suites.
  • Analysts working in hosted notebooks, supported by the published Google Colab examples for tasks such as closing popups and comparing product prices.

Getting started

Install AgentQL's Python SDK or JavaScript SDK from the documented installation pages, or call the REST API if no SDK fits the stack; the quick start guide targets a five-minute first query, and the Playground exports runnable Python.

How it compares

No paid alternatives are listed in the facts, so the nearest named neighbours are the tools AgentQL integrates with rather than displaces: Playwright supplies the browser automation layer, while Langchain, Zapier, and the MCP server supply the agent and workflow layer. It sits above Playwright as a query and selection layer, and inside agent stacks as the component that reads and acts on web pages.

When to use it — and when not to

Adopt it when selector maintenance is the recurring cost in a Playwright or scraping codebase, and when page content is stable enough for natural language queries to be reviewed and refined in the debugger. Teams that need a fully self-contained query engine should note that the README documents a REST API endpoint and a hosted Playground but describes no self-hosted execution service, and the debugger ships only as a Chrome extension. The README also leaves rate limits, quotas, and operational requirements unstated, so those should be confirmed before committing a production pipeline to it.

project readme (upstream, from github) — read inline

AgentQL
</a>

AgentQL connects LLMs and AI agents to the entire web

Follow on X Follow on LinkedIn Join our Discord

Python version GitHub Repo stars GitHub watchers

What is AgentQL?

AgentQL is a suite of tools for extracting data and automating workflows on live web sites featuring an AI-powered query language, Python and JavaScript SDKs, a browser-based debugger, and a REST API endpoint. It uses natural language queries to pinpoint data and elements on any web page, including authenticated and dynamically generated content. Users can define structured data output and apply transforms within queries. AgentQL's natural language selectors find elements intuitively based on the content of the web page and work across similar web sites, self-healing as UI changes over time.

Get started in 5 minutes with our quick start!

Features

  • Playwright AgentQL's Python SDK and JavaScript SDK seamlessly integrates with Playwright for advanced automation and testing.
  • Cross-site compatibility lets you use the same query across different sites with similar content.
  • Structured output defined by the shape of your query.
  • Natural language selectors find elements and data anywhere on a site using intuitive queries.
  • Transforms and extracts data in your queries.
  • Works on any page, public or private, any site, any URL, even behind authentication.
  • Resilience to UI changes means queries work regardless of how a page's structure changes over time.

Tools

  • Python SDK for running automation and scraping scripts with AgentQL queries in Python.
  • JavaScript SDK for running automation and scraping scripts with AgentQL queries in JavaScript.
  • REST API for executing queries without an SDK.
  • Integrations with agent frameworks and automation tools like Langchain and Zapier and our MCP server.
  • Debugger Browser Extension lets you debug and finesse queries in real-time on live sites.
  • AgentQL Query Language lets you define queries with natural language.
  • Playground for playing with AgentQL lets you export python scripts and optimize queries with prompts.

Examples

Example Name Python JavaScript
Getting Started Script Script
Close Cookie Dialog Script / Colab Script
Close Popup Windows Script / Colab Script
Compare Product Prices Script / Colab Script
Get Element by Prompt Script Script
Infinite Scroll Script / Colab Script
Use Remote Browser Script Script
Use Existing Browser Script Script
Query List Items Script Script
Site Login Script / Colab Script
Headless Browser Script / Colab

readme truncated — read the full docs on github

Frequently asked questions

Is agentql free to use?

agentql is open source under the MIT licence. There is no licence fee and no seat count — you can self-host it or, where the project offers one, pay a vendor for a managed version instead.

What does agentql do?

AgentQL is a suite of tools for connecting your AI to the web. Featuring a query language and Playwright integrations for interacting with elements and extracti

What is agentql written in?

agentql is primarily written in Python. Its source is publicly available at https://github.com/tinyfish-io/agentql, and it has 1,463 GitHub stars.