💪 Hercules

Testing modern web applications can be difficult, with frequent changes and complex features making it hard to keep up. That's where Hercules comes in. Hercules is the world's first open-source testing agent, built to handle the toughest testing tasks so you don't have to. It turns simple, easy-to-write Gherkin steps into fully automated end to end tests—no coding skills needed. Whether you're working with tricky enterprise platforms or running tests in your CI/CD pipeline, Hercules adapts to your needs and takes care of the details. With Hercules, testing becomes simple, reliable, and efficient, helping teams everywhere deliver better software. Here's a quick demo of lead creation using natural english language test (without any code):

As you saw, using Hercules is as simple as feeding in your Gherkin features, and getting the results:

At TestZeus, we believe that trustworthy and open-source code is the backbone of innovation. That's why we've built Hercules to be transparent, reliable, and community-driven.

Our mission? To democratize and disrupt test automation, making top-tier testing accessible to everyone, not just the elite few. No more gatekeeping—everyone deserves a hero on their testing team!

Video Tutorials: @TestZeus

🚀 Getting Started with TestZeus Hercules

• Introduction to TestZeus Hercules
  Learn about the core features of TestZeus Hercules and how it can streamline end-to-end testing for your projects.

• Installation and Setup Guide
  Step-by-step instructions for installing and configuring TestZeus Hercules in your environment.
  Watch now

🧪 Writing and Executing Test Cases

• Creating BDD Test Cases
  Learn how to write Behavior-Driven Development (BDD) test cases for Hercules and use dynamic testdata.
  Watch now

🌐 Integrations and Advanced Features

• Testing Multilingual content
  Learn how Hercules interacts with web browsers to perform Testing on Multilingual content via Multilingual testcase.
  Watch now

🛠️ Customization and Community Tools

• Enhancing Hercules with Community-Driven Tools
  Discover how to customize Hercules and incorporate additional tools provided by the community.
  Watch now

🛠️ API Testing

• API testing all the way, new ways to do end to end
  Watch now

🛠️ Security Testing

• Security Testing done end to end
  Watch now

🛠️ Visual validations and assertions

• Using vision capabilities to check snapshots and components on the application
  Watch now

🐍 Python Sandbox Execution

• Execute custom Python scripts directly from Gherkin tests
  Run complex automation workflows, custom business logic, and reusable components with full Playwright access. Perfect for scenarios that need advanced selector strategies, conditional logic, or data processing. Read the docs

Quick Example:

# In your feature file
And execute the apply_filter function from script at "scripts/apply_filter.py" with filter_type as "Turtle Neck"

# In opt/scripts/apply_filter.py
async def apply_filter(filter_type: str) -> dict:
    """Apply filter with multiple fallback strategies."""
    # page, logger, and other tools automatically available!
    await page.wait_for_selector('[data-filter-section]')
    
    # Try multiple selector strategies
    for selector in [f'input[value="{filter_type}"]', 
                     f'label:has-text("{filter_type}") input']:
        if await page.locator(selector).count() > 0:
            await page.locator(selector).click()
            break
    
    return {"status": "success", "filter": filter_type}

Features:

• 🎯 Full Playwright API access
• 🔒 Multi-tenant security (executor, data, API, restricted modes)
• 📦 Auto-injected modules (page, browser, logger, asyncio, etc.)
• 🔄 Reusable across multiple tests
• ⚙️ Configurable via environment variables

Configuration:

# Set tenant for module access
export SANDBOX_TENANT_ID="executor_agent"  # Full access: requests, pandas, numpy, BeautifulSoup

# Or use CLI
testzeus-hercules --sandbox-tenant-id executor_agent --input-file test.feature

⚙️ Installation and Usage

Hercules offers multiple ways to get started, catering to different user preferences and requirements. If you are new to the Python ecosystem and don't know where to begin, dont worry and read the footnotes on understanding the basics.

For a quick taste of the solution, you can try the notebook here:

• Note: Colab might ask you to restart the session after installing TestZeus Hercules dependencies. Please restart the session if required and continue the execution. Also, we recommend one of the approaches below for getting the full flavor of the solution.

Approach 1: Using PyPI Package

Installation

Install Hercules from PyPI:

pip install testzeus-hercules

Hercules uses Playwright to interact with web pages, so you need to install Playwright and its dependencies:

playwright install --with-deps

For detailed information about project structure and running tests, please refer to our Run Guide. For migration-specific runtime behavior, see docs/Migration/MIGRATION.md and docs/Migration/ARCHITECTURE.md.

Basic Parameters

Once installed, you will need to provide some basic parameters to run Hercules:

• --input-file INPUT_FILE: Path to the input Gherkin feature file to be tested.
• --output-path OUTPUT_PATH: Path to the output directory. The path of JUnit XML result and HTML report for the test run.
• --test-data-path TEST_DATA_PATH: Path to the test data directory. The path where Hercules expects test data to be present; all test data used in feature testing should be present here.
• --project-base PROJECT_BASE: Path to the project base directory. This is an optional parameter; if you populate this, --input-file, --output-path, and --test-data-path are not required, and Hercules will assume all the three folders exist in the following format inside the project base:

PROJECT_BASE/
├── gherkin_files/
├── input/
│   └── test.feature
├── log_files/
├── output/
│   └── run_<timestamp>/
│       ├── test.feature_result.html
│       └── test.feature_result.xml
├── proofs/
│   └── User_opens_Google_homepage/
│       └── run_<timestamp>/
│           ├── network_logs.json
│           ├── screenshots/
│           └── videos/
└── test_data/
    └── test_data.txt

• --agents-llm-config-file AGENTS_LLM_CONFIG_FILE: Optional path to an agents_llm_config.json file for per-agent model routing.
• --agents-llm-config-file-ref-key AGENTS_LLM_CONFIG_FILE_REF_KEY: Optional top-level provider/profile key inside that file.
• --llm-model, --llm-model-api-key, --llm-model-base-url, and related direct LLM flags are still supported for simple single-profile setups. Use agents_llm_config.json when you need separate planner, navigation, memory, and helper model settings.

Environment Variables

In addition to command-line parameters, Hercules supports various environment variables for configuration:

• LLM_MODEL_NAME: Model name for direct single-model setup. Example: gpt-4o
• LLM_MODEL_API_KEY: API key for the selected model provider.
• LLM_MODEL_BASE_URL: Optional OpenAI-compatible base URL.
• AGENTS_LLM_CONFIG_FILE: Optional path to agents_llm_config.json for per-agent model routing.
• AGENTS_LLM_CONFIG_FILE_REF_KEY: Optional top-level provider/profile key to activate from that config file.
• BROWSER_TYPE: Type of browser to use (chromium, firefox, webkit). Default: chromium
• HEADLESS: Run browser in headless mode (true, false). Default: true
• BROWSER_RESOLUTION: Browser window resolution (format: width,height). Example: 1920,1080
• BROWSER_COOKIES: Set cookies for the browser context. Format: JSON array of cookie objects. Example: [{"name": "session", "value": "123456", "domain": "example.com", "path": "/"}]
• RECORD_VIDEO: Record test execution videos (true, false). Default: true
• TAKE_SCREENSHOTS: Take screenshots during test (true, false). Default: true

For a complete list of environment variables, see our Environment Variables Guide.

Running Hercules

Set the direct model environment variables and run Hercules:

export LLM_MODEL_NAME=gpt-4o
export LLM_MODEL_API_KEY=replace-me
export LLM_MODEL_BASE_URL=https://api.openai.com/v1

testzeus-hercules \
  --input-file opt/input/test.feature \
  --output-path opt/output \
  --test-data-path opt/test_data

Use agents_llm_config.json only when you need separate planner, navigation, memory, and helper model settings; see docs/run_guide.md.

⚙️ Running Hercules on a Windows Machine

To set up and run Hercules on a Windows machine:

1. Open PowerShell in Administrator Mode:

   • Click on the Start Menu, search for PowerShell, and right-click on Windows PowerShell.
   • Select Run as Administrator to open PowerShell in administrator mode.

2. Navigate to the Helper Scripts Folder:

   • Use the cd command to navigate to the folder containing the hercules_windows_setup.ps1 script. For example:

     cd path\to\helper_scripts
     

3. Run the Setup Script:

   • Execute the script to install and configure Hercules:

     .\hercules_windows_setup.ps1
     

4. Follow On-Screen Instructions:

   • The script will guide you through installing Python, Playwright, FFmpeg, and other required dependencies.

5. Run Hercules:

   • Once the setup is complete, you can run Hercules from PowerShell or Command Prompt using the following command:

     testzeus-hercules --input-file opt/input/test.feature --output-path opt/output --test-data-path opt/test_data
     

Supported AI Models for TestZeus-Hercules

• OpenAI: Use models with reliable tool-calling support, such as GPT-4o, GPT-4.1, o-series models, or GPT-5 family models where available.
• Anthropic: Use Claude models with tool-calling support.
• Gemini / Vertex AI: Supported through OpenAI-compatible gateways such as LiteLLM. Keep tool schemas simple; tuple-style public tool inputs are not provider-safe.
• Gemma: Supported through gateways or local providers that expose compatible chat and tool-calling behavior.
• Groq, Mistral, Ollama, DeepSeek, Bedrock, Azure, and other providers can be used when they support OpenAI-compatible chat and function/tool calling.
• Local: Ollama and other local providers can be used when their model reliably returns strict planner JSON and supports the required tool-calling behavior. The sample Ollama Modelfile lives at docs/Migration/Modelfile.

The planner model should be strong at structured JSON reasoning. Navigation models must support tool calling. For a quick run, direct LLM_MODEL_* environment variables are enough. Use agents_llm_config.json only when you need separate planner, navigation, memory, and helper model settings. See docs/run_guide.md and docs/environment_variables.md for full examples, including LiteLLM proxy setup.

Execution Flow

Upon running the command:

• Hercules will start and attempt to open a web browser (default is Chromium).
• SimpleHercules creates a LangGraph state graph for the run.
• The planner node turns the feature file into strict JSON containing the current plan, next_step, target_helper, and assertion fields.
• The executor node routes the planner's target_helper to a navigation helper such as browser_nav_agent, api_nav_agent, sec_nav_agent, sql_nav_agent, time_keeper_nav_agent, mcp_nav_agent, or executor_nav_agent.
• Navigation helpers bind registered LangChain StructuredTool objects and run a bounded tool-call loop. Browser tasks use tools such as open_url, click, get_interactive_elements, bulk_enter_text, and bulk_select_option.
• Helper responses are fed back to the planner until the planner terminates or routes to assertion handling.

Output and Logs

Once the execution is completed:

• Logs explaining the sequence of events are generated.
• The best place to start is the output-path, which will have the JUnit XML result file as well as an HTML report regarding the test case execution.
• You can also find proofs of execution such as video recordings, screenshots per event, and network logs in the proofs folder.
• To inspect the planner and helper trace, start with agent_inner_thoughts.json under log_files//run_/.

Sample Feature File

Here's a sample feature file:

Feature: Product Search on Demo Store

 Scenario: Successfully find a product

   Given I am on the demo store home page
   When I enter "wireless headphones" in the search field
   And I click on the "Search" button
   And I open the first matching product result
   Then I should see the product details page
   And I should see "wireless headphones" in the product title or description

Sample Result Screenshot

Sample HTML Result Screenshot

Sample XML Result Screenshot

Approach 2: Using Docker

For all the scale lovers, Hercules is also available as a Docker image.

Pull the Docker Image

docker pull testzeus/hercules:latest

Running Hercules in Docker

Run the container using:

docker run --env-file=.env \
  -v ./agents_llm_config.json:/testzeus-hercules/agents_llm_config.json \
  -v ./opt:/testzeus-hercules/opt \
  --rm -it testzeus/hercules:latest

• Environment Variables: All the required environment variables can be set by passing an .env file to the docker run command.
• LLM Configuration: Either pass direct LLM_MODEL_* values in .env or mount agents_llm_config.json and set AGENTS_LLM_CONFIG_FILE=agents_llm_config.json plus AGENTS_LLM_CONFIG_FILE_REF_KEY=. LiteLLM is one useful provider profile when you run through an OpenAI-compatible proxy.
• Mounting Directories: Mount the opt folder to the Docker container so that all the inputs can be passed to Hercules running inside the container, and the output can be pulled out for further processing. The repository has a sample opt folder that can be mounted easily.
• Simplified Parameters: In the Docker case, there is no need for using --input-file, --output-path, --test-data-path, or --project-base as they are already handled by mounting the opt folder in the docker run command.

Browser Access in Docker

• While running in Docker mode, understand that Hercules has access only to a headless web browser.
• If you want Hercules to connect to a visible web browser, try the CDP URL option in the environment file. This option can help you connect Hercules running in your infrastructure to a remote browser like BrowserBase or your self-hosted grid.
• Use CDP_ENDPOINT_URL to set the CDP URL of the Chrome instance that has to be connected to the agent.

Docker and Remote Browser Support

When running Hercules in Docker, you can connect to remote browser instances using various platforms:

1. BrowserStack Integration:

export BROWSERSTACK_USERNAME=your_username
export BROWSERSTACK_ACCESS_KEY=your_access_key
export CDP_ENDPOINT_URL=$(python helper_scripts/browser_stack_generate_cdp_url.py)

2. LambdaTest Integration:

export LAMBDATEST_USERNAME=your_username
export LAMBDATEST_ACCESS_KEY=your_access_key
export CDP_ENDPOINT_URL=$(python helper_scripts/lambda_test_generate_cdp_url.py)

3. BrowserBase Integration:

export CDP_ENDPOINT_URL=wss://connect.browserbase.com?apiKey=your_api_key

4. AnchorBrowser Integration:

export CDP_ENDPOINT_URL=wss://connect.anchorbrowser.io?apiKey=your_api_key

Note: Video recording is only supported on platforms that use connect_over_cdp (BrowserBase, AnchorBrowser). Platforms using the connect API (BrowserStack, LambdaTest) do not support video recording.

Output and Logs

After the command completion:

• The container terminates, and output is written in the mounted opt folder, in the same way as described in the directory structure.
• You will find the JUnit XML result file, HTML reports, proofs of execution, and logs in the respective folders.

Approach 3: Building and running from Source Code

For the hardcore enthusiasts, you can use Hercules via the source code to get a complete experience of customization and extending Hercules with more tools.

Prerequisites

• Ensure you have Python 3.13 installed for the branch CI target. The package metadata supports Python `>=3.11,/ │ ├── test.feature_result.html │ └── test.feature_result.xml ├── log_files/ ├── proofs/ │ └── User_opens_Google_homepage/ │ └── run_/ │ ├── network_logs.json │ ├── screenshots/ │ └── videos/ └── test_data/ └── test_data.txt

8. Interactive Mode

   You can also run Hercules in interactive mode as an instruction execution agent, which is more useful for RPA and debugging test cases and Hercules's behavior on new environments while building new tooling and extending the agents.

   make run-interactive
   

   • This will trigger an input prompt where you can chat with Hercules, and it will perform actions based on your commands.

Approach 4: Setting Up via helper_script_custom.sh

For those who want a fully automated setup experience on Linux/macOS environments, create the script below locally. It installs TestZeus Hercules and sets up the base opt project directories with direct LLM_MODEL_* environment variables for a simple local run.

Prerequisites

• Ensure you have Python 3.13 installed, or another supported Python version in the >=3.11, .env opt/input/test.feature opt/test_data/test_data.txt =3.11,<3.14, and this branch targets Python 3.13 in CI. You can verify your version by running:

python --version
# or
python3 --version

If Python isn't installed, download it here or if you are on Windows, just follow the instructions here.

🏰 2. Creating and Activating a Virtual Environment

To set up a clean Python environment for your project:

# Create a virtual environment named 'venv'
python -m venv venv

# Activate the virtual environment
# On Windows:
venv\Scripts\activate

# On macOS/Linux:
source venv/bin/activate

You'll notice your terminal prompt changes—this indicates the virtual environment is active.

To deactivate the virtual environment later, simply run:

deactivate

📥 3. Installing TestZeus Hercules

Once your virtual environment is activated, install the latest version of TestZeus Hercules directly from PyPI:

pip install testzeus-hercules

🔄 4. Updating TestZeus Hercules

To update to the latest version, use:

pip install --upgrade testzeus-hercules

❌ 5. Uninstalling TestZeus Hercules

If you need to remove the package:

pip uninstall testzeus-hercules

🔍 6. Checking Installed Packages

To verify that TestZeus Hercules is installed and check its version:

pip list

🔗 Additional Tips for Java/JS Developers

• pip is like npm or mvn—it's used for installing and managing Python packages.
• Python's simplicity means fewer configuration files—most things can be done directly from the command line.

💡 Next Steps

Now that TestZeus Hercules is installed and ready to go, dive into our documentation to learn how to create and run your first test cases with ease!

Star History

✅ Citation

If you use Hercules in your research or project, please cite:

@software{testzeus_hercules2024,
  author = {Agnihotri, Shriyansh and Gupta, Robin},
  title = {Hercules: World's first open source testing agent},
  year = {2024},
  publisher = {GitHub},
  url = {https://github.com/test-zeus-ai/testzeus-hercules/}
}