Waverless is a Go-based, Apache-2.0-licensed serverless GPU task orchestration system — the scheduling layer behind wavespeed.ai — built for teams that run GPU inference workers across Kubernetes, Novita Serverless, or Docker and need one queue, autoscaler, and HTTP API in front of them.

What it is

Waverless is a serverless GPU task orchestration system written in Go. It sits between clients and GPU workers: clients submit tasks to a V1 HTTP API, an API server holds them in a task queue, worker management and an autoscaler decide how many workers should exist, and provider integrations create and destroy those workers on Kubernetes, Novita Serverless, or Docker. State lives in Redis and MySQL, and a Web UI is part of the core. The project is the scheduling layer behind wavespeed.ai and carries the topics go, gpu, inference, orchestration, scheduler, serverless, and wavespeed.

The concrete problem it solves is the glue code teams write when they put GPU inference behind a queue. That usually means a hand-rolled task queue, a custom autoscaler reacting to queue depth, provider-specific worker provisioning, and a shutdown path that drops in-flight tasks during rolling updates. Waverless replaces that stack with one pull-based design: workers actively pull tasks instead of receiving pushed work, autoscaling reacts to queue depth, priority, and resource usage, and graceful shutdown is built in so rolling updates lose no tasks. For teams already on the runpod-python SDK, the README describes RunPod compatibility that allows zero-code migration, so the hosted RunPod scheduling path can be swapped for a self-run deployment.

Key capabilities

• Pull-based architecture: workers actively pull tasks rather than receiving pushes, which the README presents as better load balancing.
• RunPod compatibility with zero-code migration from the runpod-python SDK.
• Multi-provider backends covering Kubernetes, Novita Serverless, and Docker.
• Smart autoscaling driven by queue depth, task priority, and resource awareness.
• Graceful shutdown with zero task loss during rolling updates.
• Task API with POST /v1/my-endpoint/run to submit work and GET /v1/status/{task_id} to check it.
• Redis and MySQL as the backing store, plus a Web UI inside the core.

Who uses it and how

• Teams running GPU inference workers on Kubernetes, deployed with ./deploy.sh install.
• Developers working locally with docker-compose up -d mysql redis, then copying config/config.example.yaml to config/config.yaml, then go run cmd/main.go.
• Teams migrating an existing runpod-python workload, which the README says requires no code changes.
• Operators running a mixed fleet, since one core manages Kubernetes, Novita Serverless, and Docker workers behind a single queue.
• Workloads with mixed priorities, because autoscaling is priority-aware and shutdown is graceful.

Getting started

Bring up MySQL and Redis with docker-compose up -d mysql redis, copy config/config.example.yaml to config/config.yaml, and run go run cmd/main.go for local development; for Kubernetes, use ./deploy.sh install.

How it compares

The facts do not provide a list of paid products that Waverless replaces, so no licence, cost-model, or data-ownership comparison against commercial alternatives can be made here. It slots alongside Kubernetes, Docker, Novita Serverless, and RunPod, which appear as provider backends or as a migration source rather than as products it displaces.

When to use it — and when not to

A self-hoster must operate Redis and MySQL, run a Go service, and manage the provider accounts and clusters that host the GPU workers. Teams that want a fully hosted serverless GPU product with no operational surface should not pick this one. Two weaknesses are evident in the facts: the licence is inconsistent, since the registry metadata lists Apache-2.0 while the README closes with "MIT License", and the README itself is thin — a feature list and an architecture diagram, with detail pushed into docs/ARCHITECTURE.md, docs/USER_GUIDE.md, and docs/DEVELOPER_GUIDE.md.

flowchart TB
    subgraph Clients
        direction LR
        Client[Client V1 API]
        WebUI[Web UI]
    end

    subgraph Core["Waverless API Server"]
        direction TB
        Queue[Task Queue]
        WM[Worker Mgmt]
        Autoscaler[Autoscaler]
        Store[(Redis + MySQL)]
    end

    subgraph Provider
        direction LR
        K8s[K8s]
        Novita[Novita]
        Docker[Docker]
    end

    subgraph Workers
        direction LR
        W1[Worker A]
        W2[Worker B]
        W3[Worker ...]
    end

    Clients -->|submit| Core
    Core --> Provider
    Provider -->|manage| Workers
    Workers -->|pull tasks| Core

    style Clients fill:#4a90a4,color:#fff
    style Core fill:#2d5a7b,color:#fff
    style Provider fill:#5d8aa8,color:#fff
    style Workers fill:#7fb3d3,color:#000

# Local development
docker-compose up -d mysql redis
cp config/config.example.yaml config/config.yaml
go run cmd/main.go

# Kubernetes deployment
./deploy.sh install

# Submit task
curl -X POST http://localhost:8090/v1/my-endpoint/run \
  -H "Content-Type: application/json" \
  -d '{"input": {"prompt": "hello world"}}'

# Check status
curl http://localhost:8090/v1/status/{task_id}