SurfSense is a free, open source note taking & knowledge management project written in Python and released under a custom open-source licence. It has 16,159 GitHub stars, 1,537 forks and 130 open issues, and was last pushed 4 hours ago. On this registry it ranks #11 of 25 tracked projects in Note Taking & Knowledge Management, with 5 head-to-head comparisons available. It gained 28 stars over the last 6 tracked days.

SurfSense — Privacy-first knowledge base with any AI model you choose

What is SurfSense?

What it is

SurfSense is an open-source NotebookLM alternative and privacy-first knowledge base that can work with any AI model chosen by the user. It is a Python productivity and knowledge-management project in the agent, RAG, LangChain, LangGraph, FastAPI, Next.js, and Ollama ecosystem. The project presents itself as an open web research platform for AI agents, with live data connectors and a built-in knowledge base that keeps findings searchable with citations.

The concrete problem it solves is the gap between capable agents and trustworthy live data. Official platform APIs are rate-limited, priced for enterprises, or missing, while scraping is brittle and browser-driven research can waste time and tokens. SurfSense aims to give agents one typed surface for structured JSON results from public sources, plus scheduled and event-triggered agents that turn findings into briefs and alerts. The README states that self-hosting remains free and open source.

Key capabilities

  • It exposes live web data through a REST API and an MCP server, so agents can query sources through one structured surface.
  • It supports connectors for Reddit, YouTube, Instagram, TikTok, Google Maps, Google Search, Indeed, Amazon, Walmart, and any page on the open web.
  • It returns structured JSON objects such as posts, comments, transcripts, reviews, search results, and pages.
  • It provides an agent harness with retries, structured output, and credit metering built into the research workflow.
  • It keeps findings in a knowledge base with searchable citations, and it supports briefs, alerts, chat, reports, podcasts, presentations, automations, and collaborative chats.

Who uses it and how

  • Agent developers use it as a tool layer for live web research, giving frameworks such as Claude, Cursor, or other agent systems native access to connectors.
  • Research teams use it to monitor public discussions, reviews, job postings, maps data, and search results without building separate scraping pipelines for each source.
  • Knowledge workers use the NotebookLM-style workflow to collect sources, ask questions, and generate cited outputs such as briefs, reports, podcasts, and presentations.
  • Self-hosters deploy it to keep research data and knowledge base content on their own infrastructure.

Getting started

The README describes the main deployment methods as self-hosting, using the REST API, and using the MCP server. It does not provide a specific package manager, Docker image, or hosted option in the excerpt.

When to use it — and when not to

SurfSense is useful when the goal is agent-driven research over live public web data with citations and self-hosting. Compared with NotebookLM, it emphasizes open web connectors, API access, MCP tools, and agent workflows. The trade-off is early-stage maturity and operational ambiguity: the repository metadata lists license NOASSERTION, zero contributors, zero years of age, and 130 open issues, and the excerpt does not specify database, storage, or SMTP requirements.

project readme (upstream, from github) — read inline

SurfSense, the open-source NotebookLM alternative for open web research

Discord Reddit

English | Español | Português | हिन्दी | 简体中文

MODSetter%2FSurfSense | Trendshift

SurfSense: The Open-Source NotebookLM Alternative for Open Web Research

SurfSense is the open-source NotebookLM alternative for AI agents, an open web research platform with live data connectors. Your agents research the live web with structured data from Reddit, YouTube, Instagram, TikTok, Amazon, Walmart, Google Maps, Google Search, Indeed, and any page on the open web, through one REST API or MCP server. Scheduled and event-triggered agents turn what they find into briefs and alerts, and a built-in knowledge base keeps every finding searchable with citations.

[!NOTE] 📢 A note for our NotebookLM-alternative users

For the past couple of months we built SurfSense as the best general research agent for your own knowledge, and that chapter earned us a community we are genuinely proud of. Agentic tools like Claude, OpenCode, Hermes, and OpenClaw have now proven that agents are the future, and reasoning over a static index is becoming something every capable agent does out of the box. What agents still lack is live data from the places where answers actually live, and the workflows around it. That is where we are pointing all of our energy: giving agents the primitives to research the open web.

Nothing you rely on is going away. Your knowledge base, chat with citations, reports, podcasts, presentations, automations, and collaborative chats all keep working, and self-hosting stays free and open source. Read the full announcement on our changelog.

Table of contents

Why agents need SurfSense

Ask any capable agent "what is Reddit saying about this product since launch?" or "what do the reviews of these ten places actually complain about?" and it has nowhere trustworthy to look. Official platform APIs are rate-limited, priced for enterprises, or missing entirely; scraping plumbing is brittle; and driving a browser with an LLM burns minutes and tokens per page. SurfSense gives agents the primitives instead:

  • One typed surface for wherever the data lives. Every connector is a REST endpoint returning structured JSON — posts, comments, transcripts, reviews, SERPs, pages. No rate-limit roulette, no HTML parsing, no browser loop.
  • An MCP server that exposes every connector as a native tool (surfsense_reddit_scrape, surfsense_google_search, and more) to Claude, Cursor, or any agent framework.
  • An agent harness, not just raw data: retries, structured output, and credit metering are built in, so agents go from a question to a cited brief without you building the plumbing.
  • Open source and self-hostable, so your research stays on your own infrastructure.

Live data connectors

Connector What your agents get Learn more
Reddit Posts, comments, and subreddit streams without the official API's rate limits Reddit Scraper API
YouTube Videos, transcripts, and comment threads at scale YouTube Scraper API
Instagram Public profiles, posts, and reels without the Graph API Instagram Scraper API
TikTok Videos, comments, hashtags, and profiles without Research API approval TikTok Scraper API
Google Maps Places, ratings, and reviews for local business research Google Maps Scraper API
Google Search Live SERPs for search research and monitoring Google Search API
Indeed Public job postings with salaries and full descriptions, by search or company Indeed Scraper API
Amazon Public product data: prices, ratings, offers, sellers, and best-seller ranks Amazon Product API
Walmart Public product data plus the full review history: prices, ratings, sellers, and variants Walmart Product & Review API
Web Crawl Any page on the open web as clean, structured content Web Crawling API
External MCP Connectors Bring any MCP server to your agents, with one-click OAuth for Notion, Slack, Jira, and more External MCP Connectors

The connector catalog is growing beyond social platforms and search; every new source lands as a typed endpoint on the same API and MCP server.

Billing is pay as you go: connectors bill per item actually returned, crawls per page successfully fetched, and failed calls are never billed. Self-hosted installs run with billing off. See pricing.

Quick start

Call a connector from code

Every connector is a REST endpoint you can call from any language with your SurfSense API key:

curl -X POST "$SURFSENSE_API_URL/workspaces/$WORKSPACE_ID/scrapers/reddit/scrape" \
  -H "Authorization: Bearer $SURFSENSE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "search_queries": ["your brand"],
    "community": "SaaS",
    "sort": "top",
    "time_filter": "week"
  }'

Each connector page has copy-paste examples in Python, JavaScript, Go, PHP, Ruby, Java, and C#.

Give the tools to your agents over MCP

Add the SurfSense MCP server to Claude, Cursor, or your own agent framework:

{
  "mcpServers": {
    "surfsense": {
      "url": "https://mcp.surfsense.com/mcp",
      "headers": { "Authorization": "Bearer ${SURFSENSE_API_KEY}" }
    }
  }
}

Your agent can now call every connector as a native tool. See the SurfSense MCP server page for the full tool list, or run the server locally from surfsense_mcp.

Use the cloud

Go to surfsense.com, log in, and ask the agent for live web data in plain English. New accounts start with $5 of free credit and no subscription.

Self-host for free

Run the entire platform, connectors, agents, automations, and the MCP server on your own infrastructure. Self-hosted installs ship with billing off, so scraping, crawling, and agent runs are limited only by your hardware and the model keys you bring.

Prerequisites: Docker Desktop must be installed and running.

For Linux/macOS:

curl -fsSL https://raw.githubusercontent.com/MODSetter/SurfSense/main/docker/scripts/install.sh | bash

For Windows:

irm https://raw.githubusercontent.com/MODSetter/SurfSense/main/docker/scripts/install.ps1 | iex

The install script sets up Watchtower automatically for daily auto-updates. To skip it, add the --no-watchtower flag. For Docker Compose, manual installation, and other deployment options, see the docs.

Everything else in the box

The research workspace that made SurfSense the leading open-source NotebookLM alternative is still here, and everything your agents gather lands in it.

Knowledge base

  • Upload PDFs, Office docs, images, and audio, or sync Google Drive, OneDrive, and Dropbox. 50+ file formats supported.
  • Hybrid semantic and full-text search with cited, Perplexity-style answers.
  • AI file sorting auto-organizes documents by source, date, and topic.

Deliverable studio

  • Downloadable

readme truncated — read the full docs on github

Frequently asked questions

Is SurfSense free to use?

SurfSense is open source. There is no licence fee and no seat count — you can self-host it or, where the project offers one, pay a vendor for a managed version instead.

What does SurfSense do?

Privacy-first knowledge base with any AI model you choose

What is SurfSense written in?

SurfSense is primarily written in Python. Its source is publicly available at https://github.com/modsetter/surfsense, and it has 16,159 GitHub stars.