Open source llm-eval projects
Every project in the registry tagged llm-eval, ranked by real GitHub adoption.
Test your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT, Claude, Gemini, DeepSeek, and more. Simpl
Open-source LLM tracing & evaluation for AI optimization
🐢 Open-Source Evaluation & Testing library for LLM Agents
Related tags
Frequently asked questions
How many open source llm-eval projects are there?
This registry tracks 3 projects tagged llm-eval, with 44,755 GitHub stars between them. The most-adopted is promptfoo at 26,719 stars.
Are these llm-eval projects free to use?
Yes — 2 of the 3 carry an explicit open-source licence across 2 distinct licences, so there is no licence fee. Where a project also sells a hosted or enterprise version, the self-hosted path remains free.
Which llm-eval project should I choose?
The list above is ranked by GitHub stars, but stars measure attention rather than fit. Check three things on each card: the licence (permissive versus copyleft), the language it is written in, and the last-push date — a high-star project that has not been pushed in a year is a liability.
Are these llm-eval projects still maintained?
3 of the 3 were pushed in the last 90 days, and every card shows its exact last-push date so you can see the rest. Sort your shortlist by that date before committing to a migration.