head to head · open source
promptfoo vs codex-keysmith
promptfoo has 26,809 GitHub stars, 2,648 forks, 651 open issues and last shipped yesterday. codex-keysmith has 4,792 stars, 740 forks, 5 open issues and last shipped 1 months ago. promptfoo leads on adoption by 459% (26,809 vs 4,792 stars). promptfoo is written in TypeScript under MIT; codex-keysmith is written in Python under MIT. promptfoo has attracted 10% as many forks as stars, codex-keysmith 15%. promptfoo was the more recently maintained of the two, and both are self-hostable with no licence fee. The two share 1 topic tag (prompt-engineering), so they are genuine substitutes rather than adjacent tools.
Two open source projects, one decision. Both are free and self-hostable — the differences are community size, license terms, language stack and release pace.
← all 20902 open source comparisons
Side by side
| promptfoo | codex-keysmith | |
|---|---|---|
| GitHub stars | ★ 27K | ★ 4.8K |
| License | MIT | MIT |
| Written in | TypeScript | Python |
| Last push | 2026-11-04 | 2026-09-27 |
| Forks | ⑂ 2.6K | ⑂ 740 |
| Self-hosting | Yes | Yes |
| Data ownership | Your server | Your server |
pick promptfoo if
- You weight community size — 27K stars and counting
- You want the MIT license terms
- Your stack matches TypeScript
- You value the larger contributor base for long-term maintenance
pick codex-keysmith if
- You want the codex-keysmith feature set and don't need the biggest community
- You prefer the MIT license terms
- Your stack matches Python
- You evaluated both and codex-keysmith fits your workflow better
About promptfoo
Promptfoo: LLM evals & red teaming promptfoo is a CLI and library for evaluating and red teaming LLM apps. Stop the trial and error approach start shipping secure, reliable AI apps. Website · Getting Started · Red Teaming · Documentation · Discord Promptfoo is now part of OpenAI. Promptfoo remains open source and MIT licensed. Read the company update. Quick Start Requires Node.js =22.22.0 for npm and npx usage. Node.js 24 LTS is recommended; see the runtime support guide. Also available via brew install promptfoo and pip install promptfoo . You can also use npx promptfoo@latest to run any command without…
read the full promptfoo overview →
About codex-keysmith
简体中文 · English · 使用说明 · License codex keysmith 给 Codex 装上一份可撤销的指令。先看计划，确认了再写入。 简体中文 Keysmith 给本机的 AI 编程工具装指令：先预览，再写入，能验证，能撤走。 codex keysmith 面向 Codex 。装上之后，新开的对话会按这份指令工作。不改 Codex 软件本身，也不读取账号和密钥。默认只装一份稿。 [!IMPORTANT] 这会改变该 Codex 配置下 之后新开的对话 。默认只给你看计划，加上确认才会写入。装完后请开一个新任务。 使用方式 1. 先看计划。 确认之前什么都不会写入。 2. 确认后装上。 指令交给本机 Codex，软件保持原样。 3. 新开一轮对话。 关掉旧任务，开一个新的。 4. 随时撤走。 同样先看计划，确认后恢复成原来的样子。 选哪个 Keysmith 每个工具一份安装器。也有未签名桌面版（Apple Silicon / Windows x64）。 开始使用 本机需要已经装好 Codex。稳妥路径是 最新稳定 Release 的单文件脚本（当前公开稳定脚本包括 codex instruct v0.5.1.py ）。也有未签名桌面版。 源码路径： git clone && cd codex keysmith ，再把上述脚本名换成 codex instruct.py 。装完后关掉旧任务，开一个新…
read the full codex-keysmith overview →
More in AI & Machine Learning
Related comparisons
More AI Development Platforms projects
Compare either of these against the rest of the AI Development Platforms field.
Frequently asked questions
Is promptfoo or codex-keysmith more popular?
promptfoo has 26,809 GitHub stars and codex-keysmith has 4,792. promptfoo has the larger community by that measure.
Are promptfoo and codex-keysmith free?
Both are open source. promptfoo is licensed under MIT and codex-keysmith under MIT. Neither carries a licence fee.
What is the difference between promptfoo and codex-keysmith?
promptfoo is written in TypeScript and codex-keysmith in Python. The practical differences are community size, licence terms, language stack and release cadence — all compared in the table above.
Which should I choose, promptfoo or codex-keysmith?
Choose promptfoo if you want the larger community (26,809 stars) or its MIT licence terms. Choose codex-keysmith if its feature set, stack or MIT licence fits better. Both are self-hostable.