head to head · open source
promptfoo vs codex-keysmith
promptfoo has 25,297 GitHub stars, 2,344 forks, 651 open issues and last shipped today. codex-keysmith has 4,544 stars, 714 forks, 5 open issues and last shipped 2 days ago. promptfoo leads on adoption by 457% (25,297 vs 4,544 stars). promptfoo is written in TypeScript under MIT; codex-keysmith is written in Python under MIT. promptfoo has attracted 9% as many forks as stars, codex-keysmith 16%. promptfoo was the more recently maintained of the two, and both are self-hostable with no licence fee. The two share 1 topic tag (prompt-engineering), so they are genuine substitutes rather than adjacent tools.
Two open source projects, one decision. Both are free and self-hostable — the differences are community size, license terms, language stack and release pace.
← all 20902 open source comparisons
Side by side
| promptfoo | codex-keysmith | |
|---|---|---|
| GitHub stars | ★ 25K | ★ 4.5K |
| License | MIT | MIT |
| Written in | TypeScript | Python |
| Last push | 2026-09-20 | 2026-09-18 |
| Forks | ⑂ 2.3K | ⑂ 714 |
| Self-hosting | Yes | Yes |
| Data ownership | Your server | Your server |
pick promptfoo if
- You weight community size — 25K stars and counting
- You want the MIT license terms
- Your stack matches TypeScript
- You value the larger contributor base for long-term maintenance
pick codex-keysmith if
- You want the codex-keysmith feature set and don't need the biggest community
- You prefer the MIT license terms
- Your stack matches Python
- You evaluated both and codex-keysmith fits your workflow better
About promptfoo
Promptfoo is an open source command line tool and TypeScript library for evaluating and red teaming large language model applications, aimed at developers, AI engineers, and security teams who need measurable evidence that a prompt, agent, or retrieval augmented generation pipeline behaves correctly and safely before it reaches users.
read the full promptfoo overview →
About codex-keysmith
codex keysmith is a local first Python command line tool that installs, previews, versions, and removes instruction files for Codex, built for developers who want their Codex sessions to follow a controlled, reversible prompt configuration.
read the full codex-keysmith overview →
More in AI & Machine Learning
Related comparisons
More AI Development Platforms projects
Compare either of these against the rest of the AI Development Platforms field.
Frequently asked questions
Is promptfoo or codex-keysmith more popular?
promptfoo has 25,297 GitHub stars and codex-keysmith has 4,544. promptfoo has the larger community by that measure.
Are promptfoo and codex-keysmith free?
Both are open source. promptfoo is licensed under MIT and codex-keysmith under MIT. Neither carries a licence fee.
What is the difference between promptfoo and codex-keysmith?
promptfoo is written in TypeScript and codex-keysmith in Python. The practical differences are community size, licence terms, language stack and release cadence — all compared in the table above.
Which should I choose, promptfoo or codex-keysmith?
Choose promptfoo if you want the larger community (25,297 stars) or its MIT licence terms. Choose codex-keysmith if its feature set, stack or MIT licence fits better. Both are self-hostable.