Promptfoo

Promptfoo is an open-source tool for testing and evaluating LLM outputs. Define test cases with expected outputs, run them across multiple models and prompts, and get a comparison matrix showing which configuration performs best. It supports custom evaluation functions, red-teaming for safety, and CI/CD integration for automated prompt regression testing.