Library/promptfoo
Library/promptfooForked

promptfoo/promptfoo

promptfoo

Test your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT, Claude, Gemini, Llama, and more. Simple declarative configs with command line and CI/CD integration.

Builder

promptfoo

promptfoo

promptfoo • individual

Stars

19,201

Using upstream star count

Forks

1,645

Using upstream fork count

Open Issues

0

Activity Score

0/100

0 commits in 30d

Created

Apr 28, 2023

Project creation date

README Summary

Promptfoo is a comprehensive testing and evaluation framework for AI prompts, agents, and RAG systems. It provides red teaming, pentesting, and vulnerability scanning capabilities for AI applications. The tool supports performance comparison across multiple LLM providers like GPT, Claude, Gemini, and Llama with declarative configuration and CI/CD integration.

AI Dev Skills

Unmapped

Prompt EngineeringRetrieval-Augmented GenerationAI Red TeamingAI Security TestingLanguage Model EvaluationAI Vulnerability AssessmentMulti-model ComparisonAI System Testing

Tags

Prompt EngineeringRetrieval-Augmented GenerationAI Red TeamingAI Security TestingLanguage Model EvaluationAI Vulnerability AssessmentMulti-model ComparisonAI System TestingCommand LineMulti-LLM Performance ComparisonCompound AI SystemsDeveloper ToolsAgentic AICI/CD PipelineAI SafetyRAG System EvaluationAI Security AssessmentSelf-hostedAI Agent TestingAI Penetration TestingRed TeamingAI Prompt TestingTextAI EvaluationPrompt OptimizationLanguage Model BenchmarkingTypeScript

Taxonomy

Recent Activity

Updated 1 months ago

7 Days

0

30 Days

0

90 Days

0

Quality

production
Quality
high
Maturity
production

Categories

ML Platform & InfrastructurePrimarySafety & AlignmentOther AI / MLMLOps & InfrastructureDev Tools & AutomationRAG & RetrievalEvals & BenchmarkingFoundation ModelsAI Agents

PM Skills

Developer Platform

Languages

TypeScript100.0%

Timeline

Project created
Apr 28, 2023
Forked
Mar 13, 2026
Your last push
1 months ago
Upstream last push
6 days ago
Tracked since
Mar 13, 2026

Similar Repos

pgvector cosine similarity · $0

Loading…