bigcode-project/bigcode-evaluation-harness
bigcode-evaluation-harness
A framework for the evaluation of autoregressive code generation language models.
Builder
bigcode-project
bigcode-project • individual
Stars
1,047
Using upstream star count
Forks
263
Using upstream fork count
Open Issues
0
Activity Score
0/100
0 commits in 30d
Created
Aug 9, 2022
Project creation date
README Summary
<h1 align="center">Code Generation LM Evaluation Harness</h1>
Community Evaluation
Loading…AI Dev Skills
Unmapped
Autoregressive Language ModelingCode CompletionCode Generation EvaluationFew-shot LearningLarge Language Model EvaluationModel BenchmarkingNatural Language to Code TranslationProgram Synthesis
Tags
Autoregressive Language ModelingCode CompletionCode Generation EvaluationFew-shot LearningLarge Language Model EvaluationModel BenchmarkingNatural Language to Code TranslationProgram SynthesisBenchmarkingC++DockerEvalsForkedGPU / CUDAHuggingFaceHumanEvalJavaScriptLM Eval HarnessOpenAIPyTorchPythonResearch / PapersSecurityTensorFlowTransformers
Taxonomy
category
Deployment Context
Skill Areas
tag
Recent Activity
Updated 10 months ago
7 Days
0
30 Days
0
90 Days
0
Quality
research- Quality
- high
- Maturity
- research
Categories
Evals & BenchmarkingPrimaryInference & ServingMLOps & InfrastructureDev Tools & AutomationLearning ResourcesSecurity & SafetyFoundation ModelsModel TrainingSearch & KnowledgeOther AI / ML
PM Skills
Scale & ReliabilityData & Evaluation
Languages
Python100.0%
Timeline
- Project created
- Aug 9, 2022
- Forked
- Mar 22, 2026
- Your last push
- 10 months ago
- Upstream last push
- 10 months ago
- Tracked since
- Jul 22, 2025
Similar Repos
pgvector cosine similarity · $0
Loading…