huggingface/text-generation-inference
text-generation-inference
Large Language Model Text Generation Inference
Builder
HuggingFace
huggingface • ai-lab
Stars
10,818
Using upstream star count
Forks
1,260
Using upstream fork count
Open Issues
0
Activity Score
0/100
0 commits in 30d
Created
Oct 8, 2022
Project creation date
README Summary
Text Generation Inference (TGI) is a toolkit for deploying and serving Large Language Models (LLMs) with high performance and scalability. It provides optimized inference capabilities with features like tensor parallelism, dynamic batching, and support for popular model architectures like Llama, Falcon, and StarCoder. The toolkit is designed for production environments and offers both Docker deployment and native installation options.
AI Dev Skills
Unmapped
Tags
Taxonomy
Deployment Context
Modalities
Skill Areas
Recent Activity
Updated 3 months ago
7 Days
0
30 Days
0
90 Days
0
Quality
production- Quality
- medium
- Maturity
- production
Categories
PM Skills
Languages
Timeline
- Project created
- Oct 8, 2022
- Forked
- Mar 13, 2026
- Your last push
- 3 months ago
- Upstream last push
- 23 days ago
- Tracked since
- Jan 8, 2026
Similar Repos
pgvector cosine similarity · $0
Loading…