lmdeploy

LMDeploy is a toolkit for compressing, deploying, and serving LLMs.

View on GitHub↗Upstream InternLM/lmdeploy↗

Builder

InternLM

InternLM • individual

Stars

7,747

Using upstream star count

Forks

678

Using upstream fork count

Open Issues

Activity Score

0/100

0 commits in 30d

Created

Jun 15, 2023

Project creation date

README Summary

LMDeploy is a comprehensive toolkit designed for compressing, deploying, and serving Large Language Models (LLMs). It provides efficient solutions for model optimization and deployment in production environments. The toolkit focuses on making LLM deployment more accessible and performant for various use cases.

AI Dev Skills

Unmapped

Tags

Large Language Model DeploymentModel Compression and QuantizationHigh-Performance Inference ServingModel OptimizationProduction ML SystemsGPU AccelerationDistributed Model ServingTransformer Model OptimizationHigh-throughput Text GenerationProduction AI SystemsLarge-scale Language Model DeploymentEfficient AI InferenceModel Compression and OptimizationModel Inference OptimizationEdge AI DeploymentOn-premiseModel QuantizationLLM Performance TuningResource-constrained LLM ServingCloud APIModel Serving InfrastructureSelf-hostedProduction LLM API ServingLarge Language ModelsEnterprise LLM InfrastructureEdge/MobileTextGPU Memory ManagementInference OptimizationPython

Recent Activity

Updated 23 days ago

7 Days

30 Days

90 Days

Quality

production

Quality: high
Maturity: production

PM Skills

Developer Platform

Languages

Python100.0%

Timeline

Project created: Jun 15, 2023
Forked: Mar 22, 2026
Your last push: 23 days ago
Upstream last push: 6 days ago
Tracked since: Mar 22, 2026

Similar Repos

pgvector cosine similarity · $0

Loading…