Library/ColossalAI
Library/ColossalAIForked

hpcaitech/ColossalAI

ColossalAI

Making large AI models cheaper, faster and more accessible

Builder

hpcaitech

hpcaitech

hpcaitech • individual

Stars

41,372

Using upstream star count

Forks

4,522

Using upstream fork count

Open Issues

0

Activity Score

0/100

0 commits in 30d

Created

Oct 28, 2021

Project creation date

README Summary

ColossalAI is an integrated large-scale model training system that provides a collection of parallel components for distributed deep learning. It aims to make large AI models more accessible by offering efficient parallelization strategies including data, pipeline, tensor, and sequence parallelism. The system supports training models with billions of parameters while reducing memory costs and accelerating training speed.

AI Dev Skills

Unmapped

Distributed Deep LearningModel ParallelismData ParallelismPipeline ParallelismMemory OptimizationLarge Language Model TrainingTransformer ArchitectureMixed Precision TrainingGradient CompressionModel ShardingZero Redundancy OptimizationTensor ParallelismMulti-GPU TrainingCluster ComputingCUDA ProgrammingModel Fine-tuningInference Optimization

Tags

Distributed Deep LearningModel ParallelismData ParallelismPipeline ParallelismMemory OptimizationLarge Language Model TrainingTransformer ArchitectureMixed Precision TrainingGradient CompressionModel ShardingZero Redundancy OptimizationTensor ParallelismMulti-GPU TrainingCluster ComputingCUDA ProgrammingModel Fine-tuningInference OptimizationCost-Effective AI TrainingLarge-Scale Neural Network OptimizationDistributed AI SystemsDistributed AI TrainingOn-premise HPCComputer Vision Model ScalingMulti-GPU ClustersLarge Language ModelsAI Model ParallelizationModel OptimizationHigh Performance ComputingTextMemory-Efficient Model TrainingMultimodalScalable AI InfrastructureMulti-GPU Model InferenceImageDistributed SystemsGradient SynchronizationEfficient AI TrainingCloud ComputingCommunication OptimizationGPU ComputingPython

Taxonomy

Recent Activity

Updated 28 days ago

7 Days

0

30 Days

0

90 Days

0

Quality

production
Quality
high
Maturity
production

Categories

MLOps & InfrastructurePrimaryDev Tools & AutomationML Platform & InfrastructureMultimodal AIOther AI / MLInference & ServingFoundation ModelsModel TrainingComputer Vision

PM Skills

Scale & ReliabilityDeveloper Platform

Languages

Python100.0%

Timeline

Project created
Oct 28, 2021
Forked
Mar 22, 2026
Your last push
28 days ago
Upstream last push
14 days ago
Tracked since
Mar 16, 2026

Similar Repos

pgvector cosine similarity · $0

Loading…