Reporium
GraphWikiTaxonomyStacksInsightsTrendsArchitectureAI-NativeFAQ
Ask anything about the repo library…
Loading repo…
←Library/LLMs-from-scratch
Library/LLMs-from-scratchForked

rasbt/LLMs-from-scratch

LLMs-from-scratch

Implement a ChatGPT-like LLM in PyTorch from scratch, step by step

View on GitHub↗Upstream rasbt/LLMs-from-scratch↗

Builder

rasbt

rasbt

rasbt • individual

Stars

96,302

Using upstream star count

Forks

14,727

Using upstream fork count

Open Issues

0

Activity Score

0/100

0 commits in 30d

Created

Jul 23, 2023

Project creation date

README Summary

This repository contains the code for developing, pretraining, and finetuning a GPT-like LLM and is the official code repository for the book [Build a Large Language Model (From Scratch)](https://amzn.to/4fqvn0D).

Community Evaluation

Loading…

AI Dev Skills

Unmapped

Attention MechanismsDeep Learning from ScratchLanguage Model ArchitectureLarge Language Model ImplementationModel TrainingNatural Language ProcessingNeural Network ProgrammingPyTorch DevelopmentTransformer Architecture

Tags

Attention MechanismsDeep Learning from ScratchLanguage Model ArchitectureLarge Language Model ImplementationModel TrainingNatural Language ProcessingNeural Network ProgrammingPyTorch DevelopmentTransformer ArchitectureAI SafetyBenchmarkingCourseDPODeep LearningDistillationDockerEmbeddingsEvalsFine-TuningForkedGPTGRPOGemmaJupyterKV CacheLarge Language ModelsLlamaLoRA / PEFTMMLUOllamaOpenAIPyTorchPythonQwenReasoning ModelsReinforcement LearningTutorial

Taxonomy

AI Trends

Large Language ModelsTransformer ArchitectureEducational AI

category

Model TrainingFoundation ModelsRAG & RetrievalEvals & BenchmarkingInference & ServingMLOps & InfrastructureLearning ResourcesSecurity & SafetyData Science & Analytics

Deployment Context

Self-hosted

Industries

EducationDeveloper Tools

Modalities

Text

Skill Areas

Transformer ArchitectureLarge Language Model ImplementationNeural Network ProgrammingDeep Learning from ScratchPyTorch DevelopmentNatural Language ProcessingAttention MechanismsModel TrainingLanguage Model Architecture

tag

AI SafetyBenchmarkingCourseDPODeep LearningDistillationDockerEmbeddingsEvalsFine-TuningForkedGPTGRPOGemmaJupyterKV CacheLarge Language ModelsLlamaLoRA / PEFTMMLUOllamaOpenAIPyTorchPythonQwenReasoning ModelsReinforcement LearningTutorial

Use Cases

Educational LLM TrainingUnderstanding Transformer InternalsCustom Language Model DevelopmentResearch PrototypingDeep Learning Education

Recent Activity

Updated 2 months ago

7 Days

0

30 Days

0

90 Days

4

harded the link checker

rasbt • Mar 7, 2026

130cc1f

Minor typo fix (#974)

Sebastian Raschka • Mar 7, 2026

9ab6e89

Bpe whitespace fixes (#975)

Sebastian Raschka • Mar 7, 2026

052c2de

Quality

research
Quality
medium
Maturity
research

Categories

RAG & RetrievalPrimaryEvals & BenchmarkingInference & ServingMLOps & InfrastructureLearning ResourcesSecurity & SafetyData Science & AnalyticsFoundation ModelsModel TrainingSafety & AlignmentOther AI / ML

PM Skills

Cost & EfficiencySafety & AlignmentScale & ReliabilityData & EvaluationProduct Discovery

Languages

Jupyter Notebook100.0%

Timeline

Project created
Jul 23, 2023
Forked
Mar 12, 2026
Your last push
2 months ago
Upstream last push
16 days ago
Tracked since
Mar 7, 2026

Similar Repos

pgvector cosine similarity · $0

Loading…