Library/mtp-lmForked

CelestialCreator/mtp-lm

mtp-lm

Source code to accompany research paper on training multi token prediction language models using self-distillation.

View on GitHub↗Upstream CelestialCreator/mtp-lm↗

Builder

CelestialCreator

CelestialCreator • individual

Stars

Using upstream star count

Forks

Using upstream fork count

Open Issues

Activity Score

0/100

0 commits in 30d

Created

Mar 5, 2026

Project creation date

README Summary

This repository contains the source code implementation for a research paper focused on training multi-token prediction language models using self-distillation techniques. The codebase provides the experimental framework and algorithms needed to reproduce the research findings on improving language model training through multi-token prediction capabilities.

AI Dev Skills

Unmapped

Multi-Token PredictionSelf-DistillationLanguage Model TrainingTransformer ArchitectureKnowledge DistillationLarge Language Model DevelopmentDeep Learning ResearchNeural Network Optimization

Taxonomy

AI Trends

Model Efficiency Knowledge Distillation Language Model Innovation Research Reproducibility

Deployment Context

Self-hosted

Modalities

Text

Skill Areas

Multi-Token Prediction Self-Distillation Language Model Training Transformer Architecture Knowledge Distillation Large Language Model Development Deep Learning Research Neural Network Optimization

Use Cases

Language Model Research Efficient Model Training Academic Research Implementation Model Compression Techniques

Recent Activity

Updated 1 months ago

7 Days

30 Days

90 Days

Quality

research

Quality: medium
Maturity: research

PM Skills

Product Discovery

Languages

Python100.0%

Timeline

Project created: Mar 5, 2026
Forked: Mar 12, 2026
Your last push: 1 months ago
Upstream last push: 1 months ago
Tracked since: Mar 5, 2026

Similar Repos

pgvector cosine similarity · $0

Loading…