Data Curation

The Pile

• EleutherAI
• ThePile
• The Pile Dataset
• The Pile Dataset

CommonCrawl

• CommonCrawl
• CommonCrawl Dataset
• CommonCrawl

Colossal Clean Common Crawl

• C4
• C4 Dataset
• C4

Dolma

• Dolma Dataset
• Dolma
• Dolma

RefinedWeb

• RefinedWeb Dataset
• RefinedWeb

OpenWebText2

• OpenWebText2
• OpenWebText2 Dataset
• OpenWebText2
• OpenWebText2

RedPajama

• RedPajama Dataset
• Open Dataset

TheStack

• TheStack Dataset

Laion-5B

• Laion-5B Dataset

Massive

• Massive