| Dataset | Source Url | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| The Pile | https://the-eye.eu/public/AI/pile/ | ||||||||
| Wikipedia | https://dumps.wikimedia.org/ | ||||||||
| Common crawl | https://commoncrawl.org/ | ||||||||
| 百度百科 | https://baike.baidu.com/ | ||||||||
| 搜狗百科 | https://baike.sogou.com/ | ||||||||
| 知乎 | https://www.zhihu.com/ | ||||||||
| Quora | https://quoradata.quora.com/First-Quora-Dataset-Release-Question-Pairs | ||||||||
| 书生万卷/wanjuan-1.0 | https://github.com/opendatalab/WanJuan1.0 | ||||||||
| C4 | https://huggingface.co/datasets/allenai/c4 | ||||||||
| Wudao | https://github.com/BAAI-WuDao/Data | ||||||||
| The Stack | https://huggingface.co/datasets/bigcode/the-stack | ||||||||
| OLMO | https://github.com/allenai/OLMo | ||||||||
| project gutenberg | https://gutenberg.org/ | ||||||||
| openwebtext | https://huggingface.co/datasets/Skylion007/openwebtext |