Note The MaLA monolingual corpus's noisy version that integrates texts from different sources without cleaning.

Note The MaLA monolingual corpus's filtered version that performs further data filtering

Note The MaLA monolingual corpus's deduplicated version that removes repeated data points

Note The MaLA monolingual corpus's final version is processed by splitting the filtered and deduplicated version into training and test sets

Note The MaLA bilingual translation corpus contains parallel data in more than 2,500 language pairs (500+ languages).

Note The first version of the MaLA code and reasoning dataset used for training https://huggingface.co/MaLA-LM/emma-500-llama2-7b

Note The 2nd version of the MaLA code and reasoning dataset used for training EMMA-500 Llama 3(.1) Mono/Bi model series.

Note A sampled set of MaLA-LM/mala-opus-dedup-2410