Meltemi

Meltemi is the first open Greek large language model, developed by the Institute for Language and Speech Processing (ILSP) at the Athena Research Center. It starts from Mistral-7B and extends it through continual pretraining on roughly 40 billion tokens, of which about 28.5 billion are high-quality Greek texts, plus English and Greek-English parallel data to preserve bilingual ability. A Greek vocabulary extension to the tokenizer cuts Greek token fertility substantially.

Meltemi-7B-v1 and an instruction-tuned chat variant were released under Apache 2.0 with 8k context, showing about a 15% average improvement over Mistral-7B on ILSP's Greek evaluation suite. A v1.5 generation later expanded the Greek corpus to over 43 billion tokens. The paper is published as arXiv 2407.20743.