Norwegian version of this page

NORA.LLM

The collection of Large Language Models for the Norwegian language, NorLM (now more commonly known as NORA.LLM), originated as a collaborative initiative involving several notable projects. These include EOSC-Nordic (European Open Science Cloud), SANT (Sentiment Analysis for Norwegian), and HPLT (High-Performance Language Technologies). The initiative was developed in partnership with the AI Laboratory of the National Library of Norway and the National e-Infrastructure Services, under the coordination of the Language Technology Group (LTG) at the University of Oslo.

Models

The NorLM initiative has produced several notable models tailored for various NLP applications in the Norwegian language. The team behind NorLM aims to provide these models and complementary tools for researchers and developers in the field of Natural Language Processing (NLP) focused on the Norwegian language. Their objective is to facilitate scientific experimentation and enable practical applications using cutting-edge NLP architectures. Moreover, they aspire to empower others to build their own large-scale models for various tasks, whether domain- or application-specific, or even for different language variants or other languages altogether.

These models include:

NorMistral-7b-warm

  • Description: NorMistral-7b-warm is a large Norwegian language model initialized from Mistral-7b-v0.1 and continuously pretrained on a total of 260 billion subword tokens (using six repetitions of open Norwegian texts). Read more here...
  • Model Developers: Language Technology Group at the University of Oslo.
  • Model Architecture: NorMistral is an auto-regressive language model that uses an optimized transformer architecture based on the Mistral/Llama language models.
  • License: apache-2.0
  • Code Repository: GitHub

NorMistral-7b-warm-instruct

  • Description: NorMistral-7b-warm-instruct is a large Norwegian language model initialized from Mistral-7b-v0.1 and continuously pretrained on a total of 260 billion subword tokens (using six repetitions of open Norwegian texts). Read more here...
  • Model Developers: Language Technology Group at the University of Oslo.
  • Model Architecture: NorMistral is an auto-regressive language model that uses an optimized transformer architecture based on the Mistral/Llama language models.
  • License: apache-2.0
  • Code Repository: GitHub

NorMistral-7b-scratch

  • Description: NorMistral-7b-scratch is a large Norwegian language model pretrained from scratch on a total of 260 billion subword tokens (using six repetitions of open Norwegian texts). Read more here...
  • Model Developers: Language Technology Group at the University of Oslo.
  • Model Architecture: NorMistral is an auto-regressive language model that uses an optimized transformer architecture based on the Mistral/Llama language models.
  • License: apache-2.0
  • Code Repository: GitHub

NorBLOOM-7b-scratch

  • Description: NorBLOOM-7b-scratch is a large Norwegian language model pretrained from scratch on a total of 260 billion subword tokens (using six repetitions of open Norwegian texts). Read more here...
  • Model Developers: Language Technology Group at the University of Oslo.
  • Model Architecture: NorBLOOM is an auto-regressive language model that uses the BLOOM architecture.
  • License: apache-2.0
  • Code Repository: GitHub

Other Resources

- CuttingEdgeAI - NORA.LLM: Open Norwegian Language Models as a Common Good (12 April 2024)

Competence Group

Coming soon.

Published Sep. 24, 2024 7:57 AM - Last modified Oct. 9, 2025 1:34 PM
E-mail this page