Sign inSign inSign upSign up for freeTalk to sales

Every AI lab assumes more languages make a model worse.We bet the opposite.

Born as the general-purpose LLM underneath Lara 4 Translation AI, Lara Mini was built as the most multilingual and multicultural model from day one.
At the end of its 6T‑token training it ranks #1 of 36 models in its class worldwide.

Get the technical reportGet the model

Translated · the story behind Lara

Diversity isn't a cost to intelligence. It's a source of it.

  • The belief

    The big labs train mostly on English and treat other languages and cultures as noise that costs quality.

  • The bet

    Translated, whose job is translation, built Lara on real text written in many languages and cultures. Seeing one idea through many of them, the model understands the idea better.

  • The proof

    #1 of 36 models with up to 2B active parameters worldwide, trained on 6T tokens, 128K context.

The run at a glance

Architecture

  • 0.6Bactive parameters per token (4 experts)
  • 7.2Btotal parameters, MoE with 128 experts, 1 shared
  • 34layers: memory (GDN), window and global attention
  • 128Ktokens of context window

Data

  • 6Ttraining tokens
  • 39%of raw tokens filtered
  • 44 + ENlanguages, all 24 EU official ones
  • 151,643tokens in vocabulary, low resource rebalanced

How we built it

A hybrid, sparse design for the most multilingual model

A hybrid, sparse model: 34 layers, 128 experts

Three kinds of layer, one repeating unit

  • 14 memory · Gated DeltaNet: a linear-attention state per head, updated token by token
  • 13 window · attention over the last 512 tokens, RoPE positions
  • 7 global · full attention, no positional encoding; the last layer is always global

Mixture of experts in every layer

  • 4 of 128 routed experts chosen per token by a router
  • 1 shared expert always on
  • 0.6B of 7.2B parameters active per token; hidden size 1,024, tied input and output embeddings

Optimizer

Muon + AdamW

Each momentum update is orthogonalised (Newton–Schulz) before it is applied, so every direction of a weight matrix moves at a similar rate instead of a few dominating.

4,096-token context window in training; 151,643-token vocabulary.

The most multilingual and multi-cultural model

6T training tokens from a pool deduplicated across all sources: 39% of raw tokens were filtered out. No source is repeated more than 2 epochs (wiki and high-quality math excepted); plain web is never repeated.

  • English2.49TWeb HQ 1.34T · Synthetic HQ 0.66T · PDFs 0.32T · Web medium 0.15T · Wiki 0.02T
  • Multilingual2.18TWeb 0.79T · Web HQ 0.66T · PDFs 0.4T · Synthetic 0.28T · Wiki 0.05T
  • Code0.77TReal code 0.45T · Synthetic 0.31T
  • Math0.35TMath HQ 0.21T · Math web 0.15T
  • SFT-like0.21TQA 0.12T · Math 0.07T · Code 0.02T

Three stages, one learning-rate decay

  1. Main · 4.8T0–80% · to step 572.2k
  2. Transition · 0.6T80–90% · to step 643.7k
  3. Anneal · 0.6T90–100% · to step 715.3k
Learning-rate scheduleEvaluated model: end of training, 6T tokens.

Each stage is a separate data blend loaded at its step boundary. Transition: a bridge between the two at the stable learning rate, where the mix already moves halfway toward the anneal's (more high-quality, QA, code and math data) so the model adapts before the decay. Anneal: the highest-quality mix, while the learning rate decays to zero over the last 10%.

Each stage leans further into code, math and SFT-like data

Share of each stage's tokens. The data are shuffled, not fed domain by domain: every batch of 2,048 sequences mixes all domains in about these proportions. SFT-like (QA, math and code instruction data): 2% in main, 3.5% over the whole run.

Methodology

26 benchmarks, one harness, every model

Overall = ¼ English + ¼ Multilingual + ¼ Math + ¼ Code. Every model is scored by us with lm-evaluation-harness and the same answer-format corrections. Each multilingual benchmark is averaged over our 44 non-English training languages.

  • English10

    • NLI
    • PAWS
    • Belebele (en)
    • COPA
    • PIQA
    • HellaSwag
    • WinoGrande
    • ARC-C
    • MMLU
    • MMLU-Pro

  • Multilingual9

    • XNLI
    • PAWS-X
    • Belebele
    • XCOPA
    • Global PIQA NP
    • Global PIQA P
    • T-HellaSwag
    • INCLUDE
    • Global MMLU

  • Math3

    • GSM8K
    • MATH
    • MGSM

  • Code4

    • MBPP
    • MBPP+
    • CRUX-I
    • CRUX-O

No peeking: the final training stage skips every document that holds a benchmark item.

Results

#1 of 36 in tiny models worldwide

First on Overall, Text and Multilingual, 3.1 points ahead of granite-4.0-h-tiny on Overall.

Overall

Tiny models ranking

#1

out of 36 models

Category ranking

  • Math & Code#1
  • Text#1
  • Multilingual#1

Benchmark definition. Our model is the final checkpoint of the 6T‑token run, with its context extended to 128K; its anneal skips every document holding a benchmark item. Its class: models with up to 2B active parameters (gemma-4-E2B at its 2B effective size). Multilingual scores are averaged over the 44 non-English languages we train on, for every model.

Rankings

Top 12 tiny models

36 models with up to 2B active parameters (gemma-4-E2B at its 2B effective size, 5.1B total). Multilingual on our training languages. Bars start from 0.

  1. #1Lara Mini Base (A0.6B/7.2B)57.0
  2. #2IBM - granite-4.0-h-tiny (1B active)53.9
  3. #3Alibaba - Qwen3-1.7B53.8
  4. #4Falcon-H1-1.5B-Deep53.2
  5. #5Hunyuan-1.8B52.2
  6. #6LFM2.5-8B-A1B (1.7B active)50.8
  7. #7Trinity-Nano (1B active)50.7
  8. #8LFM3-1.7B49.2
  9. #9Qwen3.5-2B48.9
  10. #10Falcon-H1-1.5B48.8
  11. #11Qwen2.5-1.5B47.6
  12. #12granite-4.0-1b47.2
  13. #17Google - gemma-4-E2B41.9

Overall = ¼ English + ¼ Multilingual + ¼ Math + ¼ Code.

Efficiency

#1 under 2B, with just 0.6B active

Lara Mini Base outscores all 35 other models of its class, including models that use up to 3× more active parameters, such as Qwen3-1.7B and gemma-4-E2B.

20304050600.5B1B2BLara Mini Base (A0.6B/7.2B) · 57.0granite-4.0-h-tiny · 53.9gemma-4-E2B
Overall score against active parameters, log scale. The step line is the frontier: no model with fewer active parameters scores higher. gemma-4-E2B is drawn at its 2B effective size (5.1B total).

Limited release · who gets it

What Lara Mini is for

  • Lara Translate app

    Offline Translation

    Lara Mini is the base model behind Lara 4 offline translation feature for the next Lara Translate app. Your text never leaves the phone, and each translation still takes under a second.

  • Translated customers

    Tiny Base LLM

    Selected Translated enterprise customers will have access. Lara Mini is designed to be fine-tuned for specialized sovereign applications, runs on mobile devices and on our cloud delivers an end-to-end latency under 20 ms delivering millions of tokens per sec.

  • Research

    AI researchers

    Access for AI researchers to reproduce our results and build on them, pushing multilingual research further.

What's next

Three competitions, one rule

We train the next stage only if we win a medal in the current one.

  1. Stage 1

    Local

    Tiny · <2B active

    Nov 2026Lara Mini Base

    Win the tiny category with Lara Base (A0.6B/7.2B).

    Dec 2026Lara Mini

    Lara 4 Mini and Lara Mini LLM, for ultra-low latency and mobile devices.

    Medal won → we train the National stage. No medal, we stop.

  2. Stage 2

    National

    Small · <10B active

    Mar 2027Lara Piccola Base (A7B/138B)

    Enter the small category and win a medal there too.

    Apr 2027Lara 4 & Lara Piccola LLM

    Lara 4 translation model and Lara Piccola LLM, an EU-sovereign A7B/138B model.

    Medal won → we train the Olympics stage. No medal, we stop.

  3. Stage 3

    Olympics

    Frontier · 40B+ active

    Dec 2027Lara Grande Base (A60B/1.2T)

    Enter the frontier category, against the world's best models.

    Mar 2028Lara 5 & Lara Grande LLM

    Lara 5 surpasses P95 pro translators. Lara Grande: an EU-sovereign frontier LLM, ahead of Chinese models and approaching Anthropic and OpenAI.

Reference models: tiny Gemma 4 E2B · small Gemma 4B/26B · medium Qwen 17B/397B · frontier GPT-6 Astra, Claude 5.6, Kimi K3.

Training budget

Training single-run budgets, next releases

ModelTokensFLOPsHardwareDaysTraining cost
Lara 4 PiccolaMoE 6.7B / 138B20T8.0 × 10²³512 × H10031$1.0M
Lara 4 GrandeMoE 60B / 1.2T24T8.6 × 10²⁴512 × Vera Rubin25$2.4M
Lara 4 SaggiaMoE 600B / 12T36T1.3 × 10²⁶2,304 × Vera Rubin83$36.6M
For reference
Typical frontierMoE 150B / 3T36T3.2 × 10²⁵4,096 × B20051$18.8M

Source: Lara 4 LLM Training Budget Planning, 3 Nov 2026 · FP8 · compute cost only · Saggia days estimated with the plan's method

Next training

Lara Piccola Candidate

60 layers and 138B parameters: the model to beat Gemma 4 31B.

  • 138Btotal parameters
  • 6.7Bactive per token
  • Tokenizernew, rebalanced
  • 60layers
  • 2816hidden size
  • 512 × 512experts, top-8 + 2 shared
  • 400kH100 GPU-hours for 20T tokens
  • 30 dayson 512 H100 GPUs
  • $1MTraining run cost

  • Goal: win the small model class (<10B active)

    Current leaders: Qwen3.5-122B-A10B, GLM-4.5-Air

  • Goal: beat Gemma 4 31B on multilingual

    Beat highly multilingual dense models in the medium model class.

30.7 s per step at 32 nodes · same hybrid attention as Lara Mini Base (GDN, window, global)

Technical report

Get the technical report

Architecture, training data, benchmarks and methodology behind Lara Mini Base, in one document. Leave your details and we'll email it to you.

I consent to be contacted by Lara for further information and demo. I have read and agree to the Privacy Policy*

Work with Lara

Two ways to put Lara to work

  • Interested in the generative model?

    Selected Translated enterprise customers will have access. Lara Mini is designed to be fine-tuned for specialized sovereign applications, runs on mobile devices and on our cloud delivers an end-to-end latency under 20 ms delivering millions of tokens per sec.

    Contact Sales

  • Translate with Lara

    Translate text, documents and conversations with Lara, the translator built on this research.

    Start using Lara for Translation