Instructions to use MEYNG/nllb-sango-finetuned-600m-3ep with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use MEYNG/nllb-sango-finetuned-600m-3ep with PEFT:
from peft import PeftModel from transformers import AutoModelForSeq2SeqLM base_model = AutoModelForSeq2SeqLM.from_pretrained("facebook/nllb-200-distilled-600M") model = PeftModel.from_pretrained(base_model, "MEYNG/nllb-sango-finetuned-600m-3ep") - Transformers
How to use MEYNG/nllb-sango-finetuned-600m-3ep with Transformers:
# Use a pipeline as a high-level helper # Warning: Pipeline type "translation" is no longer supported in transformers v5. # You must load the model directly (see below) or downgrade to v4.x with: # pip install "transformers<5.0.0" from transformers import pipeline pipe = pipeline("translation", model="MEYNG/nllb-sango-finetuned-600m-3ep")# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("MEYNG/nllb-sango-finetuned-600m-3ep", device_map="auto") - Notebooks
- Google Colab
- Kaggle
nllb-sango-finetuned-600m-3ep β LoRA adapter from the v1 training run
This is an adapter, not a model, and it is superseded. For actual use, take
MEYNG/nllb-sango-finetuned-600m-v2-2(SangoβFrench) orfacebook/nllb-200-distilled-600Mdirectly (FrenchβSango).
This repository contains only adapter_config.json and adapter_model.safetensors. There is
no config.json and no model.safetensors, so AutoModelForSeq2SeqLM.from_pretrained will
not work on it. It must be loaded with peft on top of the base model.
What this is
The LoRA adapter produced by MEYNG's first Sango fine-tuning run, trained for 3 epochs
(hence 3ep), 26,187 steps.
It is a distinct run from MEYNG/nllb-sango-finetuned-600m (v1).
An earlier version of this card said the two were the same training run. That was wrong, and the
correction is recorded here rather than quietly removed:
| steps | source | |
|---|---|---|
| this adapter | 26,187 | the Trainer log reproduced below |
| v1 | 25,395 | last-checkpoint/trainer_state.json in v1's own repository |
Different step counts, so different runs. The earlier claim rested on both repositories carrying
LoRA configs with identical hyper-parameters (r=16, lora_alpha=32, lora_dropout=0.1,
target_modules=[q_proj, v_proj]) β but matching hyper-parameters are what you would expect of
two runs from the same project, and they are not evidence of one run.
On v1's LoRA files. v1's repository also contains adapter_config.json and
adapter_model.safetensors. Its own documentation, and its training_args.bin
(gradient_checkpointing enabled, adamw_torch, batch size 12 β the signature of fitting a
full-parameter fine-tune onto a small GPU, which LoRA at this scale does not require), both
describe v1 as a full fine-tune. The adapter files there are most likely strays. Prefer v1's own
card over any inference from this one.
Use v1 if you want loadable weights; this repo is the adapter alone.
Why it is superseded
This adapter comes from MEYNG's first Sango fine-tuning phase (May 2026), the phase whose published model β v1 β improved in-distribution scores but regressed below its own untrained baseline on out-of-distribution text (FLORES-200). That result is published in full on the v1 model card and was the reason MEYNG introduced an out-of-distribution gate: OOD chrF must meet or beat the baseline, or the model does not ship. Every later release passes that gate; this run predates it.
No benchmark has been run against this repository directly, and none of MEYNG's published
numbers describe it. BENCH-001 and BENCH-002 evaluate MEYNG/nllb-sango-finetuned-600m β a
different run (see the step counts above). Do not attribute those scores to this adapter. The only
measured figures that belong to it are its own training curve, below.
If you want a benchmarked artifact, use
MEYNG/nllb-sango-finetuned-600m-v2-2,
which passes the OOD gate in its recommended direction.
The only measured figure that belongs to this repository is its own training curve, preserved below exactly as the Trainer recorded it.
Usage
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
from peft import PeftModel
base = "facebook/nllb-200-distilled-600M"
tok = AutoTokenizer.from_pretrained(base)
model = AutoModelForSeq2SeqLM.from_pretrained(base)
model = PeftModel.from_pretrained(model, "MEYNG/nllb-sango-finetuned-600m-3ep")
tok.src_lang = "sag_Latn"
enc = tok("Mbi yeke gue na da ti nganga ti kobela.", return_tensors="pt")
out = model.generate(**enc, forced_bos_token_id=tok.convert_tokens_to_ids("fra_Latn"),
max_length=160, num_beams=4)
print(tok.batch_decode(out, skip_special_tokens=True)[0])
Training procedure
Training data
The NLLB web-mined parallel corpus for this language pair. MEYNG's later forensic pass on that corpus found it to be 78% duplicate Sango targets, with verse numbers, speaker labels and scripture references throughout β artifacts this run learned, and the documented cause of the out-of-distribution regression described above. Later runs train on a cleaned 45,451-pair subset instead. See the v2-2 card for that analysis.
Hyperparameters
- learning_rate: 0.0003
- train_batch_size: 16 Β· eval_batch_size: 16
- seed: 42
- gradient_accumulation_steps: 8 Β· total_train_batch_size: 128
- optimizer:
adamw_torch_fused, betas=(0.9,0.999), epsilon=1e-08 - lr_scheduler_type: linear Β· warmup_steps: 500
- num_epochs: 3
- mixed_precision_training: Native AMP
- LoRA: r=16, Ξ±=32, dropout=0.1, target_modules=[
q_proj,v_proj]
Training results
| Training Loss | Epoch | Step | Validation Loss |
|---|---|---|---|
| 15.0623 | 0.1146 | 1000 | 1.7556 |
| 14.4898 | 0.2291 | 2000 | 1.6745 |
| 14.0211 | 0.3437 | 3000 | 1.6137 |
| 13.6606 | 0.4583 | 4000 | 1.5683 |
| 13.4614 | 0.5728 | 5000 | 1.5280 |
| 13.1058 | 0.6874 | 6000 | 1.4951 |
| 13.0098 | 0.8019 | 7000 | 1.4680 |
| 12.8789 | 0.9165 | 8000 | 1.4467 |
| 12.4444 | 1.0310 | 9000 | 1.4260 |
| 12.6331 | 1.1456 | 10000 | 1.4081 |
| 12.3781 | 1.2602 | 11000 | 1.3935 |
| 12.1731 | 1.3747 | 12000 | 1.3782 |
| 12.2067 | 1.4893 | 13000 | 1.3670 |
| 12.0454 | 1.6039 | 14000 | 1.3578 |
| 11.9698 | 1.7184 | 15000 | 1.3479 |
| 11.9595 | 1.8330 | 16000 | 1.3398 |
| 11.8094 | 1.9476 | 17000 | 1.3327 |
| 11.6344 | 2.0621 | 18000 | 1.3272 |
| 11.7943 | 2.1767 | 19000 | 1.3200 |
| 11.7281 | 2.2912 | 20000 | 1.3155 |
| 11.8180 | 2.4058 | 21000 | 1.3110 |
| 11.5384 | 2.5203 | 22000 | 1.3079 |
| 11.7371 | 2.6349 | 23000 | 1.3051 |
| 11.6320 | 2.7495 | 24000 | 1.3031 |
| 11.7702 | 2.8640 | 25000 | 1.3018 |
| 11.6175 | 2.9786 | 26000 | 1.3012 |
| 11.6356 | 3.0 | 26187 | 1.3012 |
Final validation loss 1.3012. Note that validation loss here is measured on held-out data from the same distribution as training β it is exactly the metric that looked healthy while out-of-distribution quality regressed. It is reported because it was measured, not because it is evidence of quality.
Framework versions
- PEFT 0.19.1 Β· Transformers 5.8.1 Β· Pytorch 2.12.0+cu126 Β· Datasets 4.8.5 Β· Tokenizers 0.22.2
β οΈ Unverified caveat. MEYNG's later work documents that Transformers v5 breaks the NLLB tokenizer language prefix (v2-2 onward pin
transformers>=4.40,<5.0). This run recorded Transformers 5.8.1 and predates that finding; its per-direction language-token conditioning has not been re-verified. Treat outputs from this adapter with that in mind.
Limitations
- Adapter only β not loadable as a standalone model (see above).
- Superseded β v1, its merged form, regresses out-of-distribution; v2-2 exists and passes the OOD gate.
- Never benchmarked directly β no BENCH-001/BENCH-002 numbers belong to this repo.
- Training corpus is heavily duplicated religious/NGO-register web text (see Training data).
- Possible tokenizer-prefix issue from the framework version (see caveat).
Licence
CC BY-NC 4.0 β non-commercial use only.
Inherited, not chosen: facebook/nllb-200-distilled-600M is released by Meta under CC BY-NC 4.0,
and this adapter is a derivative work of it, so the NonCommercial term carries through. It applies
to the adapter weights and to any merge of them into the base model.
Permitted: research, academic work, teaching, humanitarian and NGO deployment, public-sector and personal use. Not permitted: paid APIs or subscriptions, licensing to commercial entities, bundling into paid products, advertising-supported services.
Under Creative Commons' guidance the NonCommercial term restricts the use, not the user.
Commercial use requires separate permission from Meta (for the base model) and from MEYNG (for the fine-tune). MEYNG cannot grant the first.
Attribution
Derivative of NLLB-200 by Meta AI (facebook/nllb-200-distilled-600M, CC BY-NC 4.0).
Redistribution or adaptation must credit both the base model and this fine-tune.
NLLB Team et al., No Language Left Behind: Scaling Human-Centered Machine Translation, 2022. arXiv:2207.04672
About MEYNG
MEYNG builds language AI infrastructure for under-resourced African languages, starting with Sango (Central African Republic, 5M+ speakers). Out-of-distribution gates before every release, failures documented and left public β this adapter is from the run that taught us to build the gate. https://huggingface.co/MEYNG Β· https://meyng.com
- Downloads last month
- 7
Model tree for MEYNG/nllb-sango-finetuned-600m-3ep
Base model
facebook/nllb-200-distilled-600M