guigssss/steno-fr-v1-base

Sténo v1 base — fast French speech recognition on EmbeddingGemma 2 (CTC)

Sténo turns the audio tower of Google's embedding model EmbeddingGemma 2 into a French speech-to-text model. A trained CTC reader reads the tower's hidden states and writes French word pieces, with capitals and punctuation, in a single non-autoregressive pass, many times faster than real time.

En français. Sténo transcrit le français à partir de l'« oreille » d'EmbeddingGemma 2, le modèle d'embedding de Google. Un lecteur CTC de 6 couches, entraîné sur environ 3 200 heures de français ouvert, écrit le texte avec majuscules et ponctuation, en une seule passe. Version base : la plus simple et la plus rapide. 10,0 % de mots faux sur FLEURS avec le petit modèle de langue fourni. Français seulement.

Two versions: base and large

	base	large
Reader	6 blocks of width 768	10 blocks of width 768
Audio tower fine-tuned	layers 11–12	layers 10–12
Parameters in model.safetensors	110 M (438 MB)	163 M (652 MB)
FLEURS test, greedy / + LM	13.2 / 10.0	12.8 / 9.7
Training, on one NVIDIA L40S	about 38 GPU hours	base + about 17 GPU hours

base is the model trained in one line: the same architecture from the first step to the last, with the data growing as it was labelled. large starts from base and trains six more passes over the same data, first with one more tower layer fine-tuned (layer 10), then with the reader grown from 6 to 10 blocks. It is better or equal on every test set, by a small margin (0.2 to 0.5 point), and a little slower (four more reader blocks).

large is an experiment. We did not train base for the same six extra passes without changing anything: part of the gain, perhaps all of it, may come from the longer training (about 17 more GPU hours) rather than from the larger reader and the extra tower layer. When in doubt, base is the simpler choice; large is the more accurate one.

This repository: base. The other one: large.

Results

Word error rate (%), greedy / with the 4-gram LM (lm/), on the first 2 000 clips of each test set (FLEURS: all 675), after a common normalisation (lower case, no punctuation, numbers as words).

Test set	What it is	Greedy	+ 4-gram LM
FLEURS fr test	read sentences	13.2	10.0
Multilingual LibriSpeech fr test	audiobooks	10.5	9.0
Common Voice 19 fr test	volunteers, many microphones	14.4	11.8
VoxPopuli fr test	European Parliament	12.7	11.3
African Accented French test	West and Central African accents	16.2	15.2
FLEURS fr dev (validation)	read sentences	11.8	9.5

Use

With the steno_asr package (needs ffmpeg on the PATH; accept the EmbeddingGemma 2 terms and run hf auth login once):

pip install "git+https://github.com/GuigsEvt/steno-asr"

from steno_asr import Steno, to_srt

asr = Steno.from_pretrained("base")                       # or "large"; same as "guigssss/steno-fr-v1-base"
print(asr.transcribe("interview.mp3"))                     # one string
out = asr.transcribe("talk.mp4", return_words=True)        # {"text", "words": [{"word", "start", "end", "conf"}]}
open("talk.srt", "w").write(to_srt(out["words"]))

asr = Steno.from_pretrained("base", lm=True)              # + the 4-gram in lm/ (pip install "steno-asr[lm] @ ...")

steno interview.mp3 --model base --srt subs/

The audio is cut into blocks of at most 30 s at the quietest point and read in batches; word times stay on the original timeline.

What is in this repository

File	Content
model.safetensors	the trained weights (fp32, 110 M parameters): the reader (6 blocks of width 768), the head (30 002 rows of 512), the 13 mixing weights, and the fine-tuned layers 11–12 of the audio tower (audio_tower.layers.10–11). The frozen layers are not included: they are loaded from google/embeddinggemma-2 at revision 914f7f8.
config.json	architecture, base model and revision, layers overlaid on the tower, mixing weights
units.json	the 30 002 output units: blank, unknown, and 30 000 Gemma tokenizer pieces (case and punctuation included), as text (symbols) and as ids in the Gemma tokenizer (gemma_ids)
lm/fr_4gram.bin	KenLM 4-gram, 8-bit quantised trie (0.85 GB), with unigrams.txt and lm_config.json (α 0.4, β 2.0, beam 128); the same in both versions
results.json	the evaluation figures and protocol

How it was made

Architecture

16 kHz audio -> log-mel (128 bands, Gemma recipe) -> EmbeddingGemma 2 audio tower, 12 conformer layers
             -> softmax mix of its 13 hidden states (learned) -> reader: Linear 1024->768, causal depthwise conv,
                6 pre-norm transformer blocks (12 heads) -> Linear 768->512 -> head over 30 002 word pieces -> CTC

One frame every 40 ms. Layers 1–10 of the tower are Google's, untouched; layers 11–12 are lightly fine-tuned. The head starts from Gemma's own input embedding of each word piece, so from the first step the head "knows" that ▁facture and ▁factures are close: the reader only has to learn to aim at the right place of an already organised space (the idea of Shahir's Embedding Gemma Transcribe). A rare word whose piece is not among the 30 000 is spelt with smaller pieces, so no word is ever dropped as unknown.

Data (≈ 3 205 h, all openly licensed)

Source	Hours used	Text
Multilingual LibriSpeech fr (audiobooks)	782	human; capitals and punctuation restored by Qwen2.5-7B, kept only if the words are unchanged (73 % kept)
Common Voice 19 fr	≈ 845	human
YODAS fr (YouTube), labelled for this model	≈ 866 kept of 1 469	two teachers must agree: Whisper large-v3 and Qwen3-ASR-1.7B, kept if their CER ≤ 0.2 and Whisper is confident; Qwen's punctuated text kept
Granary (YODAS subset labelled by NVIDIA)	463	NVIDIA labels
VoxPopuli fr	202	human
FLEURS fr	52 (10 h × 5)	human
African Accented French	12	human
MediaSpeech fr	10	human

No training data is redistributed here. The test and validation clips of every set are excluded from training and from the language model (8-gram decontamination).

Training

All the training ran on a single NVIDIA L40S (48 GB, Ada Lovelace generation). CTC loss, AdamW, bf16, SpecAugment on the log-mel, dynamic attention chunks in the reader; the tower layers are fine-tuned at a learning rate 30 times lower than the reader's, and only after the first 2 000 steps of each stage.

base (about 38 GPU hours): a 6-block reader with tower layers 11–12 fine-tuned, trained in four stages as the data grew from the human-labelled sets (1 521 h) to the full 3 205 h with the YouTube hours labelled meanwhile. Each stage starts from the best checkpoint of the previous one and restarts the learning rate. The published weights are the checkpoint with the lowest validation WER: 11.7 % on FLEURS dev (greedy), step 56 000.

large (base + about 17 GPU hours): two more stages on the same 3 205 h, three passes each, from the best checkpoint of base. First, tower layer 10 is fine-tuned as well (it starts from Google's weights; validation 11.5 %). Then the reader is grown from 6 to 10 blocks, the four new blocks starting as the identity so that the model's output is unchanged at the first step (validation 11.35 %, step 54 000: the published weights). As said above, an equally long continuation of base alone was not tried.

The checkpoint kept is always the one with the lowest validation WER, never the lowest loss, and never an average: averaging checkpoints did not help.

Language model

KenLM 4-gram on 21.5 M sentences (French Wikipedia and the training transcripts), decontaminated against all validation and test sets, quantised to 8 bits (1.7 GB → 0.85 GB for +0.03 WER). α and β tuned on a mixed dev set (FLEURS dev, held-out Common Voice and MLS clips): tuning on FLEURS alone cost up to 0.5 point elsewhere.

Limits

• French only. Strongest on read speech and parliament; spontaneous conversation is harder; untested on telephone audio and children.
• About one error in seven is right to the ear but wrong in writing (agreement, silent endings, homophones): a 4-gram only sees three words back. A rescoring LLM would help; not included.
• Proper nouns the model never heard come out phonetically ("Zeghidour", "Gradium" → "grad").
• Digits are written as words ("deux mille huit"), as in most of the training text.
• The punctuated labels of YouTube come from a model (Qwen3-ASR) and of MLS from an LLM: punctuation is good but not human-grade.

Credits

• Google DeepMind for EmbeddingGemma 2 (Apache 2.0).
• Shahir for the idea of reading speech in an embedding model's space; Quan Nguyen (QUAN-Transcribe) for the reader recipe; Pixel ML (sea-asr) for the two-teacher labelling and the layout of the published files; the open training code of norsk-asr, eg2-transcribe and smol-audio.
• Data: Multilingual LibriSpeech, Common Voice (Mozilla), VoxPopuli (Meta), FLEURS (Google), YODAS and Granary (ESPnet, NVIDIA), African Accented French, MediaSpeech. Teachers: Whisper large-v3 (OpenAI, MIT), Qwen3-ASR-1.7B (Alibaba, Apache 2.0), Qwen2.5-7B-Instruct (Apache 2.0).

Licence

Weights and code: Apache 2.0. The base model google/embeddinggemma-2 keeps its own terms, to accept on Hugging Face. The language model is built from French Wikipedia (CC BY-SA) and the training transcripts.