Rumble-2-Flash

Speech recognition model with native Hugging Face Transformers and vLLM support.

Usage (Transformers)

uv pip install torch librosa transformers

import librosa
import torch
from transformers import AutoProcessor, Qwen3ASRForConditionalGeneration

model_id = "Reson8/Rumble-2-Flash"
processor = AutoProcessor.from_pretrained(model_id)
model = Qwen3ASRForConditionalGeneration.from_pretrained(
    model_id, dtype=torch.bfloat16
).to("cuda")

audio, _ = librosa.load("audio.wav", sr=16000)
inputs = processor.apply_transcription_request(
    audio=audio, language="English", return_tensors="pt"
).to(model.device, dtype=model.dtype)

with torch.inference_mode():
    output = model.generate(**inputs, max_new_tokens=512)

generated = output[:, inputs["input_ids"].shape[-1]:]
print(processor.batch_decode(generated, skip_special_tokens=True)[0])

Omit language to detect it automatically.

Usage (vLLM)

uv pip install "vllm[audio]" openai
vllm serve Reson8/Rumble-2-Flash --max-model-len 4096

import base64
from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")
audio = base64.b64encode(open("audio.wav", "rb").read()).decode()

response = client.chat.completions.create(
    model="Reson8/Rumble-2-Flash",
    messages=[{
        "role": "user",
        "content": [{
            "type": "audio_url",
            "audio_url": {"url": f"data:audio/wav;base64,{audio}"},
        }],
    }],
    temperature=0,
    extra_body={"chat_template_kwargs": {"language": "English"}},
)
print(response.choices[0].message.content)

Transcription options

Pass these options through vLLM's chat_template_kwargs, or directly to processor.apply_chat_template in Transformers.

Option	Default	Effect
language	unset	Forces the language using its English name, such as "Dutch" or "English". If unset, output starts with language <Name><asr_text>.
digits	true	Numbers as digits; false writes them as words.
filler	false	true keeps fillers such as "um" and "uh".
verbatim	true	false collapses repeated words.

Auxiliary CTC head

The repo also contains the model's auxiliary CTC head: one linear layer (896 -> 99,631) on the output of the audio encoder's final LayerNorm (model.audio_tower.ln_post). It was trained jointly with the model and gives a transcript from the audio encoder alone, with no LLM decoding: greedy CTC, 13 frames per second of audio, 8 s encoder attention windows. It is fast, but much less accurate than the model's normal output.

File	Contents
ctc/ctc_head.safetensors	weight [99631, 896] and bias [99631], fp32
ctc/ctc_vocab.json	token_ids: head class -> token id of this repo's tokenizer (class k = token k); blank: 99630
ctc_head.py	ResonantCTC: loads the head, runs the audio encoder with 8 s windows, greedy CTC -> text

Transformers and vLLM ignore these files (the head sits in ctc/ so that vLLM does not load it as model weights).

import sys
import librosa
import torch
from huggingface_hub import snapshot_download
from transformers import AutoProcessor, Qwen3ASRForConditionalGeneration

path = snapshot_download("Reson8/Rumble-2-Flash")
sys.path.insert(0, path)
from ctc_head import ResonantCTC

processor = AutoProcessor.from_pretrained(path)
model = Qwen3ASRForConditionalGeneration.from_pretrained(path, dtype=torch.bfloat16).to("cuda")
ctc = ResonantCTC.from_pretrained(path, model, processor)

audio, _ = librosa.load("audio.wav", sr=16000)
print(ctc.transcribe(audio))

Command line, from the downloaded repo: python ctc_head.py <repo dir> audio.wav (needs soundfile).

FLEURS-30, greedy CTC (test split, all 22,586 utterances = 77 h; WER, CER for zh / ja / ko / th; no language prompt): mean 47.1 (the previous CTC head: 50.1). Best: en 16.3, ko 20.6, vi 23.2, zh 24.1. Weakest: el 95, ar 70, fi 66.

ar	cs	da	de	el	en	es	fa	fi	fil	fr	hi	hu	id	it	ja	ko	mk	ms	nl	no	pl	pt	ro	ru	sv	th	tr	vi	zh
70.1	60.3	55.0	39.4	94.7	16.3	25.4	63.1	66.3	49.5	38.2	58.1	63.0	44.3	33.7	27.7	20.6	66.0	51.9	43.0	47.2	57.3	31.8	49.8	55.6	56.0	24.4	57.2	23.2	24.1

Known limitations: the head emits sub-word pieces frame by frame, so word starts are sometimes doubled or mis-cased; scripts whose characters span several byte-level tokens can come out as invalid UTF-8 (U+FFFD); numbers mix digits and words, as the head has no digits option.