gaggle-reranker-20261005

Gaggle (base reranker): the pointwise decoder-only reranker described in our paper, and the model behind its headline results. It is a model soup — the equal-weight parameter average of four fine-tuning runs that share one recipe — built on castorini/gaggle-nanochat-pretrained-climbmix-20260924, which we pre-trained from scratch on ClimbMix. No open-weight third-party backbone is involved at any stage: the corpus, the pre-training recipe and the relevance fine-tuning data are all public.

The model scores one (query, passage) pair at a time and is meant to reorder the candidates returned by a first-stage retriever.

Model

Architecture	the backbone's nanochat depth-34 transformer (34 layers, width 2,176, 17 heads of dimension 128, 2,048-token context), causal attention
Parameters	3.22B. The language-model output rows are discarded except the two used for scoring, so the reranker is smaller than the 3.29B base model
Scoring	s(q, d) = logit(" true") − logit(" false") at the last prompt token, read from two pre-trained LM head rows (2 × 2,176 = 4,352 parameters). No classification head was added
Prompt	<|bos|>Query: {query}\nPassage: {passage}\nRelevant:
Truncation	trained with 128 query tokens and 512 passage tokens; not a hard limit at inference, where the backbone's 2,048-token context is the only bound
Tokenizer	the backbone's 32,768-entry byte-level BPE

Training

• Initialization: the causal pre-trained checkpoint, step 227,158.
• Data: RLHN-250K (247,534 instances), a curated subset of seven retrieval collections from the BGE training mixture in which an LLM cascade relabels false negatives as positives.
• Objective: localized contrastive estimation (LCE) — a softmax over one positive and K sampled hard negatives per query at temperature τ = 1, with K ∈ {7, 15, 23} chosen per instance as the largest value its negative pool supports. No distillation, no synthetic data, no teacher.
• Optimization: one epoch, AdamW (β = 0.9/0.95, weight decay 0.01), peak learning rate 5 × 10⁻⁵ with 150 warm-up steps then cosine decay to zero, gradient clipping at global norm 12, an effective batch of 128 query groups per update, ≈1,931 updates on two GPUs. FP32 weights and optimizer state with TF32 matmuls.

The soup

Four runs of that recipe were averaged with weight 0.25 each, in float64:

parent	seed	updates	hardware
run 1	17	1,931	2 × RTX PRO 6000 Blackwell
run 2	18	1,933	2 × RTX PRO 6000 Blackwell
run 3	17	1,931	2 × RTX PRO 6000 Blackwell
run 4	17	1,931	2 × H100, gradient caching

The runs differ in seed, hardware and microbatch packing, so the soup averages over implementation nondeterminism as well as seed. Averaging cost nothing at inference: the result is one checkpoint of the same size and shape as each parent.

Evaluation

nDCG@10, reranking the top-100 BM25 candidates retrieved with Pyserini, 128/512 truncation, scored with trec_eval.

	DL19	DL20	DL21	DL22	DL23	mean
BM25 (first stage)	0.506	0.480	0.446	0.269	0.263	0.393
This model	0.757	0.723	0.714	0.533	0.480	0.641
mean of the four parents	0.754	0.721	0.712	0.531	0.481	0.640

	COVID	News	Robust04	NFCorpus	SciFact	SCIDOCS	FiQA	mean
BM25 (first stage)	0.595	0.395	0.407	0.322	0.679	0.149	0.236	0.398
This model	0.859	0.515	0.592	0.380	0.790	0.236	0.455	0.547
mean of the four parents	0.858	0.513	0.589	0.380	0.783	0.234	0.451	0.544

Run-to-run sample standard deviation across the four parents is 0.001 on benchmark means and 0.003 on individual collections; differences of that size should be read as noise. The soup matches the strongest parent on TREC DL and exceeds all four on BEIR.

Four of the seven BEIR collections — TREC-COVID, TREC-News, Robust04 and NFCorpus — are out of domain with respect to RLHN-250K; FiQA and SCIDOCS have sibling training data in it.

Usage

Using Transformers

pip install "transformers>=4.52" "torch>=2.5"

import torch
from transformers import AutoModelForSequenceClassification, AutoTokenizer

repo = "castorini/gaggle-reranker-20261005"
tokenizer = AutoTokenizer.from_pretrained(repo, trust_remote_code=True)
model = AutoModelForSequenceClassification.from_pretrained(
    repo, trust_remote_code=True, dtype=torch.bfloat16).to("cuda").eval()

query = "what are the symptoms of iron deficiency anemia?"
passages = [
    "Iron deficiency anemia causes fatigue, shortness of breath, pale skin, and brittle nails.",
    "Iron is a metallic element with atomic number 26, widely used in construction and manufacturing.",
    "A balanced diet includes leafy greens, legumes, and red meat as sources of dietary iron.",
]
for score, passage in model.rerank(query, passages, tokenizer=tokenizer):
    print(f"{score:+.3f}  {passage[:70]}")

Output on one H200 GPU:

+25.874  Iron deficiency anemia causes fatigue, shortness of breath, pale skin,
+17.497  A balanced diet includes leafy greens, legumes, and red meat as source
+14.258  Iron is a metallic element with atomic number 26, widely used in const

trust_remote_code=True is needed for the tokenizer as well as the model: the architecture is nanochat's rather than one transformers ships, so the two files in this repo define it. They are plain PyTorch, import nothing beyond torch and transformers, and are short enough to read first. No nanochat checkout and no Flash Attention build are involved.

Three entry points, in increasing order of control:

• model.rerank(query, passages, tokenizer=tokenizer) returns (score, passage) pairs, sorted.
• model.compute_score([(query, passage), ...], tokenizer=tokenizer) returns one score per pair in order, batched internally (batch_size=16 by default).
• model(input_ids, attention_mask) returns logits of shape (B, 1) -- the score -- and true_false_logits of shape (B, 2), for your own batching; model.build_input_ids(tokenizer, query, passage) assembles the training prompt, and both padding sides are handled. AutoModel gives the trunk alone, returning final hidden states as a tensor; note that at padded positions those states are not meaningful.

Notes:

• Scores are unbounded logit differences, not probabilities. As the example shows, all three passages score positive and only their order is meaningful: use the scores to sort candidates within one query, and do not compare or threshold them across queries without calibration.
• The model is pointwise: each pair is scored independently, so candidates can be batched and scored in parallel.
• The 128/512 caps are what the model was trained with and what every number above was measured at, but they are not enforced at inference: change config.max_query_len / config.max_doc_len, or build the ids yourself, and the model still scores up to the backbone's 2,048-token context. Inputs longer than training are simply untested.
• bf16, as in the example, is what every number above was measured at. The two scoring rows follow the load dtype and the difference of their logits is taken in fp32, which is what the research code this was ported from does.
• The port is exact, not merely close: run against that research code on the same GPU, this repo produces bitwise identical scores over 1,200 SciFact pairs -- one pair per forward and in padded batches alike -- and nDCG@10 agreeing to four decimals on TREC DL19 and DL20. Right padding is what was verified, and what the shipped tokenizer and compute_score use; left padding is supported and differs only by bf16 rounding, because it shifts the absolute positions the rotary embeddings see.

Using Sentence Transformers

pip install "sentence-transformers>=5.4.0" "transformers>=4.45.0" "torch>=2.5"

from sentence_transformers import CrossEncoder

model = CrossEncoder("castorini/gaggle-reranker-20261005", trust_remote_code=True)

query = "what are the symptoms of iron deficiency anemia?"
passages = [
    "Iron deficiency anemia causes fatigue, shortness of breath, pale skin, and brittle nails.",
    "Iron is a metallic element with atomic number 26, widely used in construction and manufacturing.",
    "A balanced diet includes leafy greens, legumes, and red meat as sources of dietary iron.",
]
scores = model.predict([(query, passage) for passage in passages])
print(scores.tolist())
# [25.879966735839844, 14.230884552001953, 17.403305053710938]

rankings = model.rank(query, passages)
print([result["corpus_id"] for result in rankings])
# [0, 2, 1]

This example uses fp32. For bf16 inference, pass model_kwargs={"torch_dtype": "bfloat16"} when loading. The input adapter calls build_input_ids to preserve the training prompt, separate 128/512 query and passage token limits, and right padding. Adjust the limits through model.config.max_query_len and model.config.max_doc_len. Scores are raw logit differences, with no sigmoid applied.

Files

• model-*.safetensors (12.9 GB over four shards) and model.safetensors.index.json: the souped weights in fp32. Optimizer and RNG state are not included, so the model scores but does not resume.
• config.json: the architecture, plus the reranker's own settings -- the prompt, its token ids, the length caps, and the " true"/" false" ids that the two rows of score correspond to.
• configuration_gaggle.py, modeling_gaggle.py: the architecture itself, loaded by trust_remote_code=True.
• tokenizer.json, tokenizer_config.json: the backbone tokenizer, with pad and eos pinned to <|assistant_end|> and right padding, so no setup helper is needed.
• to_nanochat_ckpt.py: rebuilds the original training checkpoint from these files, for the paper's own evaluation harness. It is a bitwise round trip, which is why no .pt is shipped.
• SHA256SUMS: checksums of the files above.

License and data provenance

The weights are released under the MIT license. The data they were derived from carries its own terms, which downstream users should check against their intended use:

• RLHN-250K, the relevance fine-tuning data, is CC-BY-SA-4.0. It is curated from seven collections of the BGE training mixture, whose own licenses vary.
• ClimbMix, the pre-training corpus of the backbone, is CC-BY-NC-4.0.

Please cite:

@misc{lin2026projectgreenhouseprogressfully,
      title={Project Greenhouse: Progress Toward Fully Open and Sovereign Agentic Search},
      author={Jimmy Lin and Sahel Sharifymoghaddam and Lingwei Gu and Nour Jedidi},
      year={2026},
      eprint={2610.11922},
      archivePrefix={arXiv},
      primaryClass={cs.IR},
      url={https://arxiv.org/abs/2610.11922},
}

Related

• castorini/gaggle-nanochat-pretrained-climbmix-20260924: the causal base model this reranker was fine-tuned from.