gaggle-reranker-run1-20260926

One of the four fine-tuning runs averaged into castorini/gaggle-reranker-20261005, the Gaggle base reranker. The paper refers to it as baseline run 1, and it is the run every contrastive condition in the paper is compared against.

All four runs share one recipe and differ only in random seed, hardware and microbatch packing; we release them so that the run-to-run variability behind the soup can be inspected rather than taken on trust. Like the soup, this model is fine-tuned from castorini/gaggle-nanochat-pretrained-climbmix-20260924, which we pre-trained from scratch on ClimbMix, so no third-party open-weight backbone is involved.

Model

Architecture	the backbone's nanochat depth-34 transformer (34 layers, width 2,176, 17 heads of dimension 128, 2,048-token context), causal attention
Parameters	3.22B (3,215,335,146), full fine-tuning
Scoring	s(q, d) = logit(" true") − logit(" false") at the last prompt token, read from two pre-trained LM head rows (4,352 parameters). No classification head
Prompt	<|bos|>Query: {query}\nPassage: {passage}\nRelevant:
Truncation	trained with 128 query tokens and 512 passage tokens; not enforced at inference (backbone context 2,048 tokens)

Training

• Finished: 2026-09-26 (Castorini).
• Data: RLHN-250K, 247,534 instances, one epoch.
• Objective: LCE at τ = 1 over one positive and K ∈ {7, 15, 23} hard negatives per query.
• Optimization: AdamW (β = 0.9/0.95, weight decay 0.01), peak learning rate 5 × 10⁻⁵, 150 warm-up steps then cosine decay to zero, gradient clipping at global norm 12, 128 query groups per update, seed 17, 1,931 updates. FP32 weights and optimizer state, TF32 matmuls, gradient checkpointing.
• Hardware: 2 × NVIDIA RTX PRO 6000 Blackwell (96 GB); microbatch budget 39,312 tokens per GPU.

Evaluation

nDCG@10 reranking the top-100 BM25 candidates, 128/512 truncation, scored with trec_eval.

	DL19	DL20	DL21	DL22	DL23	mean
BM25 (first stage)	0.506	0.480	0.446	0.269	0.263	0.393
This model	0.7546	0.7199	0.7146	0.5329	0.4849	0.6414

	COVID	News	Robust04	NFCorpus	SciFact	SCIDOCS	FiQA	mean
BM25 (first stage)	0.595	0.395	0.407	0.322	0.679	0.149	0.236	0.398
This model	0.8584	0.5101	0.5905	0.3806	0.7838	0.2329	0.4505	0.5438

Across the four runs the sample standard deviation is 0.001 on benchmark means and 0.003 on individual collections, so differences of that size between these checkpoints are noise rather than signal. The soup of all four reaches 0.641 on TREC DL and 0.547 on BEIR.

Usage

Identical to the soup's -- see castorini/gaggle-reranker-20261005 for a runnable example; substitute this repository id. The checkpoint format, prompt, scoring rule and length caps are the same:

import torch
from transformers import AutoModelForSequenceClassification, AutoTokenizer

repo = "castorini/gaggle-reranker-run1-20260926"
tokenizer = AutoTokenizer.from_pretrained(repo, trust_remote_code=True)
model = AutoModelForSequenceClassification.from_pretrained(
    repo, trust_remote_code=True, dtype=torch.bfloat16).to("cuda").eval()
print(model.compute_score([("what causes iron deficiency anemia?",
                            "Iron deficiency anemia causes fatigue and pale skin.")],
                          tokenizer=tokenizer))

Files

• model-*.safetensors and model.safetensors.index.json: the weights in fp32. Optimizer and RNG state are not included, so the model scores but does not resume.
• config.json, configuration_gaggle.py, modeling_gaggle.py: the architecture and the reranker's settings; trust_remote_code=True loads the two code files.
• tokenizer.json, tokenizer_config.json: the backbone tokenizer, with pad and eos pinned.
• to_nanochat_ckpt.py: rebuilds the original training checkpoint from these files (a bitwise round trip), for the paper's own evaluation harness.
• SHA256SUMS: checksums of the files above.

License and data provenance

The weights are released under the MIT license. The data they were derived from carries its own terms, which downstream users should check against their intended use:

• RLHN-250K, the relevance fine-tuning data, is CC-BY-SA-4.0. It is curated from seven collections of the BGE training mixture, whose own licenses vary.
• ClimbMix, the pre-training corpus of the backbone, is CC-BY-NC-4.0.

Please cite:

@misc{lin2026projectgreenhouseprogressfully,
      title={Project Greenhouse: Progress Toward Fully Open and Sovereign Agentic Search},
      author={Jimmy Lin and Sahel Sharifymoghaddam and Lingwei Gu and Nour Jedidi},
      year={2026},
      eprint={2610.11922},
      archivePrefix={arXiv},
      primaryClass={cs.IR},
      url={https://arxiv.org/abs/2610.11922},
}

Related

• castorini/gaggle-reranker-20261005: the soup of this run and its three siblings — the model used for the paper's headline results.
• castorini/gaggle-nanochat-pretrained-climbmix-20260924: the base model.