tocsa/nemotron-3-nano-4b-width-pruned-manual

NVIDIA-Nemotron-3-Nano-4B-BF16 — Width-Pruned (manual config)

Width-pruned variant of NVIDIA-Nemotron-3-Nano-4B-BF16: hidden_size and FFN size reduced (hidden_size=2816, ffn_hidden_size=11200, layer count unchanged) for an approximate 15% total parameter reduction, following the Minitron importance-based width pruning approach (L2-norm / mean activation aggregation, single-shot).

Base model: Hybrid transformer/Mamba-SSM decoder-only LLM, 42 layers, hidden_size=3136, ffn_hidden_size=12544, 40 attention heads, 8 GQA key-value groups (~4.4B total params).

Compression Method

torchrun --nproc_per_node 1 prune_minitron.py \
    --hf_model_name_or_path checkpoints/NVIDIA-Nemotron-3-Nano-4B-BF16 \
    --calib_dataset_name wikitext \
    --calib_num_samples 512 \
    --prune_export_config '{"hidden_size": 2816, "ffn_hidden_size": 11200}' \
    --output_hf_path pruning/NVIDIA-Nemotron-3-Nano-4B-BF16-width-pruned-manual/ \
    --trust_remote_code

This is the pruned checkpoint later used as the student for knowledge distillation (see tocsa/nemotron-3-nano-4b-pruned-distilled-fp8 for the distilled + FP8 quantized descendant).

Evaluation

Evaluated 5-shot with lm_eval on WinoGrande, ARC-Easy, and PIQA:

Task	Score
winogrande	0.6338
arc_easy	0.7121
piqa	0.7144

For reference, the full BF16 teacher (NVIDIA-Nemotron-3-Nano-4B-BF16) scores 5-shot:

Task	Score
winogrande	0.6717
arc_easy	0.8136
piqa	0.7715

Framework

Produced with NVIDIA Model-Optimizer / Megatron-Bridge, as part of the DLI course "The Art of Compressing LLMs: Pruning, Distillation, and Quantization Demystified."

License

Inherits the license terms of the base model, nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16. Check the original model card for the applicable NVIDIA Open Model License terms before redistribution.