Hugging Face's logo

primitive-ai
/
Qwen3.8-Flash-Next-PLE-quant

qwen3.8-flash-next
ple
quantization
Model card Files
xet
 Community
3
Qwen3.8-Flash-Next-PLE-quant / worker_image_quant.py

Commit History

  1. Optional resident mode for the quantized tables (VLLM_PLE_QUANT_RESIDENT) (#2)⁠

    jagat-primitive-org +1: mgabor3141
    a0fa93f

  1. Overlay: drop a checkpoint's own table tensors when a sidecar is attached, including the monolithic weight/weight_scale form — lets FP8-table checkpoints (nvidia, official FP8) serve on one card

    jagat-primitive-org
    Verified signature
    0a85198

  2. Overlay: build the n-gram table parameter on the meta device when VLLM_PLE_QUANT_DIR is set, and swap the stub Parameter instead of set_data — removes the 102 GB virtual reservation that the kernel's overcommit heuristic refuses on hosts with less RAM+swap than the table (field report, 64 GB host); validated under an emulated 67/99 GiB commit limit, sanity PASS, tool-calling 77.0 (n=3), 81 tok/s c1

    jagat-primitive-org
    Verified signature
    cf50620

  1. Upload worker_image_quant.py with huggingface_hub

    jagat-primitive-org
    Verified signature
    a815cac

  1. Upload worker_image_quant.py with huggingface_hub

    jagat-primitive-org
    Verified signature
    c7550ff