Instructions to use Jundot/Qwen3.8-Flash-Next-oQ2h-mtp with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use Jundot/Qwen3.8-Flash-Next-oQ2h-mtp with MLX:
# Make sure mlx-vlm is installed # pip install --upgrade mlx-vlm from mlx_vlm import load, generate from mlx_vlm.prompt_utils import apply_chat_template from mlx_vlm.utils import load_config # Load the model model, processor = load("Jundot/Qwen3.8-Flash-Next-oQ2h-mtp") config = load_config("Jundot/Qwen3.8-Flash-Next-oQ2h-mtp") # Prepare input image = ["http://images.cocodataset.org/val2017/000000039769.jpg"] prompt = "Describe this image." # Apply chat template formatted_prompt = apply_chat_template( processor, config, prompt, num_images=1 ) # Generate output output = generate(model, processor, formatted_prompt, image) print(output) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Pi
How to use Jundot/Qwen3.8-Flash-Next-oQ2h-mtp with Pi:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "Jundot/Qwen3.8-Flash-Next-oQ2h-mtp"
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "mlx-lm": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "Jundot/Qwen3.8-Flash-Next-oQ2h-mtp" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Hermes Agent
How to use Jundot/Qwen3.8-Flash-Next-oQ2h-mtp with Hermes Agent:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "Jundot/Qwen3.8-Flash-Next-oQ2h-mtp"
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default Jundot/Qwen3.8-Flash-Next-oQ2h-mtp
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use Jundot/Qwen3.8-Flash-Next-oQ2h-mtp with OpenClaw:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "Jundot/Qwen3.8-Flash-Next-oQ2h-mtp"
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "Jundot/Qwen3.8-Flash-Next-oQ2h-mtp" \ --custom-provider-id mlx-lm \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Qwen3.8-Flash-Next-oQ2h-mtp
This is an oQh checkpoint. It needs oMLX 0.8.0 or later with oQh support. Earlier oMLX versions cannot load it.
This model was quantized using oQ mixed-precision quantization in the oQh format (jundot/omlx#5317).
A mixed-precision MLX checkpoint of Qwen/Qwen3.8-Flash-Next, an experimental preview of the architecture behind Qwen4: a mixture-of-experts vision-language model with 125B parameters and 6B activated, plus a 51B n-gram embedding and a 4B MTP module, that pairs Gated DeltaNet with Qwen Sparse Attention (QSA) and uses Gated Residual streams. It includes the language model, the n-gram embedding, the vision encoder and the multi-token prediction (MTP) head.
The effective average is 2.738 bits/weight for the language backbone and 3.442 bits/weight for the complete checkpoint.
What oQh changes
oQe already gives high quality and is the recommended setting in oMLX. It uses the standard MLX format, so any MLX engine that supports the model can run it. oQh is built for oMLX to push quality further at low bits.
oQh rotates the expert weights with a Hadamard transform before quantizing them. Rotated weights need fewer scales, and oQh spends the saved space on extra bits where they help most. The file stays the same size as oQe.
Quality vs oQe
KL divergence from the original FP8 model, lower is better. oQe and oQh at the same level are the same size. FLORES+ and Aya are prompts in 7 languages, and code is Python source. The sizes below leave out the MTP head.
| Level | Size | FLORES+ KL, oQe -> oQh | Aya KL, oQe -> oQh | Code KL, oQe -> oQh |
|---|---|---|---|---|
| oQ2 | 70.8 GiB | 0.27 -> 0.17 (-38%) | 0.36 -> 0.14 (-62%) | 0.32 -> 0.24 (-24%) |
| oQ3 | 84.9 GiB | 0.082 -> 0.060 (-27%) | 0.055 -> 0.042 (-23%) | 0.14 -> 0.11 (-18%) |
| oQ4 | 98.8 GiB | 0.034 -> 0.029 (-15%) | 0.018 -> 0.018 (+3%) | 0.065 -> 0.058 (-12%) |
At oQ4 one 1,768-token Aya document holds the whole Aya difference: without it oQh is 20% below oQe on the other 303 documents.
Quantization and Bit Distribution
oQ2h starts from the oQ2 budget and spends the bytes saved on group scales on wider codes for the narrowest and most sensitive tensors. Attention, the shared experts, the token embedding, the output head and the MTP head stay in affine groups, and the n-gram embedding stays at 4 bits.
| Storage format | Logical weights | Share of total | Tensor storage | Effective bits/weight |
|---|---|---|---|---|
| oQh 2-bit, row scale | 62.076B | 34.487% | 14.499 GiB | 2.01 |
| oQh 3-bit, row scale | 58.720B | 32.622% | 20.639 GiB | 3.02 |
| Affine 4-bit, group 64 | 2.517B | 1.398% | 1.318 GiB | 4.50 |
| Affine 8-bit, group 128 | 0.241B | 0.134% | 0.231 GiB | 8.25 |
| Affine 8-bit, group 64 | 4.731B | 2.628% | 4.681 GiB | 8.50 |
| BF16 | 0.516B | 0.287% | 0.961 GiB | 16.00 |
| I64 | 0.000B | 0.000% | 0.000 GiB | 64.00 |
| N-gram affine 4-bit, group 32 | 51.200B | 28.445% | 29.802 GiB | 5.00 |
| Total | 180.000B | 100% | 72.131 GiB | 3.442 |
Storage Breakdown
| Component | Logical weights | Storage |
|---|---|---|
| Language backbone | 125.744B | 43.039 GB / 40.083 GiB |
| N-gram embedding | 51.200B | 32.000 GB / 29.802 GiB |
| Vision encoder | 0.449B | 0.898 GB / 0.836 GiB |
| MTP head | 2.607B | 1.513 GB / 1.409 GiB |
| Total | 180.000B | 77.450 GB / 72.131 GiB |
License
Qwen Community License 1.0, following the source model. See LICENSE.
- Downloads last month
- 178
2-bit
Model tree for Jundot/Qwen3.8-Flash-Next-oQ2h-mtp
Base model
Qwen/Qwen3.8-Flash-Next