Instructions to use IsValorum/Apex-35B-V1.0-APEX-I-MiniPlus-V2.1-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use IsValorum/Apex-35B-V1.0-APEX-I-MiniPlus-V2.1-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf IsValorum/Apex-35B-V1.0-APEX-I-MiniPlus-V2.1-GGUF # Run inference directly in the terminal: llama cli -hf IsValorum/Apex-35B-V1.0-APEX-I-MiniPlus-V2.1-GGUF
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf IsValorum/Apex-35B-V1.0-APEX-I-MiniPlus-V2.1-GGUF # Run inference directly in the terminal: llama cli -hf IsValorum/Apex-35B-V1.0-APEX-I-MiniPlus-V2.1-GGUF
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf IsValorum/Apex-35B-V1.0-APEX-I-MiniPlus-V2.1-GGUF # Run inference directly in the terminal: ./build/bin/llama cli -hf IsValorum/Apex-35B-V1.0-APEX-I-MiniPlus-V2.1-GGUF
Use Docker
docker model run hf.co/IsValorum/Apex-35B-V1.0-APEX-I-MiniPlus-V2.1-GGUF
- Llama new
- LM Studio
- Jan
- vLLM
How to use IsValorum/Apex-35B-V1.0-APEX-I-MiniPlus-V2.1-GGUF with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "IsValorum/Apex-35B-V1.0-APEX-I-MiniPlus-V2.1-GGUF" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "IsValorum/Apex-35B-V1.0-APEX-I-MiniPlus-V2.1-GGUF", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/IsValorum/Apex-35B-V1.0-APEX-I-MiniPlus-V2.1-GGUF
- Ollama
How to use IsValorum/Apex-35B-V1.0-APEX-I-MiniPlus-V2.1-GGUF with Ollama:
ollama run hf.co/IsValorum/Apex-35B-V1.0-APEX-I-MiniPlus-V2.1-GGUF
- Unsloth Desktop
- Pi
How to use IsValorum/Apex-35B-V1.0-APEX-I-MiniPlus-V2.1-GGUF with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf IsValorum/Apex-35B-V1.0-APEX-I-MiniPlus-V2.1-GGUF
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:9931/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "IsValorum/Apex-35B-V1.0-APEX-I-MiniPlus-V2.1-GGUF" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use IsValorum/Apex-35B-V1.0-APEX-I-MiniPlus-V2.1-GGUF with Docker Model Runner:
docker model run hf.co/IsValorum/Apex-35B-V1.0-APEX-I-MiniPlus-V2.1-GGUF
- Lemonade
How to use IsValorum/Apex-35B-V1.0-APEX-I-MiniPlus-V2.1-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull IsValorum/Apex-35B-V1.0-APEX-I-MiniPlus-V2.1-GGUF
Run and chat with the model
lemonade run user.Apex-35B-V1.0-APEX-I-MiniPlus-V2.1-GGUF-{{QUANT_TAG}}List all available models
lemonade list
- Hermes Agent
How to use IsValorum/Apex-35B-V1.0-APEX-I-MiniPlus-V2.1-GGUF with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf IsValorum/Apex-35B-V1.0-APEX-I-MiniPlus-V2.1-GGUF
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:9931/v1 hermes config set model.default IsValorum/Apex-35B-V1.0-APEX-I-MiniPlus-V2.1-GGUF
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use IsValorum/Apex-35B-V1.0-APEX-I-MiniPlus-V2.1-GGUF with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf IsValorum/Apex-35B-V1.0-APEX-I-MiniPlus-V2.1-GGUF
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:9931/v1 \ --custom-model-id "IsValorum/Apex-35B-V1.0-APEX-I-MiniPlus-V2.1-GGUF" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Apex-35B-V1.0 APEX-I MiniPlus V2.1
APEX Quantization Series by IsValorum
Independently computed using dedicated cloud compute clusters, funded 100% out-of-pocket.
If you consider these releases to be of high quality and they help you in your day-to-day workflow, consider supporting on Ko-fi. Every contribution directly sustains future cloud compute hours and helps support my university studies.
Family cross-links
Architecture and precision
- Base model:
Apexintelligence-AI/Apex-35B-V1.0(Apache-2.0). BF16 single GGUF reference from prithivMLmods/Apex-35B-V1.0-MXFP4-MoE-GGUF. - Architecture: 40 layers, 30 linear-attention/SSM + 10 full-attention; 256 routed experts + 1 shared, 8 active; hidden 2048; expert FF 512; 262,144-token context; 248,320 vocab; 753 tensors including the MTP block 40.
- iMatrix: public
Apexintelligence-AI_Apex-35B-V1.0-imatrix.ggufby bartowski, 384,343,168 bytes, SHA256b1f64857f0fa390657d8e9155e0bb0ac39ba7503b1f9dbd3c5705237016440e3, verified before use. - Recurrent shielding:
ssm_alpha,ssm_a,ssm_dt.bias,ssm_conv1dand RMSNorms remainF32. - Layer 0: no special boundary; the same SSM types as the rest of the layers.
- MTP: block 40, 20 tensors; F32 norms,
Q4_Kq/k/v andnextn.eh_proj,Q6_Koutput,Q4_0routed experts,Q5_Kshared experts. No DSpark draft is included for this family. - Vision projector: this repository ships the text GGUF; the upstream
mmproj-Apexintelligence-AI_Apex-35B-V1.0-bf16.ggufcompanion is not included.
Surgical Tensor Quantization Map (Audited from GGUF)
The exact tensor breakdown below has been verified directly from the compiled binary weights:
| Layer Group | Sub-Component / Tensor | Qty | Precision | Engineering Rationale |
|---|---|---|---|---|
| Global Output Head | output.weight |
1 | Q6_K |
Preserves near-FP16 token classification; stable syntax and bracket behavior. |
| Global Embeddings | token_embd.weight |
1 | Q4_K |
High-fidelity vocabulary representation. |
| All Normalizations | output_norm, attn_*_norm, post_attention_norm, ssm_norm + MTP norms |
138 | F32 |
Uncompressed numerical stability across all 40 layers plus block 40. |
| Expert Routers | blk.*.ffn_gate_inp, ffn_gate_inp_shexp (40 layers + MTP) |
82 | F32 |
Uncompressed routing over 256 experts; zero router drift. |
| Attention Gates | blk.*.attn_gate.weight (30 Hybrid Layers) |
30 | Q8_0 |
High-precision gating across the hybrid linear-attention layers. |
| Shared Foundation Experts | blk.*.ffn_{gate,down,up}_shexp (All 40 Layers) |
120 | Q5_K |
Active on every token; protected in high precision. |
| Periodic Full Attention | blk.{3,7,11,...}.attn_q/k/v (10 Anchor Layers) |
30 | Q4_K |
Full quadratic-attention anchors for deep retrieval. |
| Periodic Full Attention | blk.{3,7,11,...}.attn_output (10 Anchor Layers) |
10 | Q6_K |
Armored attention output projection over long context. |
| Recurrent SSM Scales | blk.*.ssm_alpha, ssm_a, ssm_conv1d, ssm_dt |
120 | F32 |
F32 shielding against recurrent-state drift. |
| Linear Attention & SSM | blk.*.attn_qkv, ssm_beta, ssm_out |
90 | Q3_K |
Linear-attention projections kept lean for CPU execution. |
| Edge MoE Experts | Layers 0-9 & 30-39 (ffn_*_exps) |
60 | Q3_K |
Routed experts calibrated with the importance matrix. |
| Core MoE Experts | Layers 10-29 (ffn_*_exps) |
60 | IQ3_XXS |
Deep-band routed experts calibrated with the importance matrix. |
| MTP Block 40 | blk.40.{attn_output,attn_q/k/v,nextn.eh_proj,ffn_*_exps,ffn_*_shexp} |
11 | Q6_K output; Q4_K q/k/v + eh_proj; Q4_0 routed experts; Q5_K shared |
Companion MTP head; 20 tensors total including its 7 F32 norms and 2 routers counted above. |
The Qty column sums to 753. The complete audited list is tensor-map.tsv.
- Downloads last month
- 1,427
We're not able to determine the quantization variants.
Model tree for IsValorum/Apex-35B-V1.0-APEX-I-MiniPlus-V2.1-GGUF
Base model
Qwen/Qwen3.6-35B-A3B