Qengineering/smolvlm2-500m-rk3588

SmolVLM2-500M for RK3588 NPU

This repository provides a hardware-accelerated port of SmolVLM2-500M optimized for Rockchip RK3588 NPU.

User:<image>Describe the image.

Answer: The image is a surreal and fantastical representation of a space station orbiting a planet in outer space. The station, which appears to be a large, cylindrical structure with multiple levels, is depicted as being made entirely of ice and water. The station's surface is covered in a thick layer of ice, creating a stark contrast against the barren landscape below.

The planet itself is a dark, icy world, with no visible signs of life or vegetation. However, there are several large, glowing orbs scattered across the surface, which seem to be floating in mid-air. These orbs could be either stars or other celestial bodies, adding an element of mystery and wonder to the scene.

The station's interior is also depicted as being made entirely of ice and water, with no visible doors or windows. The walls are covered in a thick layer of ice, creating a sense of isolation and confinement. However, there are several small, glowing orbs scattered throughout the interior, which could be either stars or other celestial bodies, adding an element of mystery and wonder to the scene.

The overall composition of the image creates a sense of awe and wonder, as if the viewer is witnessing a scene from a science fiction movie or a fantasy novel. The use of ice and water as the primary material for the station's structure adds a unique and otherworldly element to the scene. The glowing orbs could be either stars or other celestial bodies, adding an element of mystery and wonder to the scene.

Overall, the image is a beautiful and imaginative representation of a space station orbiting a planet in outer space, with ice and water as its primary materials. It creates a sense of isolation and confinement, while also adding an element of mystery and wonder to the scene. The use of ice and water as the primary material for the station's structure adds a unique and otherworldly element to the scene. The glowing orbs could be either stars or other celestial bodies, adding an element of mystery and wonder to the scene. The overall composition of the image creates a sense of awe and wonder, as if the viewer is witnessing a scene from a science fiction movie or a fantasy novel.

---

Model Files

Component	File	Precision
LLM	smolvlm2_500m_llm_w8a8_rk3588.rkllm	W8A8
Vision Encoder	smolvlm2_500m_vision_fp16_rk3588.rknn	FP16

Hardware Requirements

• Rockchip RK3588 / RK3588S
• RKNPU2 driver
• Tested on:
  • Rock 5C
  • Ubuntu 22.04 / 24.04 (Joshua Riek)

Runtime Requirements

• RKLLM runtime
• RKNN runtime (rknpu2)
• OpenCV (for image preprocessing)

Model performance benchmark (FPS)

All models, with C++ examples, can be found on the Q-engineering GitHub.

All LLM models are quantized to w8a8, while the VLM vision encoders use fp16.

model	RAM (GB)1	llm cold sec2	llm warm sec3	vlm cold sec2	vlm warm sec3	Resolution	Tokens/s
Qwen3-2B	3.1	21.9	2.6	10.0	0.9	448 x 448	11.5
Qwen3-4B	8.7	49.6	5.6	10.6	1.1	448 x 448	5.7
Qwen2.5-3B	4.8	48.3	4.0	17.9	1.8	392 x 392	7.0
Qwen2-7B	8.7	86.6	34.5	37.1	20.7	392 x 392	3.7
Qwen2-2.2B	3.3	29.1	2.5	17.1	1.7	392 x 392	12.5
InternVL3-1B	1.3	6.8	1.1	7.8	0.75	448 x 448	30
SmolVLM2-2.2B	3.4	21.2	2.6	10.5	0.9	384 x 384	11
SmolVLM2-500M	0.8	4.8	0.7	2.5	0.25	384 x 384	31
SmolVLM2-256M	0.5	1.1	0.4	2.5	0.25	384 x 384	54

¹ The total used memory; LLM plus the VLM.
² When an llm/vlm model is loaded for the first time from your disk to RAM or NPU, it is called a cold start.
The duration depends on your OS, I/O transfer rate, and memory mapping.
³ Subsequent loading (warm start) takes advantage of the already mapped data in RAM. Mostly, only a few pointers need to be restored.

Example Usage

• see: https://github.com/Qengineering/SmolVLM2-500M-NPU

Notes

• This is not a Transformers-compatible model
• This repository provides precompiled NPU binaries
• CPU fallback is not supported