Hugging Face's logo

AXERA-TECH
/
jina-embeddings-v5-omni-small

Feature Extraction
Transformers
multilingual
axera
ax650
embedding
multimodal
vision
audio
lora
Model card Files
xet
 Community

jina-embeddings-v5-omni-small on AXERA NPU

Ready-to-run deployment package for jinaai/jina-embeddings-v5-omni-small on AX650 / NPU3.

  • Runtime: bundled axllm service with an OpenAI-compatible /v1/embeddings API
  • Inputs: text and images through the packaged API and demo; fixed-profile image and audio encoder models are included
  • Output: one L2-normalized 1024-dimensional embedding per request
  • Tasks: retrieval, clustering, classification, and text matching, selected at runtime without loading four complete text backbones
  • Included assets: compiled .axmodel files, tokenizer data, packed task adapters, sample images, validation results, and a browser demo

Supported Platform

  • AX650 / NPU3
  • Single NPU

Performance

All measurements below were taken on one AX650 NPU with the files in this package. Standalone .axmodel latency uses 3 warm-up runs followed by 20 measured runs. API latency includes tokenization, adapter selection, model execution, and local HTTP overhead.

End-to-End Text Embedding and Task Switching

Scenario Average time
Same-task short-text embedding request 256.933 ms
Alternating retrieval / clustering requests 378.273 ms
Internal switch between loaded adapters 117.000 ms

The model encodes each input in one forward pass. It has a prefill stage but no autoregressive decode stage.

Standalone Module Latency

Module Fixed input profile Average time
Text backbone, sum of 28 layer models 256 tokens, shape group 1 189.957 ms
Embedding post-processing [1, 1024] output 8.192 ms
Vision encoder 256×256 image 50.675 ms
Vision merger, retrieval 64 visual tokens 1.144 ms
Vision merger, clustering 64 visual tokens 1.140 ms
Vision merger, classification 64 visual tokens 1.142 ms
Vision merger, text matching 64 visual tokens 1.142 ms
Audio encoder 800 mel frames 209.989 ms
Audio projector, retrieval task-specific 0.290 ms
Audio projector, clustering task-specific 0.290 ms
Audio projector, classification task-specific 0.289 ms
Audio projector, text matching task-specific 0.298 ms

Standalone results were measured with ax_run_model --warmup=3 --repeat=20; the text layer models additionally use --group=1. The recorded averages are available in validation/ax_run_model_average.json. Task-switch samples are available in validation/task_switch_client_latency.json and validation/task_switch_internal_latency.json.

Runtime Memory

Measured runtime state CMM used OS memory used
axllm initialized and retrieval/clustering smoke test completed 1438.258 MiB 89.048 MiB PSS

CMM used is the increase in allocated CMM between the stopped-service baseline and the measured running state on the same board. PSS is the proportional set size reported for the axllm process after the smoke test. These are consumed amounts and do not depend on the board's installed CMM capacity.

The audio encoder is initialized lazily and was not loaded in this text smoke-test state. The measurement record is available in validation/runtime_memory.json.

Package Size

Packaged component Size
Shared text backbone, embedding table, and post model 1007.6 MiB
Four packed LoRA task adapters 154.0 MiB total
Vision models 402.1 MiB
Audio models 675.1 MiB
Complete runtime package 2.21 GiB

The four tasks share the same text backbone. Package sizes describe files on storage and are independent of the runtime CMM and OS-memory measurements above.

Functional Validation

Check Result
Retrieval relevant cosine 0.738921
Retrieval irrelevant cosine 0.034047
Clustering within-topic cosine 0.758002
Clustering cross-topic cosine 0.033445
Retrieval round-trip max absolute difference 0.0
Complete web-demo checks 5 / 5 PASS

Machine-readable results are stored under validation/.

Package Layout 

.
├── README.md
├── config.json
├── start_axllm.sh
├── bin/axllm
├── jina_v5_omni_tokenizer.txt
├── tokenizer.json
├── tokenizer_config.json
├── model.embed_tokens.weight.bfloat16.bin
├── jina_embeddings_v5_omni_p256_l0_together.axmodel
├── ...
├── jina_embeddings_v5_omni_p256_l27_together.axmodel
├── jina_embeddings_v5_omni_post.axmodel
├── jina_v5_omni_small_vision_tower_256x256.axmodel
├── jina_v5_omni_small_vision_merger_<task>_256x256.axmodel
├── jina_v5_omni_small_audio_tower_800frames.axmodel
├── jina_v5_omni_small_audio_projector_<task>_800frames.axmodel
├── lora/{retrieval,clustering,classification,text-matching}/
├── scripts/test_retrieval_clustering.py
├── demo/
├── assets/
├── validation/
└── SHA256SUMS

The root directory is the axllm model directory. Keep the relative paths unchanged because config.json resolves the text, media, and adapter files from this layout.

Download the Package 

mkdir -p AXERA-TECH/jina-embeddings-v5-omni-small
cd AXERA-TECH/jina-embeddings-v5-omni-small
hf download AXERA-TECH/jina-embeddings-v5-omni-small --local-dir .

Copy the downloaded directory to the AX650 board before continuing.

Run on the Board

From the package root, start the bundled service on port 18201:

bash ./start_axllm.sh 18201

In another terminal, verify the service and model id:

curl http://127.0.0.1:18201/health
curl http://127.0.0.1:18201/v1/models

A healthy service returns:

{
  "concurrency": 0,
  "max_concurrency": 1,
  "status": "healthy"
}

The model list contains AXERA-TECH/jina-embeddings-v5-omni-small.

Select a Task Adapter

Set task_id in each embedding request:

task_id Intended use Input role
retrieval Cross-modal search, semantic retrieval, and RAG query for queries; document for indexed items
clustering Topic discovery, grouping, and deduplication document
classification Features for zero-shot, few-shot, or downstream classifiers document
text-matching Semantic similarity, paraphrase, and near-duplicate matching document

Adapter files are validated during model initialization. A task switch then loads the selected matrices without repeating the full file validation.

OpenAI-Compatible Text Request 

curl http://127.0.0.1:18201/v1/embeddings \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "AXERA-TECH/jina-embeddings-v5-omni-small",
    "task_id": "retrieval",
    "input_type": "query",
    "input": "a photo of a cat"
  }'

The embedding is returned in data[0].embedding. It contains 1024 floating-point values with an L2 norm of approximately 1.0. For retrieval, use input_type: query for the search query and input_type: document for indexed text, images, or audio.

The packaged smoke test verifies retrieval ranking, clustering separation, and a bit-exact retrieval -> clustering -> retrieval round trip:

python3 scripts/test_retrieval_clustering.py \
  --url http://127.0.0.1:18201/v1/embeddings

Browser Demo

Keep axllm running and start the packaged web application in another terminal:

python3 demo/openai_task_switch_web.py \
  --axllm-url http://127.0.0.1:18201/v1/embeddings \
  --host 0.0.0.0 \
  --port 8080

The web process checks the embedding service before it starts. Open http://<board-ip>:8080 to search the 12 packaged animal images with natural language. The first search creates an in-memory image index; later searches reuse the cached image embeddings and compute only the new text embedding.

Open http://<board-ip>:8080/index.html for the complete capability validation. It exercises all four tasks and then checks a bit-exact switch back to retrieval.

For the query 找出所有猫的图像, all three cat images are ranked first:

cat_2.jpg
1. cat_2.jpg
cosine 0.343559		 cat_1.jpeg
2. cat_1.jpeg
cosine 0.336765		 cat_0.jpeg
3. cat_0.jpeg
cosine 0.296639

The complete log below was captured from the packaged demo on AX650.

Complete AX650 web-demo log 
Jina Embeddings v5 Omni - OpenAI API 多任务切换 Demo
Model: AXERA-TECH/jina-embeddings-v5-omni-small
Assets: cat=3, dog=3, fox=3, rabbit=3

--- retrieval：文本查询召回动物图片 ---

=== task switch: <startup> -> retrieval ===
[request] task=retrieval     input=text:a photo of a  latency=  256.29 ms dim=1024 norm=1.000001
[request] task=retrieval     input=image:cat_0.jpeg   latency=  526.62 ms dim=1024 norm=1.000001
[request] task=retrieval     input=image:cat_1.jpeg   latency=  694.87 ms dim=1024 norm=1.000001
[request] task=retrieval     input=image:cat_2.jpg    latency=  491.13 ms dim=1024 norm=1.000001
[request] task=retrieval     input=image:dog_0.jpeg   latency=  382.88 ms dim=1024 norm=1.000001
[request] task=retrieval     input=image:dog_1.jpg    latency=  406.06 ms dim=1024 norm=1.000001
[request] task=retrieval     input=image:dog_2.jpeg   latency=  823.02 ms dim=1024 norm=1.000001
[request] task=retrieval     input=image:fox_0.jpeg   latency=  400.50 ms dim=1024 norm=1.000001
[request] task=retrieval     input=image:fox_1.jpeg   latency=  358.35 ms dim=1024 norm=1.000001
[request] task=retrieval     input=image:fox_2.jpeg   latency=  368.29 ms dim=1024 norm=1.000001
[request] task=retrieval     input=image:rabbit_0.jpeg latency=  361.31 ms dim=1024 norm=1.000001
[request] task=retrieval     input=image:rabbit_1.jpeg latency=  346.02 ms dim=1024 norm=1.000001
[request] task=retrieval     input=image:rabbit_2.jpeg latency=  345.37 ms dim=1024 norm=1.000002
[result] retrieval ranking:
   1. cat_1.jpeg        cosine=0.376784
   2. cat_2.jpg         cosine=0.374494
   3. cat_0.jpeg        cosine=0.333584
   4. fox_0.jpeg        cosine=0.255053
   5. rabbit_1.jpeg     cosine=0.250108
   6. fox_2.jpeg        cosine=0.245141
   7. fox_1.jpeg        cosine=0.240511
   8. rabbit_2.jpeg     cosine=0.228791
   9. rabbit_0.jpeg     cosine=0.220517
  10. dog_1.jpg         cosine=0.210978
  11. dog_0.jpeg        cosine=0.185020
  12. dog_2.jpeg        cosine=0.156387
[check] cat top-1: PASS

--- clustering：同类与跨类图片聚类 ---

=== task switch: retrieval -> clustering ===
[request] task=clustering    input=image:cat_0.jpeg   latency=  678.72 ms dim=1024 norm=1.000001
[request] task=clustering    input=image:cat_1.jpeg   latency=  703.22 ms dim=1024 norm=1.000001
[request] task=clustering    input=image:cat_2.jpg    latency=  492.12 ms dim=1024 norm=1.000001
[request] task=clustering    input=image:dog_0.jpeg   latency=  386.99 ms dim=1024 norm=1.000001
[request] task=clustering    input=image:dog_1.jpg    latency=  406.84 ms dim=1024 norm=1.000001
[request] task=clustering    input=image:dog_2.jpeg   latency=  828.96 ms dim=1024 norm=1.000001
[request] task=clustering    input=image:fox_0.jpeg   latency=  398.71 ms dim=1024 norm=1.000001
[request] task=clustering    input=image:fox_1.jpeg   latency=  358.83 ms dim=1024 norm=1.000001
[request] task=clustering    input=image:fox_2.jpeg   latency=  368.50 ms dim=1024 norm=1.000001
[request] task=clustering    input=image:rabbit_0.jpeg latency=  363.37 ms dim=1024 norm=1.000001
[request] task=clustering    input=image:rabbit_1.jpeg latency=  345.02 ms dim=1024 norm=1.000001
[request] task=clustering    input=image:rabbit_2.jpeg latency=  345.02 ms dim=1024 norm=1.000001
[result] within-class average cosine=0.722070
[result] cross-class  average cosine=0.504200
[check] clustering separation: PASS

--- classification：动物文本分类特征分离 ---

=== task switch: clustering -> classification ===
[request] task=classification input=text:A cat sleeps  latency=  543.08 ms dim=1024 norm=1.000001
[request] task=classification input=text:A kitten res  latency=  255.83 ms dim=1024 norm=1.000002
[request] task=classification input=text:A dog runs i  latency=  256.54 ms dim=1024 norm=1.000001
[request] task=classification input=text:A puppy play  latency=  256.36 ms dim=1024 norm=1.000001
[request] task=classification input=text:A red fox wa  latency=  257.11 ms dim=1024 norm=1.000001
[request] task=classification input=text:A wild fox h  latency=  257.35 ms dim=1024 norm=1.000001
[request] task=classification input=text:A rabbit eat  latency=  257.31 ms dim=1024 norm=1.000001
[request] task=classification input=text:A bunny sits  latency=  256.40 ms dim=1024 norm=1.000001
[result] cat    same-class cosine=0.892006
[result] dog    same-class cosine=0.883812
[result] fox    same-class cosine=0.890544
[result] rabbit same-class cosine=0.859263
[result] same-class average cosine=0.881406
[result] cross-class average cosine=0.835429
[check] classification separation: PASS

--- text-matching：动物文本匹配 ---

=== task switch: classification -> text-matching ===
[request] task=text-matching input=text:A red fox st  latency=  424.52 ms dim=1024 norm=1.000001
[request] task=text-matching input=text:A domestic c  latency=  255.87 ms dim=1024 norm=1.000001
[request] task=text-matching input=text:A pet dog ru  latency=  257.68 ms dim=1024 norm=1.000001
[request] task=text-matching input=text:A wild red f  latency=  255.85 ms dim=1024 norm=1.000001
[request] task=text-matching input=text:A small rabb  latency=  256.55 ms dim=1024 norm=1.000001
[result] text-matching ranking:
  1. fox    cosine=0.927695
  2. rabbit cosine=0.539868
  3. dog    cosine=0.536886
  4. cat    cosine=0.363692
[check] fox top-1: PASS

--- round-trip：切回 retrieval ---

=== task switch: text-matching -> retrieval ===
[request] task=retrieval     input=text:a photo of a  latency=  378.39 ms dim=1024 norm=1.000001
[request] task=retrieval     input=image:cat_0.jpeg   latency=  555.39 ms dim=1024 norm=1.000001
[result] retrieval query max_abs_diff=0.0000000000
[result] retrieval image max_abs_diff=0.0000000000
[check] retrieval round-trip: PASS

=== summary ===
retrieval       PASS
clustering      PASS
classification  PASS
text-matching   PASS
round-trip      PASS

Fixed Input Profiles

The final embedding output is always [1, 1024] and L2 normalized.

Input Compiled profile
Text 256-token processing chunks; 2047-token runtime limit
Image 256×256
Audio 800 mel frames

The vision and audio task-specific mapper files are selected together with the active adapter. Image resolutions or audio lengths outside these profiles require rebuilding the corresponding media modules.

Adapter selection is mutable state within one service instance. Serialize requests that change task_id, or run separate service instances when requests must be isolated by task. The first switch may include filesystem cache overhead; use the warm measurements above for steady-state planning.

Conversion References

If you need the original model files or want to rebuild the deployment artifacts, start with:

The public conversion workflow requires Pulsar2 7.0 or later. Pulsar2 is needed only to rebuild the deployment artifacts, not to run this downloaded package.

Discussion

Downloads last month
34
Inference Providers NEW
Feature Extraction
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for AXERA-TECH/jina-embeddings-v5-omni-small

Base model

jinaai/jina-embeddings-v5-omni-small
Adapter
(1)
this model