Hugging Face's logo

qualcomm
/
Segment-Anything-Model-2

Image Segmentation
PyTorch
foundation
android
Model card Files
xet
 Community

Segment-Anything-Model-2: Optimized for Qualcomm Devices

SAM 2, the successor to Meta's Segment Anything Model (SAM), is a cutting-edge tool designed for comprehensive object segmentation in both images and videos. It excels in handling complex visual data through a unified, promptable model architecture that supports real-time processing and zero-shot generalization.

This is based on the implementation of Segment-Anything-Model-2 found here. This repository contains pre-exported model files optimized for Qualcomm® devices. You can use the Qualcomm® AI Hub Models library to export with custom configurations. More details on model performance across various devices, can be found here.

Qualcomm AI Hub Models uses Qualcomm AI Hub Workbench to compile, profile, and evaluate this model. Sign up to run these models on a hosted Qualcomm® device.

Getting Started

There are two ways to deploy this model on your device:

Option 1: Download Pre-Exported Models

Below are pre-exported model assets ready for deployment.

Runtime Precision Chipset SDK Versions Download
ONNX float Universal QAIRT 2.45, ONNX Runtime 1.25.0 Download
ONNX w8a8 Universal QAIRT 2.45, ONNX Runtime 1.25.0 Download
QNN_DLC float Universal QAIRT 2.45 Download
TFLITE w8a8 Universal QAIRT 2.45 Download

For more device-specific assets and performance metrics, visit Segment-Anything-Model-2 on Qualcomm® AI Hub.

Option 2: Export with Custom Configurations

Use the Qualcomm® AI Hub Models Python library to compile and export the model with your own:

  • Custom weights (e.g., fine-tuned checkpoints)
  • Custom input shapes
  • Target device and runtime configurations

This option is ideal if you need to customize the model beyond the default configuration provided here.

See our repository for Segment-Anything-Model-2 on GitHub for usage instructions.

Model Details

Model Type: Model_use_case.semantic_segmentation

Model Stats:

  • Model checkpoint: sam2.1_hiera_t
  • Input resolution: 720p (720x1280)
  • Number of parameters (encoder): 33.5M
  • Model size (encoder) (float): 128 MB
  • Number of parameters (decoder): 6.22M
  • Model size (decoder) (float): 23.7 MB

Performance Summary

Model Runtime Precision Chipset Inference Time (ms) Peak Memory Range (MB) Primary Compute Unit
decoder ONNX float Snapdragon® X2 Elite 3.313 ms 197 - 197 MB NPU
decoder ONNX float Snapdragon® 8 Gen 3 Mobile 4.756 ms 2 - 243 MB NPU
decoder ONNX float Qualcomm® QCS8550 (Proxy) 6.758 ms 16 - 20 MB NPU
decoder ONNX float Snapdragon® 8 Elite Mobile 3.779 ms 3 - 222 MB NPU
decoder ONNX float Snapdragon® 8 Elite Gen 5 Mobile 3.198 ms 2 - 219 MB NPU
decoder ONNX float Qualcomm® QCS9075 8.242 ms 16 - 61 MB NPU
decoder ONNX float Qualcomm® QCS8750 3.779 ms 3 - 222 MB NPU
decoder ONNX w8a8 Snapdragon® X2 Elite 1.328 ms 209 - 209 MB NPU
decoder ONNX w8a8 Snapdragon® X Elite 2.764 ms 178 - 178 MB NPU
decoder ONNX w8a8 Snapdragon® 8 Gen 3 Mobile 1.745 ms 0 - 144 MB NPU
decoder ONNX w8a8 Snapdragon® 8 Gen 1 Mobile 3.493 ms 4 - 156 MB NPU
decoder ONNX w8a8 Qualcomm® QCS6490 11.004 ms 4 - 49 MB NPU
decoder ONNX w8a8 Qualcomm® QCS8550 (Proxy) 2.642 ms 4 - 8 MB NPU
decoder ONNX w8a8 Qualcomm® QCS8450 3.493 ms 4 - 156 MB NPU
decoder ONNX w8a8 Snapdragon® 8 Elite Gen 5 Mobile 1.253 ms 0 - 155 MB NPU
decoder ONNX w8a8 Snapdragon® 7 Gen 4 Mobile 3.365 ms 4 - 296 MB NPU
decoder ONNX w8a8 Qualcomm® QCM6690 16.61 ms 2 - 249 MB NPU
decoder ONNX w8a8 Qualcomm® QCS9075 3.267 ms 1 - 49 MB NPU
decoder ONNX w8a8 Snapdragon® 8 Elite Mobile 1.423 ms 0 - 131 MB NPU
decoder ONNX w8a8 Qualcomm® QCS7790 3.365 ms 4 - 296 MB NPU
decoder ONNX w8a8 Qualcomm® QCS8750 1.423 ms 0 - 131 MB NPU
decoder ONNX w8a8 Qualcomm® QCS7181 2.764 ms 178 - 178 MB NPU
decoder QNN_DLC float Snapdragon® X2 Elite 3.062 ms 16 - 16 MB NPU
decoder QNN_DLC float Snapdragon® 8 Gen 3 Mobile 3.714 ms 4 - 216 MB NPU
decoder QNN_DLC float Snapdragon® 8 Gen 1 Mobile 10.035 ms 4 - 218 MB NPU
decoder QNN_DLC float Qualcomm® QCS8550 (Proxy) 5.378 ms 16 - 20 MB NPU
decoder QNN_DLC float Qualcomm® QCS8450 10.035 ms 4 - 218 MB NPU
decoder QNN_DLC float Snapdragon® 8 Elite Mobile 2.932 ms 0 - 186 MB NPU
decoder QNN_DLC float Qualcomm® SA8295P 8.23 ms 0 - 195 MB NPU
decoder QNN_DLC float Snapdragon® 8 Elite Gen 5 Mobile 2.584 ms 4 - 192 MB NPU
decoder QNN_DLC float Qualcomm® QCS9075 6.682 ms 16 - 34 MB NPU
decoder QNN_DLC float Qualcomm® QCS8750 2.932 ms 0 - 186 MB NPU
decoder QNN_DLC w8a8 Snapdragon® X2 Elite 1.738 ms 4 - 4 MB NPU
decoder QNN_DLC w8a8 Snapdragon® X Elite 2.29 ms 4 - 4 MB NPU
decoder QNN_DLC w8a8 Snapdragon® 8 Gen 3 Mobile 1.29 ms 0 - 108 MB NPU
decoder QNN_DLC w8a8 Snapdragon® 8 Gen 1 Mobile 2.562 ms 0 - 109 MB NPU
decoder QNN_DLC w8a8 Qualcomm® QCS6490 8.279 ms 1 - 7 MB NPU
decoder QNN_DLC w8a8 Qualcomm® QCS8550 (Proxy) 1.987 ms 4 - 25 MB NPU
decoder QNN_DLC w8a8 Qualcomm® QCS8450 2.562 ms 0 - 109 MB NPU
decoder QNN_DLC w8a8 Snapdragon® 8 Elite Gen 5 Mobile 0.909 ms 1 - 109 MB NPU
decoder QNN_DLC w8a8 Snapdragon® 7 Gen 4 Mobile 2.671 ms 4 - 216 MB NPU
decoder QNN_DLC w8a8 Qualcomm® QCM6690 9.965 ms 4 - 273 MB NPU
decoder QNN_DLC w8a8 Qualcomm® QCS9075 2.42 ms 1 - 7 MB NPU
decoder QNN_DLC w8a8 Snapdragon® 8 Elite Mobile 1.011 ms 1 - 92 MB NPU
decoder QNN_DLC w8a8 Qualcomm® SA8295P 2.971 ms 4 - 103 MB NPU
decoder QNN_DLC w8a8 Qualcomm® QCS7790 2.671 ms 4 - 216 MB NPU
decoder QNN_DLC w8a8 Qualcomm® QCS8750 1.011 ms 1 - 92 MB NPU
decoder QNN_DLC w8a8 Qualcomm® QCS7181 2.29 ms 4 - 4 MB NPU
decoder TFLITE float Snapdragon® 8 Gen 3 Mobile 3.72 ms 0 - 213 MB NPU
decoder TFLITE float Qualcomm® QCS8550 (Proxy) 5.328 ms 0 - 8 MB NPU
decoder TFLITE float Qualcomm® SA8775P 42.72 ms 0 - 33 MB GPU
decoder TFLITE float Qualcomm® SA8650P 42.72 ms 0 - 33 MB GPU
decoder TFLITE float Qualcomm® SA8255P 42.72 ms 0 - 33 MB GPU
decoder TFLITE float Qualcomm® SA8295P 8.211 ms 0 - 196 MB NPU
decoder TFLITE float Snapdragon® 8 Elite Gen 5 Mobile 2.625 ms 0 - 182 MB NPU
decoder TFLITE float Qualcomm® QCS9075 6.904 ms 0 - 30 MB NPU
decoder TFLITE w8a8 Snapdragon® 8 Gen 3 Mobile 5.77 ms 0 - 268 MB NPU
decoder TFLITE w8a8 Snapdragon® 8 Gen 1 Mobile 10.205 ms 0 - 246 MB NPU
decoder TFLITE w8a8 Qualcomm® QCS6490 33.535 ms 12 - 58 MB NPU
decoder TFLITE w8a8 Qualcomm® QCS8275 13.534 ms 0 - 237 MB NPU
decoder TFLITE w8a8 Qualcomm® QCS8550 (Proxy) 7.744 ms 0 - 22 MB NPU
decoder TFLITE w8a8 Qualcomm® SA8775P 41.987 ms 17 - 52 MB GPU
decoder TFLITE w8a8 Qualcomm® SA8650P 41.987 ms 17 - 52 MB GPU
decoder TFLITE w8a8 Qualcomm® SA8255P 41.987 ms 17 - 52 MB GPU
decoder TFLITE w8a8 Qualcomm® QCS8450 10.205 ms 0 - 246 MB NPU
decoder TFLITE w8a8 Snapdragon® 8 Elite Gen 5 Mobile 3.107 ms 0 - 260 MB NPU
decoder TFLITE w8a8 Snapdragon® 7 Gen 4 Mobile 14.815 ms 12 - 201 MB NPU
decoder TFLITE w8a8 Qualcomm® QCM6690 32.284 ms 12 - 211 MB NPU
decoder TFLITE w8a8 Qualcomm® QCS9075 8.536 ms 0 - 14 MB NPU
decoder TFLITE w8a8 Qualcomm® SA7255P 13.534 ms 0 - 237 MB NPU
decoder TFLITE w8a8 Snapdragon® 8 Elite Mobile 3.516 ms 0 - 239 MB NPU
decoder TFLITE w8a8 Qualcomm® SA8295P 9.456 ms 0 - 239 MB NPU
decoder TFLITE w8a8 Qualcomm® QCS7790 14.815 ms 12 - 201 MB NPU
decoder TFLITE w8a8 Qualcomm® QCS8750 3.516 ms 0 - 239 MB NPU
encoder ONNX float Snapdragon® X2 Elite 86.407 ms 202 - 202 MB NPU
encoder ONNX float Snapdragon® 8 Gen 3 Mobile 126.384 ms 58 - 2157 MB NPU
encoder ONNX float Qualcomm® QCS8550 (Proxy) 168.886 ms 8 - 95 MB NPU
encoder ONNX float Snapdragon® 8 Elite Mobile 98.542 ms 44 - 1798 MB NPU
encoder ONNX float Snapdragon® 8 Elite Gen 5 Mobile 78.81 ms 15 - 1681 MB NPU
encoder ONNX float Qualcomm® QCS9075 226.812 ms 12 - 69 MB NPU
encoder ONNX float Qualcomm® QCS8750 98.542 ms 44 - 1798 MB NPU
encoder ONNX w8a8 Snapdragon® X2 Elite 35.714 ms 211 - 211 MB NPU
encoder ONNX w8a8 Snapdragon® X Elite 91.637 ms 179 - 179 MB NPU
encoder ONNX w8a8 Snapdragon® 8 Gen 3 Mobile 64.07 ms 14 - 1933 MB NPU
encoder ONNX w8a8 Snapdragon® 8 Gen 1 Mobile 123.485 ms 14 - 2449 MB NPU
encoder ONNX w8a8 Qualcomm® QCS6490 609.502 ms 9 - 54 MB NPU
encoder ONNX w8a8 Qualcomm® QCS8550 (Proxy) 88.496 ms 12 - 17 MB NPU
encoder ONNX w8a8 Qualcomm® QCS8450 123.485 ms 14 - 2449 MB NPU
encoder ONNX w8a8 Snapdragon® 8 Elite Gen 5 Mobile 33.968 ms 11 - 1675 MB NPU
encoder ONNX w8a8 Snapdragon® 7 Gen 4 Mobile 90.665 ms 4 - 2343 MB NPU
encoder ONNX w8a8 Qualcomm® QCM6690 1710.399 ms 3 - 1930 MB NPU
encoder ONNX w8a8 Qualcomm® QCS9075 93.189 ms 15 - 60 MB NPU
encoder ONNX w8a8 Snapdragon® 8 Elite Mobile 39.187 ms 11 - 2121 MB NPU
encoder ONNX w8a8 Qualcomm® QCS7790 90.665 ms 4 - 2343 MB NPU
encoder ONNX w8a8 Qualcomm® QCS8750 39.187 ms 11 - 2121 MB NPU
encoder ONNX w8a8 Qualcomm® QCS7181 91.637 ms 179 - 179 MB NPU
encoder QNN_DLC float Snapdragon® X2 Elite 86.374 ms 12 - 12 MB NPU
encoder QNN_DLC float Snapdragon® 8 Gen 3 Mobile 128.059 ms 12 - 2146 MB NPU
encoder QNN_DLC float Snapdragon® 8 Gen 1 Mobile 342.121 ms 12 - 1862 MB NPU
encoder QNN_DLC float Qualcomm® QCS8550 (Proxy) 179.716 ms 9 - 16 MB NPU
encoder QNN_DLC float Qualcomm® QCS8450 342.121 ms 12 - 1862 MB NPU
encoder QNN_DLC float Snapdragon® 8 Elite Mobile 101.744 ms 12 - 2522 MB NPU
encoder QNN_DLC float Qualcomm® SA8295P 282.43 ms 0 - 1581 MB NPU
encoder QNN_DLC float Snapdragon® 8 Elite Gen 5 Mobile 78.185 ms 12 - 1718 MB NPU
encoder QNN_DLC float Qualcomm® QCS9075 218.985 ms 12 - 46 MB NPU
encoder QNN_DLC float Qualcomm® QCS8750 101.744 ms 12 - 2522 MB NPU
encoder QNN_DLC w8a8 Snapdragon® X2 Elite 36.008 ms 3 - 3 MB NPU
encoder QNN_DLC w8a8 Snapdragon® X Elite 82.781 ms 3 - 3 MB NPU
encoder QNN_DLC w8a8 Snapdragon® 8 Gen 3 Mobile 54.397 ms 3 - 2370 MB NPU
encoder QNN_DLC w8a8 Snapdragon® 8 Gen 1 Mobile 139.076 ms 3 - 2385 MB NPU
encoder QNN_DLC w8a8 Qualcomm® QCS6490 613.773 ms 3 - 13 MB NPU
encoder QNN_DLC w8a8 Qualcomm® QCS8550 (Proxy) 79.296 ms 3 - 666 MB NPU
encoder QNN_DLC w8a8 Qualcomm® QCS8450 139.076 ms 3 - 2385 MB NPU
encoder QNN_DLC w8a8 Snapdragon® 8 Elite Gen 5 Mobile 35.326 ms 3 - 1662 MB NPU
encoder QNN_DLC w8a8 Snapdragon® 7 Gen 4 Mobile 94.722 ms 3 - 2357 MB NPU
encoder QNN_DLC w8a8 Qualcomm® QCM6690 1700.608 ms 3 - 1919 MB NPU
encoder QNN_DLC w8a8 Qualcomm® QCS9075 95.786 ms 4 - 13 MB NPU
encoder QNN_DLC w8a8 Snapdragon® 8 Elite Mobile 39.575 ms 3 - 1621 MB NPU
encoder QNN_DLC w8a8 Qualcomm® SA8295P 95.708 ms 0 - 1605 MB NPU
encoder QNN_DLC w8a8 Qualcomm® QCS7790 94.722 ms 3 - 2357 MB NPU
encoder QNN_DLC w8a8 Qualcomm® QCS8750 39.575 ms 3 - 1621 MB NPU
encoder QNN_DLC w8a8 Qualcomm® QCS7181 82.781 ms 3 - 3 MB NPU
encoder TFLITE float Snapdragon® 8 Gen 3 Mobile 129.64 ms 20 - 2227 MB NPU
encoder TFLITE float Qualcomm® QCS8550 (Proxy) 180.195 ms 20 - 23 MB NPU
encoder TFLITE float Qualcomm® SA8775P 2416.888 ms 39 - 65 MB CPU
encoder TFLITE float Qualcomm® SA8650P 2416.888 ms 39 - 65 MB CPU
encoder TFLITE float Qualcomm® SA8255P 2416.888 ms 39 - 65 MB CPU
encoder TFLITE float Qualcomm® SA8295P 282.733 ms 20 - 1665 MB NPU
encoder TFLITE float Snapdragon® 8 Elite Gen 5 Mobile 76.301 ms 20 - 1776 MB NPU
encoder TFLITE float Qualcomm® QCS9075 218.941 ms 0 - 114 MB NPU
encoder TFLITE w8a8 Snapdragon® 8 Gen 3 Mobile 69.462 ms 5 - 2221 MB NPU
encoder TFLITE w8a8 Snapdragon® 8 Gen 1 Mobile 186.937 ms 5 - 2293 MB NPU
encoder TFLITE w8a8 Qualcomm® QCS6490 1724.645 ms 76 - 338 MB CPU
encoder TFLITE w8a8 Qualcomm® QCS8275 162.696 ms 6 - 1874 MB NPU
encoder TFLITE w8a8 Qualcomm® QCS8550 (Proxy) 100.759 ms 5 - 10 MB NPU
encoder TFLITE w8a8 Qualcomm® SA8775P 1227.827 ms 78 - 104 MB CPU
encoder TFLITE w8a8 Qualcomm® SA8650P 1227.827 ms 78 - 104 MB CPU
encoder TFLITE w8a8 Qualcomm® SA8255P 1227.827 ms 78 - 104 MB CPU
encoder TFLITE w8a8 Qualcomm® QCS8450 186.937 ms 5 - 2293 MB NPU
encoder TFLITE w8a8 Snapdragon® 8 Elite Gen 5 Mobile 34.521 ms 4 - 1787 MB NPU
encoder TFLITE w8a8 Snapdragon® 7 Gen 4 Mobile 1131.84 ms 78 - 311 MB CPU
encoder TFLITE w8a8 Qualcomm® QCM6690 2361.942 ms 32 - 265 MB CPU
encoder TFLITE w8a8 Qualcomm® QCS9075 108.995 ms 4 - 62 MB NPU
encoder TFLITE w8a8 Qualcomm® SA7255P 162.696 ms 6 - 1874 MB NPU
encoder TFLITE w8a8 Snapdragon® 8 Elite Mobile 55.081 ms 5 - 1793 MB NPU
encoder TFLITE w8a8 Qualcomm® SA8295P 117.632 ms 5 - 1883 MB NPU
encoder TFLITE w8a8 Qualcomm® QCS7790 1131.84 ms 78 - 311 MB CPU
encoder TFLITE w8a8 Qualcomm® QCS8750 55.081 ms 5 - 1793 MB NPU

License

  • The license for the original implementation of Segment-Anything-Model-2 can be found here.

References

Community

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
Image Segmentation
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Paper for qualcomm/Segment-Anything-Model-2