Hugging Face's logo

qualcomm
/
EdgeTAM

Image Segmentation
PyTorch
foundation
android
Model card Files
xet
Community

EdgeTAM: Optimized for Qualcomm Devices

EdgeTAM is a lightweight and efficient version of SAM 2, optimized for on-device tracking and segmenting any object in videos and low memory usage by using a novel 2D Spatial Perceiver and RepViT backbone.

This is based on the implementation of EdgeTAM found here. This repository contains pre-exported model files optimized for Qualcomm® devices. You can use the Qualcomm® AI Hub Models library to export with custom configurations. More details on model performance across various devices, can be found here.

Qualcomm AI Hub Models uses Qualcomm AI Hub Workbench to compile, profile, and evaluate this model. Sign up to run these models on a hosted Qualcomm® device.

Getting Started

There are two ways to deploy this model on your device:

Option 1: Download Pre-Exported Models

Below are pre-exported model assets ready for deployment.

Runtime Precision Chipset SDK Versions Download
ONNX float Universal QAIRT 2.42, ONNX Runtime 1.24.1 Download
ONNX w8a8 Universal QAIRT 2.42, ONNX Runtime 1.24.1 Download
QNN_DLC float Universal QAIRT 2.43 Download
QNN_DLC w8a8 Universal QAIRT 2.43 Download
TFLITE float Universal QAIRT 2.43, TFLite 2.17.0 Download
TFLITE w8a8 Universal QAIRT 2.43, TFLite 2.17.0 Download

For more device-specific assets and performance metrics, visit EdgeTAM on Qualcomm® AI Hub.

Option 2: Export with Custom Configurations

Use the Qualcomm® AI Hub Models Python library to compile and export the model with your own:

  • Custom weights (e.g., fine-tuned checkpoints)
  • Custom input shapes
  • Target device and runtime configurations

This option is ideal if you need to customize the model beyond the default configuration provided here.

See our repository for EdgeTAM on GitHub for usage instructions.

Model Details

Model Type: Model_use_case.semantic_segmentation

Model Stats:

  • Model checkpoint: edgetam.pt
  • Input resolution (Encoder): 1024x1024
  • Number of parameters (EdgeTAMEncoder): 8.3M
  • Model size (EdgeTAMEncoder) (float): 33 MB
  • Number of parameters (EdgeTAMDecoder): 6.22M
  • Model size (EdgeTAMDecoder) (float): 23.7 MB

Performance Summary

Model Runtime Precision Chipset Inference Time (ms) Peak Memory Range (MB) Primary Compute Unit
EdgeTAMDecoder ONNX float Snapdragon® X Elite 6.843 ms 16 - 16 MB NPU
EdgeTAMDecoder ONNX float Snapdragon® 8 Gen 3 Mobile 4.706 ms 17 - 334 MB NPU
EdgeTAMDecoder ONNX float Qualcomm® QCS8550 (Proxy) 6.785 ms 16 - 24 MB NPU
EdgeTAMDecoder ONNX float Qualcomm® QCS9075 8.422 ms 16 - 19 MB NPU
EdgeTAMDecoder ONNX float Snapdragon® 8 Elite For Galaxy Mobile 3.491 ms 2 - 236 MB NPU
EdgeTAMDecoder ONNX float Snapdragon® 8 Elite Gen 5 Mobile 3.15 ms 0 - 228 MB NPU
EdgeTAMDecoder ONNX float Snapdragon® X2 Elite 3.303 ms 15 - 15 MB NPU
EdgeTAMDecoder ONNX w8a8 Snapdragon® X Elite 12.58 ms 4 - 4 MB NPU
EdgeTAMDecoder ONNX w8a8 Snapdragon® 8 Gen 3 Mobile 9.225 ms 4 - 275 MB NPU
EdgeTAMDecoder ONNX w8a8 Qualcomm® QCS6490 127.159 ms 47 - 58 MB CPU
EdgeTAMDecoder ONNX w8a8 Qualcomm® QCS8550 (Proxy) 12.466 ms 2 - 10 MB NPU
EdgeTAMDecoder ONNX w8a8 Qualcomm® QCS9075 14.219 ms 4 - 7 MB NPU
EdgeTAMDecoder ONNX w8a8 Qualcomm® QCM6690 58.694 ms 44 - 66 MB CPU
EdgeTAMDecoder ONNX w8a8 Snapdragon® 8 Elite For Galaxy Mobile 6.716 ms 2 - 243 MB NPU
EdgeTAMDecoder ONNX w8a8 Snapdragon® 7 Gen 4 Mobile 51.163 ms 50 - 68 MB CPU
EdgeTAMDecoder ONNX w8a8 Snapdragon® 8 Elite Gen 5 Mobile 5.307 ms 4 - 271 MB NPU
EdgeTAMDecoder ONNX w8a8 Snapdragon® X2 Elite 5.449 ms 4 - 4 MB NPU
EdgeTAMDecoder QNN_DLC float Snapdragon® X Elite 5.462 ms 16 - 16 MB NPU
EdgeTAMDecoder QNN_DLC float Snapdragon® 8 Gen 3 Mobile 3.6 ms 3 - 234 MB NPU
EdgeTAMDecoder QNN_DLC float Qualcomm® QCS8275 (Proxy) 12.442 ms 16 - 281 MB NPU
EdgeTAMDecoder QNN_DLC float Qualcomm® QCS8550 (Proxy) 5.255 ms 16 - 45 MB NPU
EdgeTAMDecoder QNN_DLC float Qualcomm® SA8775P 6.494 ms 16 - 217 MB NPU
EdgeTAMDecoder QNN_DLC float Qualcomm® QCS9075 6.873 ms 18 - 36 MB NPU
EdgeTAMDecoder QNN_DLC float Qualcomm® QCS8450 (Proxy) 8.859 ms 4 - 226 MB NPU
EdgeTAMDecoder QNN_DLC float Qualcomm® SA7255P 12.442 ms 16 - 281 MB NPU
EdgeTAMDecoder QNN_DLC float Qualcomm® SA8295P 8.061 ms 0 - 215 MB NPU
EdgeTAMDecoder QNN_DLC float Snapdragon® 8 Elite For Galaxy Mobile 2.904 ms 4 - 213 MB NPU
EdgeTAMDecoder QNN_DLC float Snapdragon® 8 Elite Gen 5 Mobile 2.53 ms 16 - 214 MB NPU
EdgeTAMDecoder QNN_DLC float Snapdragon® X2 Elite 2.999 ms 16 - 16 MB NPU
EdgeTAMDecoder QNN_DLC w8a8 Snapdragon® X Elite 2.253 ms 4 - 4 MB NPU
EdgeTAMDecoder QNN_DLC w8a8 Snapdragon® 8 Gen 3 Mobile 1.298 ms 0 - 102 MB NPU
EdgeTAMDecoder QNN_DLC w8a8 Qualcomm® QCS8275 (Proxy) 3.866 ms 4 - 83 MB NPU
EdgeTAMDecoder QNN_DLC w8a8 Qualcomm® QCS8550 (Proxy) 1.912 ms 0 - 31 MB NPU
EdgeTAMDecoder QNN_DLC w8a8 Qualcomm® SA8775P 9.278 ms 4 - 83 MB NPU
EdgeTAMDecoder QNN_DLC w8a8 Qualcomm® QCS9075 2.475 ms 1 - 7 MB NPU
EdgeTAMDecoder QNN_DLC w8a8 Qualcomm® QCM6690 9.968 ms 4 - 264 MB NPU
EdgeTAMDecoder QNN_DLC w8a8 Qualcomm® QCS8450 (Proxy) 2.65 ms 0 - 102 MB NPU
EdgeTAMDecoder QNN_DLC w8a8 Qualcomm® SA7255P 3.866 ms 4 - 83 MB NPU
EdgeTAMDecoder QNN_DLC w8a8 Qualcomm® SA8295P 2.964 ms 4 - 97 MB NPU
EdgeTAMDecoder QNN_DLC w8a8 Snapdragon® 8 Elite For Galaxy Mobile 1.035 ms 1 - 83 MB NPU
EdgeTAMDecoder QNN_DLC w8a8 Snapdragon® 7 Gen 4 Mobile 2.62 ms 4 - 208 MB NPU
EdgeTAMDecoder QNN_DLC w8a8 Snapdragon® 8 Elite Gen 5 Mobile 0.894 ms 1 - 106 MB NPU
EdgeTAMDecoder QNN_DLC w8a8 Snapdragon® X2 Elite 1.367 ms 4 - 4 MB NPU
EdgeTAMDecoder TFLITE float Snapdragon® 8 Gen 3 Mobile 3.705 ms 0 - 244 MB NPU
EdgeTAMDecoder TFLITE float Qualcomm® QCS8275 (Proxy) 12.818 ms 0 - 210 MB NPU
EdgeTAMDecoder TFLITE float Qualcomm® QCS8550 (Proxy) 5.359 ms 0 - 7 MB NPU
EdgeTAMDecoder TFLITE float Qualcomm® SA8775P 6.772 ms 0 - 211 MB NPU
EdgeTAMDecoder TFLITE float Qualcomm® QCS9075 6.998 ms 0 - 30 MB NPU
EdgeTAMDecoder TFLITE float Qualcomm® QCS8450 (Proxy) 8.896 ms 0 - 229 MB NPU
EdgeTAMDecoder TFLITE float Qualcomm® SA7255P 12.818 ms 0 - 210 MB NPU
EdgeTAMDecoder TFLITE float Qualcomm® SA8295P 8.043 ms 0 - 222 MB NPU
EdgeTAMDecoder TFLITE float Snapdragon® 8 Elite For Galaxy Mobile 2.899 ms 0 - 218 MB NPU
EdgeTAMDecoder TFLITE float Snapdragon® 8 Elite Gen 5 Mobile 2.542 ms 0 - 206 MB NPU
EdgeTAMDecoder TFLITE w8a8 Snapdragon® 8 Gen 3 Mobile 9.195 ms 0 - 261 MB NPU
EdgeTAMDecoder TFLITE w8a8 Qualcomm® QCS8275 (Proxy) 20.937 ms 0 - 212 MB NPU
EdgeTAMDecoder TFLITE w8a8 Qualcomm® QCS8550 (Proxy) 12.191 ms 0 - 2 MB NPU
EdgeTAMDecoder TFLITE w8a8 Qualcomm® SA8775P 51.678 ms 0 - 213 MB NPU
EdgeTAMDecoder TFLITE w8a8 Qualcomm® QCS9075 13.187 ms 0 - 14 MB NPU
EdgeTAMDecoder TFLITE w8a8 Qualcomm® QCM6690 51.184 ms 12 - 205 MB NPU
EdgeTAMDecoder TFLITE w8a8 Qualcomm® QCS8450 (Proxy) 15.573 ms 0 - 239 MB NPU
EdgeTAMDecoder TFLITE w8a8 Qualcomm® SA7255P 20.937 ms 0 - 212 MB NPU
EdgeTAMDecoder TFLITE w8a8 Qualcomm® SA8295P 14.243 ms 0 - 223 MB NPU
EdgeTAMDecoder TFLITE w8a8 Snapdragon® 8 Elite For Galaxy Mobile 6.872 ms 0 - 225 MB NPU
EdgeTAMDecoder TFLITE w8a8 Snapdragon® 7 Gen 4 Mobile 16.171 ms 12 - 208 MB NPU
EdgeTAMDecoder TFLITE w8a8 Snapdragon® 8 Elite Gen 5 Mobile 5.415 ms 0 - 239 MB NPU
EdgeTAMEncoder ONNX float Snapdragon® X Elite 20.284 ms 34 - 34 MB NPU
EdgeTAMEncoder ONNX float Snapdragon® 8 Gen 3 Mobile 14.878 ms 45 - 396 MB NPU
EdgeTAMEncoder ONNX float Qualcomm® QCS8550 (Proxy) 20.231 ms 30 - 47 MB NPU
EdgeTAMEncoder ONNX float Qualcomm® QCS9075 28.941 ms 12 - 27 MB NPU
EdgeTAMEncoder ONNX float Snapdragon® 8 Elite For Galaxy Mobile 9.871 ms 0 - 264 MB NPU
EdgeTAMEncoder ONNX float Snapdragon® 8 Elite Gen 5 Mobile 7.757 ms 45 - 328 MB NPU
EdgeTAMEncoder ONNX float Snapdragon® X2 Elite 8.844 ms 32 - 32 MB NPU
EdgeTAMEncoder ONNX w8a8 Snapdragon® X Elite 8.85 ms 16 - 16 MB NPU
EdgeTAMEncoder ONNX w8a8 Snapdragon® 8 Gen 3 Mobile 5.462 ms 13 - 321 MB NPU
EdgeTAMEncoder ONNX w8a8 Qualcomm® QCS6490 488.296 ms 203 - 213 MB CPU
EdgeTAMEncoder ONNX w8a8 Qualcomm® QCS8550 (Proxy) 8.221 ms 11 - 17 MB NPU
EdgeTAMEncoder ONNX w8a8 Qualcomm® QCS9075 9.524 ms 12 - 15 MB NPU
EdgeTAMEncoder ONNX w8a8 Qualcomm® QCM6690 323.926 ms 141 - 152 MB CPU
EdgeTAMEncoder ONNX w8a8 Snapdragon® 8 Elite For Galaxy Mobile 4.476 ms 10 - 273 MB NPU
EdgeTAMEncoder ONNX w8a8 Snapdragon® 7 Gen 4 Mobile 308.903 ms 207 - 219 MB CPU
EdgeTAMEncoder ONNX w8a8 Snapdragon® 8 Elite Gen 5 Mobile 3.303 ms 0 - 275 MB NPU
EdgeTAMEncoder ONNX w8a8 Snapdragon® X2 Elite 3.491 ms 18 - 18 MB NPU
EdgeTAMEncoder QNN_DLC float Snapdragon® X Elite 24.17 ms 12 - 12 MB NPU
EdgeTAMEncoder QNN_DLC float Snapdragon® 8 Gen 3 Mobile 16.948 ms 12 - 368 MB NPU
EdgeTAMEncoder QNN_DLC float Qualcomm® QCS8275 (Proxy) 69.756 ms 1 - 261 MB NPU
EdgeTAMEncoder QNN_DLC float Qualcomm® QCS8550 (Proxy) 23.673 ms 12 - 14 MB NPU
EdgeTAMEncoder QNN_DLC float Qualcomm® SA8775P 28.167 ms 0 - 261 MB NPU
EdgeTAMEncoder QNN_DLC float Qualcomm® QCS9075 34.311 ms 12 - 42 MB NPU
EdgeTAMEncoder QNN_DLC float Qualcomm® QCS8450 (Proxy) 98.559 ms 12 - 466 MB NPU
EdgeTAMEncoder QNN_DLC float Qualcomm® SA7255P 69.756 ms 1 - 261 MB NPU
EdgeTAMEncoder QNN_DLC float Qualcomm® SA8295P 57.556 ms 0 - 358 MB NPU
EdgeTAMEncoder QNN_DLC float Snapdragon® 8 Elite For Galaxy Mobile 11.891 ms 12 - 290 MB NPU
EdgeTAMEncoder QNN_DLC float Snapdragon® 8 Elite Gen 5 Mobile 8.637 ms 12 - 308 MB NPU
EdgeTAMEncoder QNN_DLC float Snapdragon® X2 Elite 10.279 ms 12 - 12 MB NPU
EdgeTAMEncoder QNN_DLC w8a8 Snapdragon® X Elite 12.205 ms 3 - 3 MB NPU
EdgeTAMEncoder QNN_DLC w8a8 Snapdragon® 8 Gen 3 Mobile 7.566 ms 3 - 317 MB NPU
EdgeTAMEncoder QNN_DLC w8a8 Qualcomm® QCS8275 (Proxy) 20.524 ms 3 - 266 MB NPU
EdgeTAMEncoder QNN_DLC w8a8 Qualcomm® QCS8550 (Proxy) 11.404 ms 3 - 160 MB NPU
EdgeTAMEncoder QNN_DLC w8a8 Qualcomm® SA8775P 12.088 ms 3 - 266 MB NPU
EdgeTAMEncoder QNN_DLC w8a8 Qualcomm® QCS9075 15.141 ms 3 - 12 MB NPU
EdgeTAMEncoder QNN_DLC w8a8 Qualcomm® QCM6690 59.198 ms 3 - 269 MB NPU
EdgeTAMEncoder QNN_DLC w8a8 Qualcomm® QCS8450 (Proxy) 1963.093 ms 3 - 209 MB NPU
EdgeTAMEncoder QNN_DLC w8a8 Qualcomm® SA7255P 20.524 ms 3 - 266 MB NPU
EdgeTAMEncoder QNN_DLC w8a8 Qualcomm® SA8295P 2138.343 ms 3 - 161 MB NPU
EdgeTAMEncoder QNN_DLC w8a8 Snapdragon® 8 Elite For Galaxy Mobile 5.362 ms 0 - 255 MB NPU
EdgeTAMEncoder QNN_DLC w8a8 Snapdragon® 7 Gen 4 Mobile 12.383 ms 3 - 301 MB NPU
EdgeTAMEncoder QNN_DLC w8a8 Snapdragon® 8 Elite Gen 5 Mobile 4.229 ms 3 - 269 MB NPU
EdgeTAMEncoder QNN_DLC w8a8 Snapdragon® X2 Elite 4.859 ms 3 - 3 MB NPU
EdgeTAMEncoder TFLITE float Snapdragon® 8 Gen 3 Mobile 16.568 ms 15 - 372 MB NPU
EdgeTAMEncoder TFLITE float Qualcomm® QCS8275 (Proxy) 68.178 ms 16 - 283 MB NPU
EdgeTAMEncoder TFLITE float Qualcomm® QCS8550 (Proxy) 22.855 ms 16 - 18 MB NPU
EdgeTAMEncoder TFLITE float Qualcomm® SA8775P 27.351 ms 16 - 280 MB NPU
EdgeTAMEncoder TFLITE float Qualcomm® QCS9075 32.802 ms 0 - 42 MB NPU
EdgeTAMEncoder TFLITE float Qualcomm® QCS8450 (Proxy) 99.118 ms 16 - 460 MB NPU
EdgeTAMEncoder TFLITE float Qualcomm® SA7255P 68.178 ms 16 - 283 MB NPU
EdgeTAMEncoder TFLITE float Qualcomm® SA8295P 54.228 ms 16 - 366 MB NPU
EdgeTAMEncoder TFLITE float Snapdragon® 8 Elite For Galaxy Mobile 11.455 ms 14 - 295 MB NPU
EdgeTAMEncoder TFLITE float Snapdragon® 8 Elite Gen 5 Mobile 8.409 ms 16 - 318 MB NPU
EdgeTAMEncoder TFLITE w8a8 Snapdragon® 8 Gen 3 Mobile 4.806 ms 4 - 313 MB NPU
EdgeTAMEncoder TFLITE w8a8 Qualcomm® QCS8275 (Proxy) 13.791 ms 4 - 258 MB NPU
EdgeTAMEncoder TFLITE w8a8 Qualcomm® QCS8550 (Proxy) 7.021 ms 4 - 198 MB NPU
EdgeTAMEncoder TFLITE w8a8 Qualcomm® SA8775P 7.862 ms 4 - 259 MB NPU
EdgeTAMEncoder TFLITE w8a8 Qualcomm® QCS9075 7.867 ms 4 - 19 MB NPU
EdgeTAMEncoder TFLITE w8a8 Qualcomm® QCM6690 70.404 ms 4 - 237 MB NPU
EdgeTAMEncoder TFLITE w8a8 Qualcomm® QCS8450 (Proxy) 2256.068 ms 0 - 330 MB NPU
EdgeTAMEncoder TFLITE w8a8 Qualcomm® SA7255P 13.791 ms 4 - 258 MB NPU
EdgeTAMEncoder TFLITE w8a8 Qualcomm® SA8295P 2221.179 ms 0 - 277 MB NPU
EdgeTAMEncoder TFLITE w8a8 Snapdragon® 8 Elite For Galaxy Mobile 2.996 ms 2 - 250 MB NPU
EdgeTAMEncoder TFLITE w8a8 Snapdragon® 7 Gen 4 Mobile 8.263 ms 3 - 262 MB NPU
EdgeTAMEncoder TFLITE w8a8 Snapdragon® 8 Elite Gen 5 Mobile 2.662 ms 4 - 262 MB NPU

License

  • The license for the original implementation of EdgeTAM can be found here.

References

Community

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
Image Segmentation
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Paper for qualcomm/EdgeTAM