software-mansion/react-native-executorch-multi-qa-mpnet-base-dot-v1

multi-qa-mpnet-base-dot-v1

This repository hosts the multi-qa-mpnet-base-dot-v1 models exported for the React Native ExecuTorch library as ExecuTorch .pte programs, ready to run on device.

Upstream model: multi-qa-mpnet-base-dot-v1

Variants

Path	Backend	Precision
vulkan/multi_qa_mpnet_base_dot_v1_vulkan_fp16.pte	vulkan	fp16
vulkan/multi_qa_mpnet_base_dot_v1_vulkan_int8.pte	vulkan	int8
xnnpack/multi_qa_mpnet_base_dot_v1_xnnpack_8da4w.pte	xnnpack	8da4w
xnnpack/multi_qa_mpnet_base_dot_v1_xnnpack_fp32.pte	xnnpack	fp32
xnnpack/multi_qa_mpnet_base_dot_v1_xnnpack_int8.pte	xnnpack	int8

XNNPACK int8

xnnpack/multi_qa_mpnet_base_dot_v1_xnnpack_int8.pte is 8da8w (int8 dynamic activations, int8 per-channel weights) and xnnpack/multi_qa_mpnet_base_dot_v1_xnnpack_8da4w.pte is 8da4w (int4 weights, group 32), both on the Linear layers with an int8 word-embedding table. SQuAD v1.1 dev passage retrieval (2000 questions against their 315 paragraphs), dot product:

	top-1	MRR
fp32	64.60	0.7645
int8	65.00	0.7671
8da4w	63.80	0.7578

Repository structure

config.json                                           48 B
tokenizer.json                                        694 kB
tokenizer_config.json                                 363 B
vulkan/config.json                                    1.6 kB
vulkan/multi_qa_mpnet_base_dot_v1_vulkan_fp16.pte     208 MB
vulkan/multi_qa_mpnet_base_dot_v1_vulkan_int8.pte     127 MB
xnnpack/config.json                                   2.2 kB
xnnpack/multi_qa_mpnet_base_dot_v1_xnnpack_8da4w.pte  69.1 MB
xnnpack/multi_qa_mpnet_base_dot_v1_xnnpack_fp32.pte   416 MB
xnnpack/multi_qa_mpnet_base_dot_v1_xnnpack_int8.pte   105 MB

Compatibility

These files are published for the ExecuTorch v1.4.1 runtime. ExecuTorch gives no forward compatibility guarantee, so an older runtime may fail to load them.

To use them in React Native ExecuTorch, pass the model constant shipped in the library's model registry to the corresponding task pipeline. See the documentation.

To load these files in your own ExecuTorch runtime, read the compatibility note first.