| imatrix A/B — mxbai-rerank-xsmall-v1 (cstr/mxbai-rerank-xsmall-v1-GGUF), tau vs full-precision gold, n=30, calib=14, quant_src=mxbai-rerank-xsmall-v1.gguf |
| q8_0 imatrix=0 tau_vs_full-precision=0.920000 dscore=0.0072 78.4MB [74 quantized, 126 kept] -> (A/B only) |
| q4_k imatrix=0 tau_vs_full-precision=0.728889 dscore=0.0380 67.8MB [74 quantized, 126 kept] -> (A/B only) |
| q4_k imatrix=1 tau_vs_full-precision=0.697778 dscore=0.0268 67.8MB [74 quantized, 126 kept, 72 with imatrix] -> mxbai-rerank-xsmall-v1-q4_k-imatrix-f7.gguf |
| iq4_xs imatrix=1 tau_vs_full-precision=0.697778 dscore=0.0302 67.6MB [74 quantized, 126 kept, 72 with imatrix] -> mxbai-rerank-xsmall-v1-iq4_xs-f7.gguf |
|
|
| imatrix tensor coverage (collector keys; any `leaf_N` = uncovered matmul): |
| 72 keys, leaf_N=0 (OK — no unnamed leaves) |
| 12 blk.N.attn_k.weight |
| 12 blk.N.attn_output.weight |
| 12 blk.N.attn_q.weight |
| 12 blk.N.ffn_down.weight |
| 12 blk.N.ffn_up.weight |
| 12 enc.N.attn.qkv_merged.weight |
|
|
| RAW SCORES, query "treatment for migraine headaches" x 6 docs |
| full-precision [0] -0.309 [1] -0.351 [2] -0.206 [3] -0.220 [4] -0.155 [5] -0.139 |
| q8_0 [0] -0.293 [1] -0.331 [2] -0.197 [3] -0.219 [4] -0.163 [5] -0.142 |
| q4_k [0] -0.332 [1] -0.377 [2] -0.215 [3] -0.205 [4] -0.180 [5] -0.172 |
| q4_k-im [0] -0.284 [1] -0.358 [2] -0.235 [3] -0.254 [4] -0.159 [5] -0.161 |
| iq4_xs-im [0] -0.267 [1] -0.349 [2] -0.182 [3] -0.183 [4] -0.145 [5] -0.169 |
|
|
