uploading imatrix

README.md

@@ -15,12 +15,13 @@ tags:
 ## WIP
 
 - [x] download and convert_hf_to_gguyf.py (automatically casts fp8 to bf16)
-- [ ] imatrix computation on full upcast bf16 gguf
-- [ ] upload imatrix dat for others to do custom quantizations
+- [x] imatrix computation on full upcast bf16 gguf
+- [x] upload imatrix dat for others to do custom quantizations
 - [ ] upload some logs/ files so others can see commands and early perplexity values for partial runs
-- [ ] cook q8_0 and test perplexity
+- [x] cook q8_0 and test
 - [ ] fugure out initial custom quantization recipes
-- [ ] release IQ5_K
+- [ ] release IQ5_K (ik_llama.cpp quant)
+- [ ] release IQ4_XS (mainline and ik compatible quant)
 - [ ] release smaller quants prioritizing based on what discussions are opened
 - [ ] release graph of perplexity across this quant collection
 
@@ -94,6 +95,53 @@ numactl -N ${SOCKET} -m ${SOCKET} \
 
 </details>
 
+## IQ4_XS TODO
+TODO
+
+This is the only quant in this collection that is compatible with mainline llama.cpp. ik_llama.cpp can run all of them.
+<details>
+
+<summary>👈 Secret Recipe</summary>
+
+```bash
+#!/usr/bin/env bash
+
+custom="
+# 61 Repeating Layers [0-61]
+
+# Attention [0-61] GPU
+blk\..*\.attn_q.*=q8_0
+blk\..*\.attn_k.*=q8_0
+blk\..*\.attn_v.*=q8_0
+blk\..*\.attn_output.*=q8_0
+
+# Routed Experts Layers [0-61] CPU
+blk\..*\.ffn_down_exps\.weight=iq4_xs
+blk\..*\.ffn_(gate|up)_exps\.weight=iq4_xs
+
+# Non-Repeating Layers
+token_embd\.weight=q4_K
+output\.weight=q6_K
+"
+
+custom=$(
+  echo "$custom" | grep -v '^#' | \
+  sed -Ez 's:\n+:,:g;s:,$::;s:^,::'
+)
+
+numactl -N ${SOCKET} -m ${SOCKET} \
+./build/bin/llama-quantize \
+    --custom-q "$custom" \
+    --imatrix /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat \
+    /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/MiniMax-M2.5-256x4.9B-BF16-00001-of-00010.gguf \
+    /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/MiniMax-M2.5-IQ4_XS.gguf \
+    IQ4_XS \
+    128
+```
+
+</details>
+
+
 ## Quick Start
 
 ```bash

logs/imatrix-MiniMax-M2.5-BF16.log

@@ -0,0 +1,735 @@
+model=/mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/MiniMax-M2.5-256x4.9B-BF16-00001-of-00010.gguf
+
+numactl -N ${SOCKET} -m ${SOCKET} \
+./build/bin/llama-imatrix \
+    --model "$model"\
+    -f ubergarm-imatrix-calibration-corpus-v02.txt \
+    -o /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat \
+    --no-fused-moe \
+    --no-fused-up-gate \
+    --no-fused-mul-multiadd \
+    --ctx-size 512 \
+    -ub 4096 -b 4096 \
+    --threads 96 \
+    --threads-batch 128 \
+    --no-mmap \
+    --numa numactl \
+    --verbosity 1 \
+    --layer-similarity
+
+CPU: using device CPU - 0 MiB free
+llama_model_loader: additional 9 GGUFs metadata loaded.
+llama_model_loader: loaded meta data with 40 key-value pairs and 809 tensors from /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/MiniMax-M2.5-256x4.9B-BF16-00001-of-00010.gguf (version GGUF V3 (latest))
+llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
+llama_model_loader: - kv   0:                       general.architecture str              = minimax-m2
+llama_model_loader: - kv   1:                               general.type str              = model
+llama_model_loader: - kv   2:                     general.sampling.top_k i32              = 40
+llama_model_loader: - kv   3:                     general.sampling.top_p f32              = 0.950000
+llama_model_loader: - kv   4:                      general.sampling.temp f32              = 1.000000
+llama_model_loader: - kv   5:                               general.name str              = MiniMax M2.5
+llama_model_loader: - kv   6:                         general.size_label str              = 256x4.9B
+llama_model_loader: - kv   7:                            general.license str              = other
+llama_model_loader: - kv   8:                       general.license.name str              = modified-mit
+llama_model_loader: - kv   9:                       general.license.link str              = https://github.com/MiniMax-AI/MiniMax...
+llama_model_loader: - kv  10:                               general.tags arr[str,1]       = ["text-generation"]
+llama_model_loader: - kv  11:                     minimax-m2.block_count u32              = 62
+llama_model_loader: - kv  12:                  minimax-m2.context_length u32              = 196608
+llama_model_loader: - kv  13:                minimax-m2.embedding_length u32              = 3072
+llama_model_loader: - kv  14:             minimax-m2.feed_forward_length u32              = 1536
+llama_model_loader: - kv  15:            minimax-m2.attention.head_count u32              = 48
+llama_model_loader: - kv  16:         minimax-m2.attention.head_count_kv u32              = 8
+llama_model_loader: - kv  17:                  minimax-m2.rope.freq_base f32              = 5000000.000000
+llama_model_loader: - kv  18: minimax-m2.attention.layer_norm_rms_epsilon f32              = 0.000001
+llama_model_loader: - kv  19:                    minimax-m2.expert_count u32              = 256
+llama_model_loader: - kv  20:               minimax-m2.expert_used_count u32              = 8
+llama_model_loader: - kv  21:              minimax-m2.expert_gating_func u32              = 2
+llama_model_loader: - kv  22:            minimax-m2.attention.key_length u32              = 128
+llama_model_loader: - kv  23:          minimax-m2.attention.value_length u32              = 128
+llama_model_loader: - kv  24:                          general.file_type u32              = 32
+llama_model_loader: - kv  25:      minimax-m2.expert_feed_forward_length u32              = 1536
+llama_model_loader: - kv  26:            minimax-m2.rope.dimension_count u32              = 64
+llama_model_loader: - kv  27:               general.quantization_version u32              = 2
+llama_model_loader: - kv  28:                       tokenizer.ggml.model str              = gpt2
+llama_model_loader: - kv  29:                         tokenizer.ggml.pre str              = minimax-m2
+llama_model_loader: - kv  30:                      tokenizer.ggml.tokens arr[str,200064]  = ["Ā", "ā", "Ă", "ă", "Ą", "ą", ...
+llama_model_loader: - kv  31:                  tokenizer.ggml.token_type arr[i32,200064]  = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
+llama_model_loader: - kv  32:                      tokenizer.ggml.merges arr[str,199744]  = ["Ġ Ġ", "Ġ t", "Ġ a", "i n", "e r...
+llama_model_loader: - kv  33:                tokenizer.ggml.bos_token_id u32              = 200034
+llama_model_loader: - kv  34:                tokenizer.ggml.eos_token_id u32              = 200020
+llama_model_loader: - kv  35:            tokenizer.ggml.unknown_token_id u32              = 200021
+llama_model_loader: - kv  36:                    tokenizer.chat_template str              = {# ----------‑‑‑ special token ...
+llama_model_loader: - kv  37:                                   split.no u16              = 0
+llama_model_loader: - kv  38:                                split.count u16              = 10
+llama_model_loader: - kv  39:                        split.tensors.count i32              = 809
+llama_model_loader: - type  f32:  373 tensors
+llama_model_loader: - type bf16:  436 tensors
+load: special_eos_id is not in special_eog_ids - the tokenizer config may be incorrect
+load: printing all EOG tokens:
+load:   - 200004 ('<fim_pad>')
+load:   - 200005 ('<reponame>')
+load:   - 200020 ('[e~[')
+load: special tokens cache size = 54
+load: token to piece cache size = 1.3355 MB
+llm_load_print_meta: format           = GGUF V3 (latest)
+llm_load_print_meta: arch             = minimax-m2
+llm_load_print_meta: n_ctx_train      = 196608
+llm_load_print_meta: n_embd           = 3072
+llm_load_print_meta: n_layer          = 62
+llm_load_print_meta: n_head           = 48
+llm_load_print_meta: n_head_kv        = 8
+llm_load_print_meta: n_rot            = 64
+llm_load_print_meta: n_swa            = 0
+llm_load_print_meta: n_swa_pattern    = 1
+llm_load_print_meta: n_embd_head_k    = 128
+llm_load_print_meta: n_embd_head_v    = 128
+llm_load_print_meta: n_gqa            = 6
+llm_load_print_meta: n_embd_k_gqa     = 1024
+llm_load_print_meta: n_embd_v_gqa     = 1024
+llm_load_print_meta: f_norm_eps       = 0.0e+00
+llm_load_print_meta: f_norm_rms_eps   = 1.0e-06
+llm_load_print_meta: f_clamp_kqv      = 0.0e+00
+llm_load_print_meta: f_max_alibi_bias = 0.0e+00
+llm_load_print_meta: f_logit_scale    = 0.0e+00
+llm_load_print_meta: n_ff             = 1536
+llm_load_print_meta: n_expert         = 256
+llm_load_print_meta: n_expert_used    = 8
+llm_load_print_meta: causal attn      = 1
+llm_load_print_meta: pooling type     = 0
+llm_load_print_meta: rope type        = 2
+llm_load_print_meta: rope scaling     = linear
+llm_load_print_meta: freq_base_train  = 5000000.0
+llm_load_print_meta: freq_scale_train = 1
+llm_load_print_meta: n_ctx_orig_yarn  = 196608
+llm_load_print_meta: rope_finetuned   = unknown
+llm_load_print_meta: ssm_d_conv       = 0
+llm_load_print_meta: ssm_d_inner      = 0
+llm_load_print_meta: ssm_d_state      = 0
+llm_load_print_meta: ssm_dt_rank      = 0
+llm_load_print_meta: model type       = 230B.A10B
+llm_load_print_meta: model ftype      = BF16
+llm_load_print_meta: model params     = 228.690 B
+llm_load_print_meta: model size       = 426.060 GiB (16.003 BPW) 
+llm_load_print_meta: repeating layers = 423.771 GiB (16.003 BPW, 227.461 B parameters)
+llm_load_print_meta: general.name     = MiniMax M2.5
+print_info: vocab type       = BPE
+print_info: n_vocab          = 200064
+print_info: n_merges         = 199744
+print_info: BOS token        = 200034 ']~!b['
+print_info: EOS token        = 200020 '[e~['
+print_info: UNK token        = 200021 ']!d~['
+print_info: LF token         = 10 'Ċ'
+print_info: FIM PRE token    = 200001 '<fim_prefix>'
+print_info: FIM SUF token    = 200003 '<fim_suffix>'
+print_info: FIM MID token    = 200002 '<fim_middle>'
+print_info: FIM PAD token    = 200004 '<fim_pad>'
+print_info: FIM REP token    = 200005 '<reponame>'
+print_info: EOG token        = 200004 '<fim_pad>'
+print_info: EOG token        = 200005 '<reponame>'
+print_info: EOG token        = 200020 '[e~['
+print_info: max token length = 256
+llm_load_tensors: ggml ctx size =    0.35 MiB
+llm_load_tensors: offloading 0 repeating layers to GPU
+llm_load_tensors: offloaded 0/63 layers to GPU
+llm_load_tensors:        CPU buffer size = 436285.72 MiB
+....................................................................................................
+llama_new_context_with_model: n_ctx         = 512
+llama_new_context_with_model: n_batch       = 512
+llama_new_context_with_model: n_ubatch      = 512
+llama_new_context_with_model: flash_attn    = 1
+llama_new_context_with_model: attn_max_b    = 0
+llama_new_context_with_model: fused_moe     = 0
+llama_new_context_with_model: grouped er    = 0
+llama_new_context_with_model: fused_up_gate = 0
+llama_new_context_with_model: fused_mmad    = 0
+llama_new_context_with_model: rope_cache    = 0
+llama_new_context_with_model: graph_reuse   = 1
+llama_new_context_with_model: k_cache_hadam = 0
+llama_new_context_with_model: split_mode_graph_scheduling = 0
+llama_new_context_with_model: reduce_type   = f16
+llama_new_context_with_model: sched_async   = 0
+llama_new_context_with_model: ser           = -1, 0
+llama_new_context_with_model: freq_base     = 5000000.0
+llama_new_context_with_model: freq_scale    = 1
+llama_kv_cache_init:        CPU KV buffer size =   124.00 MiB
+llama_new_context_with_model: KV self size  =  124.00 MiB, K (f16):   62.00 MiB, V (f16):   62.00 MiB
+llama_new_context_with_model:        CPU  output buffer size =     0.76 MiB
+llama_new_context_with_model:        CPU compute buffer size =   402.75 MiB
+llama_new_context_with_model: graph nodes  = 2609
+llama_new_context_with_model: graph splits = 1
+XXXXXXXXXXXXXXXXXXXXX Setting only active experts offload
+
+system_info: n_threads = 96 (n_threads_batch = 128) / 512 | AVX = 1 | AVX_VNNI = 1 | AVX2 = 1 | AVX512 = 1 | AVX512_VBMI = 1 | AVX512_VNNI = 1 | AVX512_BF16 = 1 | FMA = 1 | NEON = 0 | SVE = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 | MATMUL_INT8 = 0 | 
+compute_imatrix: tokenizing the input ..
+compute_imatrix: tokenization took 902.359 ms
+compute_imatrix: computing over 796 chunks with batch_size 512
+compute_imatrix: 4.17 seconds per pass - ETA 55.32 minutes
+===================================== llama_new_context_with_model: f16
+======================================= HAVE_FANCY_SIMD is defined
+[1]100.8416,[2]18.0540,[3]8.7698,[4]5.1113,[5]3.8374,[6]3.1449,[7]2.6941,[8]2.4214,[9]2.3697,
+save_imatrix: entry '             blk.60.ffn_down_exps.weight' has partial data (94.14%) 15 out of 256 experts are missing data - skipping
+save_imatrix: entry '             blk.59.ffn_gate_exps.weight' has partial data (95.31%) 12 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '               blk.59.ffn_up_exps.weight' has partial data (95.31%) 12 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.58.ffn_down_exps.weight' has partial data (98.05%) 5 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.58.ffn_gate_exps.weight' has partial data (98.05%) 5 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.57.ffn_down_exps.weight' has partial data (96.09%) 10 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.57.ffn_gate_exps.weight' has partial data (96.09%) 10 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '               blk.57.ffn_up_exps.weight' has partial data (96.09%) 10 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.56.ffn_gate_exps.weight' has partial data (96.09%) 10 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.55.ffn_down_exps.weight' has partial data (97.66%) 6 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.60.ffn_gate_exps.weight' has partial data (94.14%) 15 out of 256 experts are missing data - skipping
+save_imatrix: entry '             blk.55.ffn_gate_exps.weight' has partial data (97.66%) 6 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '               blk.55.ffn_up_exps.weight' has partial data (97.66%) 6 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.54.ffn_down_exps.weight' has partial data (97.66%) 6 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.53.ffn_down_exps.weight' has partial data (98.05%) 5 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.56.ffn_down_exps.weight' has partial data (96.09%) 10 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.52.ffn_down_exps.weight' has partial data (98.05%) 5 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.52.ffn_gate_exps.weight' has partial data (98.05%) 5 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '               blk.52.ffn_up_exps.weight' has partial data (98.05%) 5 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '               blk.48.ffn_up_exps.weight' has partial data (95.70%) 11 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.47.ffn_gate_exps.weight' has partial data (98.44%) 4 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '               blk.47.ffn_up_exps.weight' has partial data (98.44%) 4 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.46.ffn_down_exps.weight' has partial data (98.05%) 5 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '               blk.46.ffn_up_exps.weight' has partial data (98.05%) 5 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.45.ffn_down_exps.weight' has partial data (95.70%) 11 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.43.ffn_gate_exps.weight' has partial data (96.88%) 8 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '               blk.43.ffn_up_exps.weight' has partial data (96.88%) 8 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.42.ffn_down_exps.weight' has partial data (96.88%) 8 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '               blk.42.ffn_up_exps.weight' has partial data (96.88%) 8 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.44.ffn_gate_exps.weight' has partial data (94.92%) 13 out of 256 experts are missing data - skipping
+save_imatrix: entry '             blk.39.ffn_gate_exps.weight' has partial data (96.09%) 10 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.38.ffn_down_exps.weight' has partial data (92.97%) 18 out of 256 experts are missing data - skipping
+save_imatrix: entry '             blk.38.ffn_gate_exps.weight' has partial data (92.97%) 18 out of 256 experts are missing data - skipping
+save_imatrix: entry '             blk.39.ffn_down_exps.weight' has partial data (96.09%) 10 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.37.ffn_gate_exps.weight' has partial data (94.92%) 13 out of 256 experts are missing data - skipping
+save_imatrix: entry '               blk.37.ffn_up_exps.weight' has partial data (94.92%) 13 out of 256 experts are missing data - skipping
+save_imatrix: entry '             blk.36.ffn_down_exps.weight' has partial data (95.31%) 12 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.40.ffn_down_exps.weight' has partial data (94.14%) 15 out of 256 experts are missing data - skipping
+save_imatrix: entry '             blk.35.ffn_down_exps.weight' has partial data (94.92%) 13 out of 256 experts are missing data - skipping
+save_imatrix: entry '             blk.35.ffn_gate_exps.weight' has partial data (94.92%) 13 out of 256 experts are missing data - skipping
+save_imatrix: entry '             blk.51.ffn_gate_exps.weight' has partial data (98.83%) 3 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '               blk.34.ffn_up_exps.weight' has partial data (94.14%) 15 out of 256 experts are missing data - skipping
+save_imatrix: entry '             blk.33.ffn_down_exps.weight' has partial data (95.70%) 11 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.33.ffn_gate_exps.weight' has partial data (95.70%) 11 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '               blk.33.ffn_up_exps.weight' has partial data (95.70%) 11 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '               blk.39.ffn_up_exps.weight' has partial data (96.09%) 10 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.32.ffn_down_exps.weight' has partial data (94.14%) 15 out of 256 experts are missing data - skipping
+save_imatrix: entry '               blk.32.ffn_up_exps.weight' has partial data (94.14%) 15 out of 256 experts are missing data - skipping
+save_imatrix: entry '             blk.34.ffn_down_exps.weight' has partial data (94.14%) 15 out of 256 experts are missing data - skipping
+save_imatrix: entry '             blk.42.ffn_gate_exps.weight' has partial data (96.88%) 8 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.50.ffn_down_exps.weight' has partial data (98.05%) 5 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '               blk.38.ffn_up_exps.weight' has partial data (92.97%) 18 out of 256 experts are missing data - skipping
+save_imatrix: entry '             blk.45.ffn_gate_exps.weight' has partial data (95.70%) 11 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.47.ffn_down_exps.weight' has partial data (98.44%) 4 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '               blk.45.ffn_up_exps.weight' has partial data (95.70%) 11 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.23.ffn_gate_exps.weight' has partial data (97.66%) 6 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '               blk.51.ffn_up_exps.weight' has partial data (98.83%) 3 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.53.ffn_gate_exps.weight' has partial data (98.05%) 5 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.41.ffn_down_exps.weight' has partial data (96.48%) 9 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '               blk.50.ffn_up_exps.weight' has partial data (98.05%) 5 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.59.ffn_down_exps.weight' has partial data (95.31%) 12 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '               blk.54.ffn_up_exps.weight' has partial data (97.66%) 6 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '                blk.4.ffn_up_exps.weight' has partial data (99.61%) 1 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.44.ffn_down_exps.weight' has partial data (94.92%) 13 out of 256 experts are missing data - skipping
+save_imatrix: entry '             blk.36.ffn_gate_exps.weight' has partial data (95.31%) 12 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.31.ffn_gate_exps.weight' has partial data (95.31%) 12 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '               blk.58.ffn_up_exps.weight' has partial data (98.05%) 5 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.29.ffn_gate_exps.weight' has partial data (94.92%) 13 out of 256 experts are missing data - skipping
+save_imatrix: entry '             blk.41.ffn_gate_exps.weight' has partial data (96.48%) 9 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '               blk.41.ffn_up_exps.weight' has partial data (96.48%) 9 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '                blk.1.ffn_up_exps.weight' has partial data (95.70%) 11 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.34.ffn_gate_exps.weight' has partial data (94.14%) 15 out of 256 experts are missing data - skipping
+save_imatrix: entry '             blk.49.ffn_down_exps.weight' has partial data (98.05%) 5 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '               blk.19.ffn_up_exps.weight' has partial data (99.61%) 1 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '               blk.56.ffn_up_exps.weight' has partial data (96.09%) 10 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '               blk.44.ffn_up_exps.weight' has partial data (94.92%) 13 out of 256 experts are missing data - skipping
+save_imatrix: entry '              blk.4.ffn_gate_exps.weight' has partial data (99.61%) 1 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.23.ffn_down_exps.weight' has partial data (97.66%) 6 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '              blk.0.ffn_gate_exps.weight' has partial data (99.22%) 2 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.37.ffn_down_exps.weight' has partial data (94.92%) 13 out of 256 experts are missing data - skipping
+save_imatrix: entry '             blk.32.ffn_gate_exps.weight' has partial data (94.14%) 15 out of 256 experts are missing data - skipping
+save_imatrix: entry '             blk.61.ffn_down_exps.weight' has partial data (81.64%) 47 out of 256 experts are missing data - skipping
+save_imatrix: entry '             blk.48.ffn_down_exps.weight' has partial data (95.70%) 11 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '               blk.29.ffn_up_exps.weight' has partial data (94.92%) 13 out of 256 experts are missing data - skipping
+save_imatrix: entry '              blk.0.ffn_down_exps.weight' has partial data (99.22%) 2 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '               blk.40.ffn_up_exps.weight' has partial data (94.14%) 15 out of 256 experts are missing data - skipping
+save_imatrix: entry '               blk.18.ffn_up_exps.weight' has partial data (99.22%) 2 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.21.ffn_down_exps.weight' has partial data (98.83%) 3 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '              blk.4.ffn_down_exps.weight' has partial data (99.61%) 1 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.54.ffn_gate_exps.weight' has partial data (97.66%) 6 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '                blk.2.ffn_up_exps.weight' has partial data (99.61%) 1 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.28.ffn_gate_exps.weight' has partial data (96.88%) 8 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.50.ffn_gate_exps.weight' has partial data (98.05%) 5 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.24.ffn_down_exps.weight' has partial data (95.70%) 11 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '               blk.28.ffn_up_exps.weight' has partial data (96.88%) 8 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.51.ffn_down_exps.weight' has partial data (98.83%) 3 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '               blk.26.ffn_up_exps.weight' has partial data (94.53%) 14 out of 256 experts are missing data - skipping
+save_imatrix: entry '             blk.30.ffn_gate_exps.weight' has partial data (96.88%) 8 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.40.ffn_gate_exps.weight' has partial data (94.14%) 15 out of 256 experts are missing data - skipping
+save_imatrix: entry '              blk.1.ffn_down_exps.weight' has partial data (95.70%) 11 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '              blk.1.ffn_gate_exps.weight' has partial data (95.70%) 11 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.18.ffn_gate_exps.weight' has partial data (99.22%) 2 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '               blk.49.ffn_up_exps.weight' has partial data (98.05%) 5 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '               blk.35.ffn_up_exps.weight' has partial data (94.92%) 13 out of 256 experts are missing data - skipping
+save_imatrix: entry '             blk.18.ffn_down_exps.weight' has partial data (99.22%) 2 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '               blk.21.ffn_up_exps.weight' has partial data (98.83%) 3 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.19.ffn_gate_exps.weight' has partial data (99.61%) 1 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '               blk.24.ffn_up_exps.weight' has partial data (95.70%) 11 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.19.ffn_down_exps.weight' has partial data (99.61%) 1 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '               blk.61.ffn_up_exps.weight' has partial data (81.64%) 47 out of 256 experts are missing data - skipping
+save_imatrix: entry '              blk.2.ffn_down_exps.weight' has partial data (99.61%) 1 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.48.ffn_gate_exps.weight' has partial data (95.70%) 11 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.21.ffn_gate_exps.weight' has partial data (98.83%) 3 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.29.ffn_down_exps.weight' has partial data (94.92%) 13 out of 256 experts are missing data - skipping
+save_imatrix: entry '               blk.22.ffn_up_exps.weight' has partial data (98.83%) 3 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.22.ffn_gate_exps.weight' has partial data (98.83%) 3 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.22.ffn_down_exps.weight' has partial data (98.83%) 3 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '               blk.23.ffn_up_exps.weight' has partial data (97.66%) 6 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '               blk.53.ffn_up_exps.weight' has partial data (98.05%) 5 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.24.ffn_gate_exps.weight' has partial data (95.70%) 11 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.49.ffn_gate_exps.weight' has partial data (98.05%) 5 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.61.ffn_gate_exps.weight' has partial data (81.64%) 47 out of 256 experts are missing data - skipping
+save_imatrix: entry '               blk.25.ffn_up_exps.weight' has partial data (97.27%) 7 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '              blk.2.ffn_gate_exps.weight' has partial data (99.61%) 1 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.25.ffn_gate_exps.weight' has partial data (97.27%) 7 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.25.ffn_down_exps.weight' has partial data (97.27%) 7 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '               blk.36.ffn_up_exps.weight' has partial data (95.31%) 12 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.26.ffn_gate_exps.weight' has partial data (94.53%) 14 out of 256 experts are missing data - skipping
+save_imatrix: entry '               blk.27.ffn_up_exps.weight' has partial data (95.70%) 11 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.27.ffn_gate_exps.weight' has partial data (95.70%) 11 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '               blk.60.ffn_up_exps.weight' has partial data (94.14%) 15 out of 256 experts are missing data - skipping
+save_imatrix: entry '             blk.26.ffn_down_exps.weight' has partial data (94.53%) 14 out of 256 experts are missing data - skipping
+save_imatrix: entry '             blk.27.ffn_down_exps.weight' has partial data (95.70%) 11 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.43.ffn_down_exps.weight' has partial data (96.88%) 8 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '                blk.0.ffn_up_exps.weight' has partial data (99.22%) 2 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.28.ffn_down_exps.weight' has partial data (96.88%) 8 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '               blk.31.ffn_up_exps.weight' has partial data (95.31%) 12 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.46.ffn_gate_exps.weight' has partial data (98.05%) 5 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '               blk.30.ffn_up_exps.weight' has partial data (96.88%) 8 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.30.ffn_down_exps.weight' has partial data (96.88%) 8 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.31.ffn_down_exps.weight' has partial data (95.31%) 12 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: warning: storing only 464 out of 497 entries
+
+save_imatrix: stored collected data after 10 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[10]2.3050,[11]2.3190,[12]2.5565,[13]2.6783,[14]2.6639,[15]2.5142,[16]2.3897,[17]2.2821,[18]2.2100,[19]2.1275,
+save_imatrix: entry '             blk.60.ffn_down_exps.weight' has partial data (94.92%) 13 out of 256 experts are missing data - skipping
+save_imatrix: entry '             blk.60.ffn_gate_exps.weight' has partial data (94.92%) 13 out of 256 experts are missing data - skipping
+save_imatrix: entry '             blk.44.ffn_gate_exps.weight' has partial data (96.48%) 9 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.38.ffn_down_exps.weight' has partial data (94.53%) 14 out of 256 experts are missing data - skipping
+save_imatrix: entry '             blk.38.ffn_gate_exps.weight' has partial data (94.53%) 14 out of 256 experts are missing data - skipping
+save_imatrix: entry '             blk.37.ffn_gate_exps.weight' has partial data (96.88%) 8 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '               blk.37.ffn_up_exps.weight' has partial data (96.88%) 8 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.40.ffn_down_exps.weight' has partial data (96.48%) 9 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.35.ffn_down_exps.weight' has partial data (97.66%) 6 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.35.ffn_gate_exps.weight' has partial data (97.66%) 6 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '               blk.34.ffn_up_exps.weight' has partial data (96.09%) 10 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.32.ffn_down_exps.weight' has partial data (96.88%) 8 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '               blk.32.ffn_up_exps.weight' has partial data (96.88%) 8 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.34.ffn_down_exps.weight' has partial data (96.09%) 10 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '               blk.38.ffn_up_exps.weight' has partial data (94.53%) 14 out of 256 experts are missing data - skipping
+save_imatrix: entry '             blk.44.ffn_down_exps.weight' has partial data (96.48%) 9 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.29.ffn_gate_exps.weight' has partial data (96.88%) 8 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.34.ffn_gate_exps.weight' has partial data (96.09%) 10 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '               blk.44.ffn_up_exps.weight' has partial data (96.48%) 9 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.37.ffn_down_exps.weight' has partial data (96.88%) 8 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.32.ffn_gate_exps.weight' has partial data (96.88%) 8 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.61.ffn_down_exps.weight' has partial data (82.81%) 44 out of 256 experts are missing data - skipping
+save_imatrix: entry '               blk.29.ffn_up_exps.weight' has partial data (96.88%) 8 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '               blk.40.ffn_up_exps.weight' has partial data (96.48%) 9 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '               blk.26.ffn_up_exps.weight' has partial data (97.66%) 6 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.40.ffn_gate_exps.weight' has partial data (96.48%) 9 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '               blk.35.ffn_up_exps.weight' has partial data (97.66%) 6 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '               blk.61.ffn_up_exps.weight' has partial data (82.81%) 44 out of 256 experts are missing data - skipping
+save_imatrix: entry '             blk.29.ffn_down_exps.weight' has partial data (96.88%) 8 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.61.ffn_gate_exps.weight' has partial data (82.81%) 44 out of 256 experts are missing data - skipping
+save_imatrix: entry '             blk.26.ffn_gate_exps.weight' has partial data (97.66%) 6 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '               blk.60.ffn_up_exps.weight' has partial data (94.92%) 13 out of 256 experts are missing data - skipping
+save_imatrix: entry '             blk.26.ffn_down_exps.weight' has partial data (97.66%) 6 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: warning: storing only 488 out of 497 entries
+
+save_imatrix: stored collected data after 20 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[20]2.0719,[21]2.0109,[22]1.9613,[23]1.9689,[24]1.9503,[25]1.8990,[26]1.9944,[27]2.0909,[28]2.2036,[29]2.1830,
+save_imatrix: entry '             blk.60.ffn_down_exps.weight' has partial data (95.70%) 11 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.60.ffn_gate_exps.weight' has partial data (95.70%) 11 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.38.ffn_down_exps.weight' has partial data (95.70%) 11 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.38.ffn_gate_exps.weight' has partial data (95.70%) 11 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '               blk.38.ffn_up_exps.weight' has partial data (95.70%) 11 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.61.ffn_down_exps.weight' has partial data (83.98%) 41 out of 256 experts are missing data - skipping
+save_imatrix: entry '               blk.61.ffn_up_exps.weight' has partial data (83.98%) 41 out of 256 experts are missing data - skipping
+save_imatrix: entry '             blk.61.ffn_gate_exps.weight' has partial data (83.98%) 41 out of 256 experts are missing data - skipping
+save_imatrix: entry '               blk.60.ffn_up_exps.weight' has partial data (95.70%) 11 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: warning: storing only 494 out of 497 entries
+
+save_imatrix: stored collected data after 30 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[30]2.2902,[31]2.2704,[32]2.3032,[33]2.2793,[34]2.2956,[35]2.2953,[36]2.2934,[37]2.2985,[38]2.3458,[39]2.3544,
+save_imatrix: entry '             blk.61.ffn_down_exps.weight' has partial data (86.72%) 34 out of 256 experts are missing data - skipping
+save_imatrix: entry '               blk.61.ffn_up_exps.weight' has partial data (86.72%) 34 out of 256 experts are missing data - skipping
+save_imatrix: entry '             blk.61.ffn_gate_exps.weight' has partial data (86.72%) 34 out of 256 experts are missing data - skipping
+save_imatrix: warning: storing only 494 out of 497 entries
+
+save_imatrix: stored collected data after 40 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[40]2.3468,[41]2.3822,[42]2.3951,[43]2.4145,[44]2.4509,[45]2.4777,[46]2.4612,[47]2.4683,[48]2.4600,[49]2.4646,
+save_imatrix: entry '             blk.61.ffn_down_exps.weight' has partial data (87.89%) 31 out of 256 experts are missing data - skipping
+save_imatrix: entry '               blk.61.ffn_up_exps.weight' has partial data (87.89%) 31 out of 256 experts are missing data - skipping
+save_imatrix: entry '             blk.61.ffn_gate_exps.weight' has partial data (87.89%) 31 out of 256 experts are missing data - skipping
+save_imatrix: warning: storing only 494 out of 497 entries
+
+save_imatrix: stored collected data after 50 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[50]2.4508,[51]2.4695,[52]2.5017,[53]2.4834,[54]2.4931,[55]2.4920,[56]2.4939,[57]2.4820,[58]2.5751,[59]2.6244,
+save_imatrix: entry '             blk.61.ffn_down_exps.weight' has partial data (90.62%) 24 out of 256 experts are missing data - skipping
+save_imatrix: entry '               blk.61.ffn_up_exps.weight' has partial data (90.62%) 24 out of 256 experts are missing data - skipping
+save_imatrix: entry '             blk.61.ffn_gate_exps.weight' has partial data (90.62%) 24 out of 256 experts are missing data - skipping
+save_imatrix: warning: storing only 494 out of 497 entries
+
+save_imatrix: stored collected data after 60 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[60]2.6769,[61]2.7058,[62]2.8303,[63]2.8597,[64]2.9258,[65]3.0000,[66]3.0671,[67]3.1653,[68]3.2582,[69]3.3452,
+save_imatrix: entry '             blk.61.ffn_down_exps.weight' has partial data (98.83%) 3 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '               blk.61.ffn_up_exps.weight' has partial data (98.83%) 3 out of 256 experts are missing data Storing **but be aware**
+save_imatrix: entry '             blk.61.ffn_gate_exps.weight' has partial data (98.83%) 3 out of 256 experts are missing data Storing **but be aware**
+
+save_imatrix: stored collected data after 70 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[70]3.4034,[71]3.4345,[72]3.4540,[73]3.4941,[74]3.5613,[75]3.6570,[76]3.6509,[77]3.6298,[78]3.6173,[79]3.6542,
+save_imatrix: stored collected data after 80 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[80]3.7726,[81]3.8429,[82]3.8435,[83]3.8303,[84]3.8033,[85]3.8926,[86]3.9366,[87]3.9326,[88]3.9539,[89]4.0010,
+save_imatrix: stored collected data after 90 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[90]4.0674,[91]4.0656,[92]4.0667,[93]4.0727,[94]4.0655,[95]4.0405,[96]4.0769,[97]4.1010,[98]4.1313,[99]4.0977,
+save_imatrix: stored collected data after 100 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[100]4.1548,[101]4.2158,[102]4.2695,[103]4.3262,[104]4.3759,[105]4.4281,[106]4.4803,[107]4.4715,[108]4.4784,[109]4.5028,
+save_imatrix: stored collected data after 110 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[110]4.5474,[111]4.5927,[112]4.6496,[113]4.7003,[114]4.7141,[115]4.6927,[116]4.6539,[117]4.6528,[118]4.6535,[119]4.6131,
+save_imatrix: stored collected data after 120 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[120]4.5697,[121]4.5580,[122]4.5537,[123]4.5605,[124]4.6021,[125]4.6008,[126]4.6414,[127]4.6940,[128]4.7415,[129]4.7023,
+save_imatrix: stored collected data after 130 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[130]4.6613,[131]4.6268,[132]4.5924,[133]4.5875,[134]4.5805,[135]4.6306,[136]4.6797,[137]4.7032,[138]4.7067,[139]4.7420,
+save_imatrix: stored collected data after 140 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[140]4.7920,[141]4.8470,[142]4.8948,[143]4.9300,[144]4.9611,[145]4.9851,[146]4.9971,[147]5.0042,[148]4.9957,[149]5.0188,
+save_imatrix: stored collected data after 150 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[150]5.0285,[151]5.0410,[152]5.0643,[153]5.0942,[154]5.0947,[155]5.0935,[156]5.1096,[157]5.1289,[158]5.1375,[159]5.1522,
+save_imatrix: stored collected data after 160 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[160]5.1597,[161]5.1701,[162]5.1797,[163]5.1788,[164]5.1719,[165]5.2009,[166]5.2150,[167]5.2166,[168]5.2443,[169]5.2703,
+save_imatrix: stored collected data after 170 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[170]5.2483,[171]5.2715,[172]5.2773,[173]5.2984,[174]5.3258,[175]5.3379,[176]5.3283,[177]5.3151,[178]5.3047,[179]5.2910,
+save_imatrix: stored collected data after 180 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[180]5.2775,[181]5.2628,[182]5.2560,[183]5.2756,[184]5.2997,[185]5.3561,[186]5.4026,[187]5.4398,[188]5.4930,[189]5.5172,
+save_imatrix: stored collected data after 190 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[190]5.5395,[191]5.5235,[192]5.5716,[193]5.5610,[194]5.5241,[195]5.4826,[196]5.5015,[197]5.5368,[198]5.5523,[199]5.5646,
+save_imatrix: stored collected data after 200 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[200]5.5927,[201]5.6174,[202]5.6358,[203]5.6606,[204]5.6842,[205]5.6873,[206]5.6534,[207]5.6246,[208]5.6023,[209]5.5719,
+save_imatrix: stored collected data after 210 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[210]5.5471,[211]5.5212,[212]5.5006,[213]5.5046,[214]5.5202,[215]5.4986,[216]5.4970,[217]5.4962,[218]5.5085,[219]5.5268,
+save_imatrix: stored collected data after 220 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[220]5.5527,[221]5.5805,[222]5.6067,[223]5.6140,[224]5.6623,[225]5.7029,[226]5.7171,[227]5.7215,[228]5.7411,[229]5.7725,
+save_imatrix: stored collected data after 230 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[230]5.7961,[231]5.8084,[232]5.8430,[233]5.8511,[234]5.8988,[235]5.9427,[236]5.9583,[237]5.9775,[238]5.9984,[239]6.0118,
+save_imatrix: stored collected data after 240 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[240]6.0316,[241]6.0590,[242]6.0846,[243]6.1071,[244]6.1235,[245]6.1450,[246]6.1690,[247]6.1905,[248]6.1945,[249]6.2078,
+save_imatrix: stored collected data after 250 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[250]6.2167,[251]6.2338,[252]6.2465,[253]6.2673,[254]6.2834,[255]6.2982,[256]6.2908,[257]6.2987,[258]6.3216,[259]6.3524,
+save_imatrix: stored collected data after 260 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[260]6.3821,[261]6.4011,[262]6.4337,[263]6.4370,[264]6.4485,[265]6.4615,[266]6.4747,[267]6.4937,[268]6.5137,[269]6.5318,
+save_imatrix: stored collected data after 270 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[270]6.5456,[271]6.5472,[272]6.5737,[273]6.5894,[274]6.6146,[275]6.6365,[276]6.6386,[277]6.6434,[278]6.6558,[279]6.6591,
+save_imatrix: stored collected data after 280 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[280]6.6757,[281]6.6873,[282]6.7073,[283]6.7188,[284]6.7405,[285]6.7606,[286]6.7826,[287]6.8010,[288]6.8311,[289]6.8524,
+save_imatrix: stored collected data after 290 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[290]6.8742,[291]6.8961,[292]6.9145,[293]6.9247,[294]6.9413,[295]6.9506,[296]6.9619,[297]6.9825,[298]6.9945,[299]7.0075,
+save_imatrix: stored collected data after 300 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[300]7.0164,[301]7.0261,[302]7.0357,[303]7.0643,[304]7.0778,[305]7.0867,[306]7.1079,[307]7.1349,[308]7.1668,[309]7.1944,
+save_imatrix: stored collected data after 310 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[310]7.1498,[311]7.1402,[312]7.1124,[313]7.0906,[314]7.1203,[315]7.1342,[316]7.1045,[317]7.1181,[318]7.1308,[319]7.1253,
+save_imatrix: stored collected data after 320 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[320]7.1261,[321]7.1310,[322]7.1548,[323]7.1540,[324]7.1694,[325]7.1873,[326]7.2010,[327]7.2126,[328]7.1693,[329]7.1886,
+save_imatrix: stored collected data after 330 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[330]7.2287,[331]7.2576,[332]7.2852,[333]7.3117,[334]7.3125,[335]7.3107,[336]7.3205,[337]7.3253,[338]7.3440,[339]7.3657,
+save_imatrix: stored collected data after 340 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[340]7.3435,[341]7.3572,[342]7.3664,[343]7.3644,[344]7.3636,[345]7.3642,[346]7.3502,[347]7.3586,[348]7.3776,[349]7.3689,
+save_imatrix: stored collected data after 350 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[350]7.3611,[351]7.3309,[352]7.2940,[353]7.2685,[354]7.2460,[355]7.2095,[356]7.1873,[357]7.1637,[358]7.1400,[359]7.1216,
+save_imatrix: stored collected data after 360 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[360]7.1003,[361]7.0804,[362]7.0803,[363]7.0626,[364]7.0332,[365]7.0162,[366]6.9900,[367]6.9742,[368]6.9599,[369]6.9362,
+save_imatrix: stored collected data after 370 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[370]6.9323,[371]6.9272,[372]6.9243,[373]6.9131,[374]6.8957,[375]6.8621,[376]6.8302,[377]6.8145,[378]6.7922,[379]6.7651,
+save_imatrix: stored collected data after 380 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[380]6.7313,[381]6.6986,[382]6.6816,[383]6.6733,[384]6.6668,[385]6.6557,[386]6.6726,[387]6.6701,[388]6.6427,[389]6.6197,
+save_imatrix: stored collected data after 390 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[390]6.6035,[391]6.5837,[392]6.5654,[393]6.5507,[394]6.5271,[395]6.5077,[396]6.4910,[397]6.4697,[398]6.4447,[399]6.4204,
+save_imatrix: stored collected data after 400 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[400]6.4055,[401]6.3955,[402]6.3811,[403]6.3761,[404]6.3723,[405]6.3606,[406]6.3514,[407]6.3373,[408]6.3106,[409]6.2835,
+save_imatrix: stored collected data after 410 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[410]6.2578,[411]6.2356,[412]6.2092,[413]6.1843,[414]6.1627,[415]6.1370,[416]6.1159,[417]6.0982,[418]6.0780,[419]6.0578,
+save_imatrix: stored collected data after 420 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[420]6.0364,[421]6.0138,[422]5.9960,[423]5.9974,[424]5.9900,[425]5.9854,[426]5.9652,[427]5.9479,[428]5.9267,[429]5.9079,
+save_imatrix: stored collected data after 430 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[430]5.8896,[431]5.8765,[432]5.8603,[433]5.8488,[434]5.8319,[435]5.8255,[436]5.8106,[437]5.7925,[438]5.7873,[439]5.7731,
+save_imatrix: stored collected data after 440 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[440]5.7589,[441]5.7448,[442]5.7350,[443]5.7176,[444]5.6973,[445]5.6774,[446]5.6583,[447]5.6380,[448]5.6184,[449]5.6014,
+save_imatrix: stored collected data after 450 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[450]5.5817,[451]5.5612,[452]5.5437,[453]5.5384,[454]5.5190,[455]5.5100,[456]5.4958,[457]5.4920,[458]5.4802,[459]5.4682,
+save_imatrix: stored collected data after 460 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[460]5.4573,[461]5.4457,[462]5.4346,[463]5.4232,[464]5.4120,[465]5.4009,[466]5.3901,[467]5.3787,[468]5.3711,[469]5.3609,
+save_imatrix: stored collected data after 470 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[470]5.3474,[471]5.3300,[472]5.3168,[473]5.3129,[474]5.3177,[475]5.2992,[476]5.2860,[477]5.2721,[478]5.2555,[479]5.2408,
+save_imatrix: stored collected data after 480 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[480]5.2253,[481]5.2161,[482]5.2046,[483]5.1984,[484]5.1886,[485]5.1801,[486]5.1738,[487]5.1648,[488]5.1541,[489]5.1513,
+save_imatrix: stored collected data after 490 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[490]5.1514,[491]5.1549,[492]5.1530,[493]5.1560,[494]5.1563,[495]5.1514,[496]5.1434,[497]5.1543,[498]5.1683,[499]5.1848,
+save_imatrix: stored collected data after 500 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[500]5.1953,[501]5.2065,[502]5.2191,[503]5.2329,[504]5.2433,[505]5.2608,[506]5.2734,[507]5.2856,[508]5.3107,[509]5.3367,
+save_imatrix: stored collected data after 510 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[510]5.3645,[511]5.3610,[512]5.3732,[513]5.3866,[514]5.3976,[515]5.4047,[516]5.4117,[517]5.4152,[518]5.4170,[519]5.4227,
+save_imatrix: stored collected data after 520 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[520]5.4288,[521]5.4392,[522]5.4388,[523]5.4370,[524]5.4467,[525]5.4737,[526]5.4977,[527]5.4991,[528]5.5006,[529]5.5045,
+save_imatrix: stored collected data after 530 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[530]5.5087,[531]5.5112,[532]5.5149,[533]5.5176,[534]5.5235,[535]5.5305,[536]5.5418,[537]5.5567,[538]5.5694,[539]5.5812,
+save_imatrix: stored collected data after 540 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[540]5.5890,[541]5.5946,[542]5.5998,[543]5.5990,[544]5.5940,[545]5.5885,[546]5.5926,[547]5.5955,[548]5.6019,[549]5.6052,
+save_imatrix: stored collected data after 550 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[550]5.6099,[551]5.6094,[552]5.6159,[553]5.6173,[554]5.6223,[555]5.6189,[556]5.6184,[557]5.6153,[558]5.6072,[559]5.6035,
+save_imatrix: stored collected data after 560 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[560]5.5952,[561]5.5850,[562]5.5765,[563]5.5694,[564]5.5621,[565]5.5584,[566]5.5569,[567]5.5523,[568]5.5545,[569]5.5518,
+save_imatrix: stored collected data after 570 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[570]5.5495,[571]5.5477,[572]5.5461,[573]5.5401,[574]5.5403,[575]5.5353,[576]5.5288,[577]5.5269,[578]5.5260,[579]5.5125,
+save_imatrix: stored collected data after 580 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[580]5.5022,[581]5.4917,[582]5.4798,[583]5.4773,[584]5.4788,[585]5.4795,[586]5.4754,[587]5.4720,[588]5.4653,[589]5.4633,
+save_imatrix: stored collected data after 590 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[590]5.4647,[591]5.4612,[592]5.4558,[593]5.4541,[594]5.4512,[595]5.4510,[596]5.4495,[597]5.4464,[598]5.4415,[599]5.4404,
+save_imatrix: stored collected data after 600 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[600]5.4267,[601]5.4170,[602]5.4070,[603]5.3969,[604]5.3890,[605]5.3797,[606]5.3675,[607]5.3527,[608]5.3380,[609]5.3235,
+save_imatrix: stored collected data after 610 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[610]5.3214,[611]5.3129,[612]5.3097,[613]5.2994,[614]5.2975,[615]5.2898,[616]5.2938,[617]5.2908,[618]5.2983,[619]5.2924,
+save_imatrix: stored collected data after 620 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[620]5.2887,[621]5.2822,[622]5.2772,[623]5.2731,[624]5.2681,[625]5.2660,[626]5.2614,[627]5.2570,[628]5.2562,[629]5.2632,
+save_imatrix: stored collected data after 630 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[630]5.2643,[631]5.2625,[632]5.2618,[633]5.2641,[634]5.2654,[635]5.2668,[636]5.2706,[637]5.2724,[638]5.2705,[639]5.2716,
+save_imatrix: stored collected data after 640 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[640]5.2677,[641]5.2722,[642]5.2740,[643]5.2776,[644]5.2780,[645]5.2779,[646]5.2791,[647]5.2754,[648]5.2646,[649]5.2578,
+save_imatrix: stored collected data after 650 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[650]5.2493,[651]5.2384,[652]5.2277,[653]5.2222,[654]5.2163,[655]5.2099,[656]5.1990,[657]5.1925,[658]5.1864,[659]5.1792,
+save_imatrix: stored collected data after 660 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[660]5.1691,[661]5.1586,[662]5.1509,[663]5.1418,[664]5.1383,[665]5.1308,[666]5.1227,[667]5.1125,[668]5.1089,[669]5.1003,
+save_imatrix: stored collected data after 670 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[670]5.0944,[671]5.0892,[672]5.0849,[673]5.0783,[674]5.0675,[675]5.0590,[676]5.0523,[677]5.0433,[678]5.0349,[679]5.0313,
+save_imatrix: stored collected data after 680 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[680]5.0272,[681]5.0244,[682]5.0225,[683]5.0190,[684]5.0163,[685]5.0119,[686]5.0102,[687]5.0088,[688]5.0057,[689]5.0013,
+save_imatrix: stored collected data after 690 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[690]4.9978,[691]4.9954,[692]4.9928,[693]4.9896,[694]4.9879,[695]4.9837,[696]4.9873,[697]4.9873,[698]4.9881,[699]4.9886,
+save_imatrix: stored collected data after 700 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[700]4.9902,[701]4.9908,[702]4.9918,[703]4.9929,[704]4.9944,[705]4.9963,[706]4.9955,[707]4.9966,[708]4.9981,[709]5.0003,
+save_imatrix: stored collected data after 710 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[710]5.0032,[711]5.0048,[712]5.0057,[713]5.0059,[714]5.0073,[715]5.0102,[716]5.0118,[717]5.0118,[718]5.0105,[719]5.0084,
+save_imatrix: stored collected data after 720 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[720]5.0099,[721]5.0128,[722]5.0140,[723]5.0153,[724]5.0171,[725]5.0186,[726]5.0179,[727]5.0184,[728]5.0193,[729]5.0207,
+save_imatrix: stored collected data after 730 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[730]5.0214,[731]5.0238,[732]5.0237,[733]5.0250,[734]5.0260,[735]5.0279,[736]5.0288,[737]5.0298,[738]5.0299,[739]5.0321,
+save_imatrix: stored collected data after 740 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[740]5.0345,[741]5.0353,[742]5.0359,[743]5.0366,[744]5.0367,[745]5.0388,[746]5.0391,[747]5.0407,[748]5.0407,[749]5.0418,
+save_imatrix: stored collected data after 750 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[750]5.0431,[751]5.0431,[752]5.0440,[753]5.0449,[754]5.0462,[755]5.0479,[756]5.0472,[757]5.0471,[758]5.0490,[759]5.0514,
+save_imatrix: stored collected data after 760 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[760]5.0541,[761]5.0539,[762]5.0545,[763]5.0543,[764]5.0539,[765]5.0543,[766]5.0549,[767]5.0556,[768]5.0554,[769]5.0577,
+save_imatrix: stored collected data after 770 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[770]5.0573,[771]5.0595,[772]5.0603,[773]5.0599,[774]5.0611,[775]5.0621,[776]5.0655,[777]5.0700,[778]5.0697,[779]5.0692,
+save_imatrix: stored collected data after 780 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[780]5.0694,[781]5.0733,[782]5.0741,[783]5.0736,[784]5.0749,[785]5.0777,[786]5.0794,[787]5.0801,[788]5.0802,[789]5.0806,
+save_imatrix: stored collected data after 790 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+[790]5.0808,[791]5.0803,[792]5.0819,[793]5.0830,[794]5.0830,[795]5.0832,[796]5.0844,
+save_imatrix: stored collected data after 796 chunks in /mnt/data/models/ubergarm/MiniMax-M2.5-GGUF/imatrix-MiniMax-M2.5-BF16.dat
+
+Final estimate: PPL = 5.0844 +/- 0.03016
+
+======================== sorted layer importances
+  0: Layer   0, <cos_sim> = 0.316882
+  1: Layer   1, <cos_sim> = 0.755875
+  2: Layer  61, <cos_sim> = 0.757509
+  3: Layer   4, <cos_sim> = 0.834586
+  4: Layer   5, <cos_sim> = 0.853398
+  5: Layer   3, <cos_sim> = 0.881666
+  6: Layer   2, <cos_sim> = 0.916932
+  7: Layer  60, <cos_sim> = 0.929488
+  8: Layer   6, <cos_sim> = 0.930659
+  9: Layer  27, <cos_sim> = 0.931451
+ 10: Layer  24, <cos_sim> = 0.937379
+ 11: Layer  31, <cos_sim> = 0.937715
+ 12: Layer   7, <cos_sim> = 0.938591
+ 13: Layer  28, <cos_sim> = 0.939637
+ 14: Layer  23, <cos_sim> = 0.941139
+ 15: Layer   8, <cos_sim> = 0.941754
+ 16: Layer  32, <cos_sim> = 0.942899
+ 17: Layer   9, <cos_sim> = 0.943956
+ 18: Layer  39, <cos_sim> = 0.943956
+ 19: Layer  26, <cos_sim> = 0.944061
+ 20: Layer  30, <cos_sim> = 0.945374
+ 21: Layer  25, <cos_sim> = 0.945891
+ 22: Layer  29, <cos_sim> = 0.946928
+ 23: Layer  37, <cos_sim> = 0.947441
+ 24: Layer  34, <cos_sim> = 0.949696
+ 25: Layer  38, <cos_sim> = 0.950203
+ 26: Layer  11, <cos_sim> = 0.950537
+ 27: Layer  18, <cos_sim> = 0.950985
+ 28: Layer  41, <cos_sim> = 0.951033
+ 29: Layer  15, <cos_sim> = 0.951085
+ 30: Layer  17, <cos_sim> = 0.951511
+ 31: Layer  35, <cos_sim> = 0.951801
+ 32: Layer  22, <cos_sim> = 0.95183
+ 33: Layer  36, <cos_sim> = 0.953065
+ 34: Layer  16, <cos_sim> = 0.953408
+ 35: Layer  43, <cos_sim> = 0.953567
+ 36: Layer  40, <cos_sim> = 0.953795
+ 37: Layer  10, <cos_sim> = 0.954455
+ 38: Layer  49, <cos_sim> = 0.954639
+ 39: Layer  12, <cos_sim> = 0.954709
+ 40: Layer  21, <cos_sim> = 0.954823
+ 41: Layer  33, <cos_sim> = 0.955162
+ 42: Layer  58, <cos_sim> = 0.955824
+ 43: Layer  46, <cos_sim> = 0.956251
+ 44: Layer  13, <cos_sim> = 0.957593
+ 45: Layer  59, <cos_sim> = 0.958925
+ 46: Layer  57, <cos_sim> = 0.960093
+ 47: Layer  19, <cos_sim> = 0.960544
+ 48: Layer  48, <cos_sim> = 0.961408
+ 49: Layer  42, <cos_sim> = 0.961889
+ 50: Layer  44, <cos_sim> = 0.962539
+ 51: Layer  45, <cos_sim> = 0.962865
+ 52: Layer  51, <cos_sim> = 0.96306
+ 53: Layer  54, <cos_sim> = 0.963524
+ 54: Layer  50, <cos_sim> = 0.9637
+ 55: Layer  20, <cos_sim> = 0.964397
+ 56: Layer  14, <cos_sim> = 0.964914
+ 57: Layer  52, <cos_sim> = 0.965439
+ 58: Layer  47, <cos_sim> = 0.965696
+ 59: Layer  56, <cos_sim> = 0.965783
+ 60: Layer  55, <cos_sim> = 0.967547
+ 61: Layer  53, <cos_sim> = 0.968062
+
+======================== sorted attention importances
+  0: Layer   0, <cos_sim> = 0.0704138
+  1: Layer   5, <cos_sim> = 0.322555
+  2: Layer   3, <cos_sim> = 0.367
+  3: Layer   2, <cos_sim> = 0.416204
+  4: Layer   4, <cos_sim> = 0.566873
+  5: Layer   7, <cos_sim> = 0.580611
+  6: Layer  11, <cos_sim> = 0.585547
+  7: Layer   6, <cos_sim> = 0.594568
+  8: Layer   9, <cos_sim> = 0.603303
+  9: Layer  12, <cos_sim> = 0.657143
+ 10: Layer   8, <cos_sim> = 0.664074
+ 11: Layer  15, <cos_sim> = 0.668341
+ 12: Layer  10, <cos_sim> = 0.670432
+ 13: Layer   1, <cos_sim> = 0.684174
+ 14: Layer  17, <cos_sim> = 0.686808
+ 15: Layer  14, <cos_sim> = 0.696922
+ 16: Layer  22, <cos_sim> = 0.704714
+ 17: Layer  23, <cos_sim> = 0.731919
+ 18: Layer  20, <cos_sim> = 0.73193
+ 19: Layer  21, <cos_sim> = 0.738054
+ 20: Layer  16, <cos_sim> = 0.738725
+ 21: Layer  18, <cos_sim> = 0.740968
+ 22: Layer  19, <cos_sim> = 0.758633
+ 23: Layer  13, <cos_sim> = 0.760839
+ 24: Layer  26, <cos_sim> = 0.791238
+ 25: Layer  24, <cos_sim> = 0.800043
+ 26: Layer  27, <cos_sim> = 0.809005
+ 27: Layer  25, <cos_sim> = 0.821364
+ 28: Layer  28, <cos_sim> = 0.82219
+ 29: Layer  29, <cos_sim> = 0.839471
+ 30: Layer  30, <cos_sim> = 0.845148
+ 31: Layer  32, <cos_sim> = 0.860903
+ 32: Layer  31, <cos_sim> = 0.861528
+ 33: Layer  33, <cos_sim> = 0.868407
+ 34: Layer  34, <cos_sim> = 0.880557
+ 35: Layer  36, <cos_sim> = 0.883921
+ 36: Layer  35, <cos_sim> = 0.892888
+ 37: Layer  37, <cos_sim> = 0.893964
+ 38: Layer  38, <cos_sim> = 0.896139
+ 39: Layer  39, <cos_sim> = 0.896584
+ 40: Layer  61, <cos_sim> = 0.897035
+ 41: Layer  40, <cos_sim> = 0.903845
+ 42: Layer  43, <cos_sim> = 0.923175
+ 43: Layer  41, <cos_sim> = 0.92698
+ 44: Layer  42, <cos_sim> = 0.936574
+ 45: Layer  49, <cos_sim> = 0.941276
+ 46: Layer  44, <cos_sim> = 0.948048
+ 47: Layer  45, <cos_sim> = 0.948974
+ 48: Layer  46, <cos_sim> = 0.951432
+ 49: Layer  47, <cos_sim> = 0.954076
+ 50: Layer  52, <cos_sim> = 0.955161
+ 51: Layer  48, <cos_sim> = 0.961256
+ 52: Layer  50, <cos_sim> = 0.961942
+ 53: Layer  60, <cos_sim> = 0.963453
+ 54: Layer  54, <cos_sim> = 0.966224
+ 55: Layer  51, <cos_sim> = 0.966641
+ 56: Layer  58, <cos_sim> = 0.966927
+ 57: Layer  53, <cos_sim> = 0.970254
+ 58: Layer  57, <cos_sim> = 0.97137
+ 59: Layer  56, <cos_sim> = 0.972376
+ 60: Layer  59, <cos_sim> = 0.973361
+ 61: Layer  55, <cos_sim> = 0.973994
+
+======================== sorted ffn importances
+  0: Layer   4, <cos_sim> = 0.314169
+  1: Layer   2, <cos_sim> = 0.461
+  2: Layer   3, <cos_sim> = 0.561104
+  3: Layer   6, <cos_sim> = 0.569159
+  4: Layer   5, <cos_sim> = 0.577845
+  5: Layer   8, <cos_sim> = 0.597897
+  6: Layer  10, <cos_sim> = 0.60102
+  7: Layer   0, <cos_sim> = 0.605144
+  8: Layer  11, <cos_sim> = 0.638799
+  9: Layer   9, <cos_sim> = 0.645062
+ 10: Layer  14, <cos_sim> = 0.649815
+ 11: Layer   7, <cos_sim> = 0.6679
+ 12: Layer  16, <cos_sim> = 0.680739
+ 13: Layer  13, <cos_sim> = 0.684857
+ 14: Layer  21, <cos_sim> = 0.687122
+ 15: Layer  19, <cos_sim> = 0.708914
+ 16: Layer   1, <cos_sim> = 0.711
+ 17: Layer  20, <cos_sim> = 0.719718
+ 18: Layer  22, <cos_sim> = 0.729699
+ 19: Layer  15, <cos_sim> = 0.732361
+ 20: Layer  17, <cos_sim> = 0.741397
+ 21: Layer  12, <cos_sim> = 0.742671
+ 22: Layer  18, <cos_sim> = 0.745961
+ 23: Layer  25, <cos_sim> = 0.777979
+ 24: Layer  23, <cos_sim> = 0.779894
+ 25: Layer  26, <cos_sim> = 0.795613
+ 26: Layer  24, <cos_sim> = 0.807908
+ 27: Layer  27, <cos_sim> = 0.809809
+ 28: Layer  29, <cos_sim> = 0.820023
+ 29: Layer  28, <cos_sim> = 0.822863
+ 30: Layer  30, <cos_sim> = 0.842264
+ 31: Layer  31, <cos_sim> = 0.846896
+ 32: Layer  32, <cos_sim> = 0.854584
+ 33: Layer  33, <cos_sim> = 0.866524
+ 34: Layer  35, <cos_sim> = 0.870263
+ 35: Layer  36, <cos_sim> = 0.882242
+ 36: Layer  34, <cos_sim> = 0.883365
+ 37: Layer  61, <cos_sim> = 0.883623
+ 38: Layer  39, <cos_sim> = 0.88929
+ 39: Layer  37, <cos_sim> = 0.891996
+ 40: Layer  38, <cos_sim> = 0.894879
+ 41: Layer  42, <cos_sim> = 0.908598
+ 42: Layer  40, <cos_sim> = 0.909085
+ 43: Layer  41, <cos_sim> = 0.917905
+ 44: Layer  48, <cos_sim> = 0.928901
+ 45: Layer  44, <cos_sim> = 0.929453
+ 46: Layer  43, <cos_sim> = 0.932476
+ 47: Layer  46, <cos_sim> = 0.935198
+ 48: Layer  45, <cos_sim> = 0.938768
+ 49: Layer  51, <cos_sim> = 0.93951
+ 50: Layer  49, <cos_sim> = 0.942412
+ 51: Layer  47, <cos_sim> = 0.945566
+ 52: Layer  50, <cos_sim> = 0.950281
+ 53: Layer  53, <cos_sim> = 0.951307
+ 54: Layer  54, <cos_sim> = 0.953679
+ 55: Layer  52, <cos_sim> = 0.953897
+ 56: Layer  56, <cos_sim> = 0.959154
+ 57: Layer  55, <cos_sim> = 0.959213
+ 58: Layer  60, <cos_sim> = 0.965233
+ 59: Layer  57, <cos_sim> = 0.966029
+ 60: Layer  58, <cos_sim> = 0.973854
+ 61: Layer  59, <cos_sim> = 0.975587
+
+llama_print_timings:        load time =  155550.25 ms
+llama_print_timings:      sample time =       0.00 ms /     1 runs   (    0.00 ms per token,      inf tokens per second)
+llama_print_timings: prompt eval time = 3025726.55 ms / 407552 tokens (    7.42 ms per token,   134.70 tokens per second)
+llama_print_timings:        eval time =       0.00 ms /     1 runs   (    0.00 ms per token,      inf tokens per second)
+llama_print_timings:       total time = 3192463.99 ms / 407553 tokens