MikeRoz commited on
Commit
8ae1d06
·
verified ·
1 Parent(s): 9990355

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -0
README.md CHANGED
@@ -21,6 +21,7 @@ Note that tensor parallelism is not currently supported for this architecture, s
21
  | [2.50 bpw h6](https://huggingface.co/MikeRoz/MiniMax-M2.5-exl3/tree/2.50bpw_H6) (optimized) | 67.838 GiB | 0.30152 | 8.88802 | 4x24GB w/ 90112 FP16 context |
22
  | [3.00 bpw h6](https://huggingface.co/MikeRoz/MiniMax-M2.5-exl3/tree/3.00bpw_H6) | 81.613 GiB | 0.17263 | 8.58626 | 4x24GB w/ 16384 FP16 context |
23
  | [3.06 bpw h6](https://huggingface.co/MikeRoz/MiniMax-M2.5-exl3/tree/3.06bpw_H6) (optimized) | 82.656 GiB | 0.15648 | 8.66856 | 4x24GB w/ 12288 FP16 context |
 
24
  | [4.00 bpw h6](https://huggingface.co/MikeRoz/MiniMax-M2.5-exl3/tree/4.00bpw_H6) | 108.087 GiB | 0.07882 | 8.45404 | 6x24GB w/ 49152 FP16 context |
25
  | [5.00 bpw h6](https://huggingface.co/MikeRoz/MiniMax-M2.5-exl3/tree/5.00bpw_H6) | 134.561 GiB | - | - | 5x24GB + 1x32GB w/ 24576 FP16 context (will not load for me with 6x24GB) |
26
 
 
21
  | [2.50 bpw h6](https://huggingface.co/MikeRoz/MiniMax-M2.5-exl3/tree/2.50bpw_H6) (optimized) | 67.838 GiB | 0.30152 | 8.88802 | 4x24GB w/ 90112 FP16 context |
22
  | [3.00 bpw h6](https://huggingface.co/MikeRoz/MiniMax-M2.5-exl3/tree/3.00bpw_H6) | 81.613 GiB | 0.17263 | 8.58626 | 4x24GB w/ 16384 FP16 context |
23
  | [3.06 bpw h6](https://huggingface.co/MikeRoz/MiniMax-M2.5-exl3/tree/3.06bpw_H6) (optimized) | 82.656 GiB | 0.15648 | 8.66856 | 4x24GB w/ 12288 FP16 context |
24
+ | [3.50 bpw h6](https://huggingface.co/MikeRoz/MiniMax-M2.5-exl3/tree/3.50bpw_H6) (optimized) | 94.328 GiB | 0.12513 | 8.58743 | 5x24 GB w/ 49152 FP16 context |
25
  | [4.00 bpw h6](https://huggingface.co/MikeRoz/MiniMax-M2.5-exl3/tree/4.00bpw_H6) | 108.087 GiB | 0.07882 | 8.45404 | 6x24GB w/ 49152 FP16 context |
26
  | [5.00 bpw h6](https://huggingface.co/MikeRoz/MiniMax-M2.5-exl3/tree/5.00bpw_H6) | 134.561 GiB | - | - | 5x24GB + 1x32GB w/ 24576 FP16 context (will not load for me with 6x24GB) |
27