Works well converted to GGUF Q8_0
#1
by
Joseph717171
- opened
Just tried your FP8-Dynamic of Llama-3.1-8B-Instruct. Its good. I converted it to GGUF Q8_0; it ran great. Can't wait to try the FP8-Block version. 😋
Just tried your FP8-Dynamic of Llama-3.1-8B-Instruct. Its good. I converted it to GGUF Q8_0; it ran great. Can't wait to try the FP8-Block version. 😋