Parameters for Llama?

#8
by Ericreator - opened

This is what I'm using for code, not sure it's totally optimal though. Any recommendations?

llama-server -m "Qwen3.6-27B-Uncensored-HauhauCS-Aggressive-Q5_K_P.gguf" --jinja --host 0.0.0.0 --port 8080 --n-gpu-layers 999 --ctx-size 32768 --temp 0.6 --top-p 0.95 --top-k 20 --min-p 0.0 --presence-penalty 1.5 -fa on --chat-template-kwargs "{"preserve_thinking":true}"

This is what I'm using for code, not sure it's totally optimal though. Any recommendations?

llama-server -m "Qwen3.6-27B-Uncensored-HauhauCS-Aggressive-Q5_K_P.gguf" --jinja --host 0.0.0.0 --port 8080 --n-gpu-layers 999 --ctx-size 32768 --temp 0.6 --top-p 0.95 --top-k 20 --min-p 0.0 --presence-penalty 1.5 -fa on --chat-template-kwargs "{"preserve_thinking":true}"

Hey, I'm about to re-upload better versions :)
https://huggingface.co/HauhauCS/Qwen3.6-27B-Uncensored-HauhauCS-Balanced Currently uploading Balanced version (read the repo for more details on it). Currently manually testing the new aggressive variant which will replace this release.

Sign up or log in to comment