The training methodology of this listing definitely made Qwen3.6 27B into something smart enough to be usable day-to-day, and I'm curious to see if it would work even better on medium sized models. Any chance of a variant of this that uses Qwen3.5 122B as a baseline? Or perhaps MTP + MXFP4/NVFP4 variants for DGX Spark / Strix Halo users? This model currently runs at 4 tk/s on Spark, but would be able to reach ~25-30 tk/s with MTP + MXFP4/NVFP4.
To clarify, I'm not specifically talking about the claude distill aspect. From my perspective this specific model's fine tuning strategy seems to be the best I've seen. The combination of heretic + unsloth enhancements + imatrix (di-imatrix neo-coder) + distilled (claude) + increased/new model paramssignificantly improved software dev reasoning capabilities better than any model (or quant) I've tested that can fit >=200K context in 128 GB of memory.
In case anyone is curious, within my real-world "torture test" (requires long-context reasoning, agentic tool use, knowledge, and instruction following) this model performed best with temperature 0.4, top-k 20, top-p 0.95, min-p 0, presence-penalty 0, repeat-penalty 1.05, cache-type f16, flash-attn 'on' and my derivative of the APEX system prompt. The only oddity I've run into is that it seems to be a bit too brief when being told to perform context compaction, so some fine-details of the original prompt can get lost within the compaction process. Possibly fixable through some additional system prompt tuning.
To clarify, I'm not specifically talking about the claude distill aspect. From my perspective this specific model's fine tuning strategy seems to be the best I've seen. The combination of heretic + unsloth enhancements + imatrix (di-imatrix neo-coder) + distilled (claude) + increased/new model paramssignificantly improved software dev reasoning capabilities better than any model (or quant) I've tested that can fit >=200K context in 128 GB of memory.
In case anyone is curious, within my real-world "torture test" (requires long-context reasoning, agentic tool use, knowledge, and instruction following) this model performed best with temperature 0.4, top-k 20, top-p 0.95, min-p 0, presence-penality 0, repeat-penality 1.05, cache-type f16, flash-attn 'on' and my derivative of the APEX system prompt. The only oddity I've run into is that it seems to be a bit too brief when being told to perform context compaction, so some fine-details of the original prompt can get lost within the compaction process. Possibly fixable through some additional system prompt tuning.
For me, The Qwen3.6-40B-Deck-Opus-NEO-CODE-HERE-2T-OT-HIGH-Q8_0.gguf (which is linked as the q8 version of this model under the model card) seems to loose track and stat outputting nonsense fot me after around 20000 tokens. anyone know what the problem might be ? (it is a version that was dropped 25 days ago).
Can you create another discussion thread for that? I have an answer typed out, but future users are unlikely to dig into this thread to find my answer.
