Slight Disappointment
While it does decent output on par with other 70B models, for whatever reason i was getting a bunch of output that felt a little.... off... Not sure why but feels a little wooden, more bland. In the RP i asked a question and got from 3 NPC's identical output like it was a per-memorized script to sell me on something without answering the damn question. Eventually in the RP i finished it by going murder hobo.
I was using Q3_K and Q4_K_M quants so maybe that was the real problem; or it being a SOAR maybe i don't have it configured right in koboldcpp; Comparing it against my baseline of AnubisLemonade (or Omega Directive) this one feels like it falls a little short.
SOAR (Self-improving Operators for Automated program Refinements) is mainly for solving highly complex problems and requires quite a specific setup. During the "Evolutionary Search (Sample & Refine)" step the model generates thousands of python programs to solve a given problem. It then tests them and uses the LLM again to try to improve the most promising ones. In the final "Learning from Hindsight" step, it takes all previously generated programs and uses them as training data to train itself. You then just let it run and let the model keep improving itself and it eventually gets extremely good at the tasks you want it to solve. For more information, please read the original model card under https://huggingface.co/julien31/Soar-qwen-72b
I unfortunately don't think SOAR is well suited for roleplay use cases as being good at roleplay is not a problem that can be solved programmatically. By the way you are not the only one that are is kind of misusing this model. I myself am mainly using it for Q&A as setting up SOAR is a pain and it takes forever and a ton of GPU resources for it to finetune itself.
I unfortunately don't think SOAR is well suited for roleplay use cases as being good at roleplay is not a problem that can be solved programmatically.
Gotcha, so it's not just me. I understand that SOAR models are suppose to self improve, but probably don't have the right setup for them all yet. It just didn't occur to me it was a SOAR model til after the fact. Honestly on a number of tasks it works on par with other 70B models just fine (analysis, basic logic, character generation, etc).
I get the feeling like with some other models, i won't have the setup or need to use them how they are intended (and far better/improved models will replace them).
Anyways the abliterated part seems to work fine, i haven't had the model reject me with hardly anything.