Naphula-Archives/Orochi-24B-v0-cp6-readme

🐉 Orochi 24B v0

Released: September 2025

Orochi (Test44) represents an experimental merging of 5 merge methods into one.

Orochi is fully uncensored, while creativity may vary depending on checkpoints.

The theory is to compare all the checkpoints, to see is certain merge methods work better than others for certain tasks, and to see if multi-stage merge stacking has any effect (positive or negative).

Safetensors for checkpoint 6 are being uploaded as they are the most time consuming to reproduce.

4 models are featured:

• TheDrummer/Cydonia-24B-v2
• TroyDoesAI/BlackSheep-24B
• dphn/Dolphin-Mistral-24B-Venice-Edition
• huihui-ai/Mistral-Small-24B-Instruct-2501-abliterated

GGUF index:

• Orochi-24B-v0-cp0-Q6_K.gguf | dare_ties
• Orochi-24B-v0-cp1-Q6_K.gguf | karcher
• Orochi-24B-v0-cp2-Q6_K.gguf | della_linear
• Orochi-24B-v0-cp3-Q6_K.gguf | breadcrumbs
• Orochi-24B-v0-cp4-Q6_K.gguf | sce
• Orochi-24B-v0-cp5-Q6_K.gguf | model_stock
• Orochi-24B-v0-cp6-Q6_K.gguf | dare_ties
• Orochi-24B-v0-cp7-Q6_K.gguf | nuslerp

Stage 0 (checkpoint 0):

• This is a balanced DARE_TIES of the 4 models. It is meant to be used as a baseline comparison to the other merges, especially checkpoint 6.

Stage 1 (checkpoints 1-5):

• The 4 models are merged via the following 5 methods:

1. karcher
2. della_linear
3. breadcrumbs
4. sce (top_k 1.0)
5. model_stock

Stage 2 (checkpoint 6):

• A DARE_TIES merge of Stage 1 merges. The theory is to test to see if a DARE_TIES merge of the Stage 1 merges had any noticeable improvements or degradations compared to Stage 0.

Stage 3 (checkpoint 7):

• NUSLERP merge of cp_0 with cp_6. The theory is to test if there is improvement or degradation.

Abandoned Checkpoints:

• Failed Merge (checkpoint 8): SCE with top_k set to 0.25 resulted in unmergeable safetensors, and while it quanted, the GGUF was unusable too. This was originally meant to be checkpoint 4 but was swapped out after merge-kit failed on Stage 2.
• Failed Merge (checkpoint 9): A stage 3 attempt to use arcee_fusion instead of nuslerp, which resulted in a safetensors twice as large.

Notes:

• dare_linear -> della = Applies precision to randomness (Incorrect Pipeline). You are asking a surgical tool (della) to operate on a model where parameter magnitudes are the result of a random process (dare_linear). The tool's fundamental assumption—that magnitude equals importance—is broken, making the pipeline conceptually flawed.

• della_linear -> dare_ties = Applies randomness to precision (Correct Pipeline). You are first creating a high-quality, precise "expert" model where magnitudes do correlate with importance (della_linear). Then, you use a robust, unbiased method (dare_ties) to randomly sample from this expert and others, safely integrating them while resolving conflicts. This pipeline is logical and synergistic.

config.yaml

base_model: huihui-ai/Mistral-Small-24B-Instruct-2501-abliterated
merge_method: dare_ties
architecture: MistralForCausalLM
dtype: bfloat16
models:
  - model: huihui-ai/Mistral-Small-24B-Instruct-2501-abliterated
    parameters:
      density: 0.5
      weight: 0.25
  - model: dphn/Dolphin-Mistral-24B-Venice-Edition
    parameters:
      density: 0.5
      weight: 0.25
  - model: TroyDoesAI/BlackSheep-24B
    parameters:
      density: 0.5
      weight: 0.25
  - model: TheDrummer/Cydonia-24B-v2
    parameters:
      density: 0.5
      weight: 0.25
tokenizer:
source: union
chat_template: auto

merge_method: karcher
architecture: MistralForCausalLM
dtype: bfloat16
models:
  - model: huihui-ai/Mistral-Small-24B-Instruct-2501-abliterated
  - model: dphn/Dolphin-Mistral-24B-Venice-Edition
  - model: TroyDoesAI/BlackSheep-24B
  - model: TheDrummer/Cydonia-24B-v2
tokenizer:
source: union
chat_template: auto

base_model: huihui-ai/Mistral-Small-24B-Instruct-2501-abliterated
merge_method: della_linear
architecture: MistralForCausalLM
dtype: bfloat16
models:
  - model: huihui-ai/Mistral-Small-24B-Instruct-2501-abliterated
    parameters:
      weight: 0.25
      density: 0.8
      epsilon: 0.1
  - model: dphn/Dolphin-Mistral-24B-Venice-Edition
    parameters:
      weight: 0.25
      density: 0.8
      epsilon: 0.1
  - model: TroyDoesAI/BlackSheep-24B
    parameters:
      weight: 0.25
      density: 0.8
      epsilon: 0.1
  - model: TheDrummer/Cydonia-24B-v2
    parameters:
      weight: 0.25
      density: 0.8
      epsilon: 0.1
tokenizer:
source: union
chat_template: auto

base_model: huihui-ai/Mistral-Small-24B-Instruct-2501-abliterated
merge_method: breadcrumbs
architecture: MistralForCausalLM
dtype: bfloat16
models:
  - model: huihui-ai/Mistral-Small-24B-Instruct-2501-abliterated
    parameters:
      weight: 0.25
      density: 0.9
      gamma: 0.01
  - model: dphn/Dolphin-Mistral-24B-Venice-Edition
    parameters:
      weight: 0.25
      density: 0.9
      gamma: 0.01
  - model: TroyDoesAI/BlackSheep-24B
    parameters:
      weight: 0.25
      density: 0.9
      gamma: 0.01
  - model: TheDrummer/Cydonia-24B-v2
    parameters:
      weight: 0.25
      density: 0.9
      gamma: 0.01
tokenizer:
source: union
chat_template: auto

base_model: huihui-ai/Mistral-Small-24B-Instruct-2501-abliterated
architecture: MistralForCausalLM
models:
  - model: huihui-ai/Mistral-Small-24B-Instruct-2501-abliterated
  - model: dphn/Dolphin-Mistral-24B-Venice-Edition
  - model: TroyDoesAI/BlackSheep-24B
  - model: TheDrummer/Cydonia-24B-v2
merge_method: sce
dtype: bfloat16
parameters:
  normalize: true
  select_topk: 1.0
tokenizer:
source: union
chat_template: auto

base_model: huihui-ai/Mistral-Small-24B-Instruct-2501-abliterated
architecture: MistralForCausalLM
models:
  - model: huihui-ai/Mistral-Small-24B-Instruct-2501-abliterated
  - model: dphn/Dolphin-Mistral-24B-Venice-Edition
  - model: TroyDoesAI/BlackSheep-24B
  - model: TheDrummer/Cydonia-24B-v2
merge_method: model_stock
dtype: bfloat16
tokenizer:
source: union
chat_template: auto

base_model: huihui-ai/Mistral-Small-24B-Instruct-2501-abliterated
merge_method: dare_ties
architecture: MistralForCausalLM
dtype: bfloat16
models:
  - model: A:\LLM\24B\Test44-24B-v0-cp1
    parameters:
      density: 0.5
      weight: 0.2
  - model: A:\LLM\24B\Test44-24B-v0-cp2
    parameters:
      density: 0.5
      weight: 0.2
  - model: A:\LLM\24B\Test44-24B-v0-cp3
    parameters:
      density: 0.5
      weight: 0.2
  - model: A:\LLM\24B\Test44-24B-v0-cp4
    parameters:
      density: 0.5
      weight: 0.2
  - model: A:\LLM\24B\Test44-24B-v0-cp5
    parameters:
      density: 0.5
      weight: 0.2
tokenizer:
source: union
chat_template: auto
name: Orochi-24B-v0-cp6

models:
  - model: A:\LLM\24B\Test44-24B-v0-cp6
    parameters:
      weight: 1
  - model: A:\LLM\24B\Test44-24B-v0-cp0
    parameters:
      weight: 1
merge_method: nuslerp
base_model: huihui-ai/Mistral-Small-24B-Instruct-2501-abliterated
tokenizer_source: A:\LLM\24B\Test44-24B-v0-cp6
parameters:
  normalize: true
  int8_mask: false
dtype: float32
out_dtype: bfloat16

base_model: A:\LLM\24B\Test44-24B-v0-cp6
architecture: MistralForCausalLM
models:
  - model: A:\LLM\24B\Test44-24B-v0-cp0
  - model: A:\LLM\24B\Test44-24B-v0-cp6
merge_method: arcee_fusion
dtype: bfloat16
tokenizer:
source: union
chat_template: auto

base_model: huihui-ai/Mistral-Small-24B-Instruct-2501-abliterated
architecture: MistralForCausalLM
models:
  - model: huihui-ai/Mistral-Small-24B-Instruct-2501-abliterated
  - model: dphn/Dolphin-Mistral-24B-Venice-Edition
  - model: TroyDoesAI/BlackSheep-24B
  - model: TheDrummer/Cydonia-24B-v2
merge_method: sce
dtype: bfloat16
parameters:
  normalize: true
  select_topk: 0.25
tokenizer:
source: union
chat_template: auto