Hugging Face's logo

Naphula-Archives
/
Qliphoth-24B-v2-Prototypes-Q4_0-GGUF

GGUF
Model card Files
xet
 Community
How to use from
llama.cpp
Install (macOS, Linux) 
curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Naphula-Archives/Qliphoth-24B-v2-Prototypes-Q4_0-GGUF:Q4_0
# Run inference directly in the terminal:
llama cli -hf Naphula-Archives/Qliphoth-24B-v2-Prototypes-Q4_0-GGUF:Q4_0
Install from WinGet (Windows) 
winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Naphula-Archives/Qliphoth-24B-v2-Prototypes-Q4_0-GGUF:Q4_0
# Run inference directly in the terminal:
llama cli -hf Naphula-Archives/Qliphoth-24B-v2-Prototypes-Q4_0-GGUF:Q4_0
Use pre-built binary 
# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf Naphula-Archives/Qliphoth-24B-v2-Prototypes-Q4_0-GGUF:Q4_0
# Run inference directly in the terminal:
./llama-cli -hf Naphula-Archives/Qliphoth-24B-v2-Prototypes-Q4_0-GGUF:Q4_0
Build from source code 
git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf Naphula-Archives/Qliphoth-24B-v2-Prototypes-Q4_0-GGUF:Q4_0
# Run inference directly in the terminal:
./build/bin/llama-cli -hf Naphula-Archives/Qliphoth-24B-v2-Prototypes-Q4_0-GGUF:Q4_0
Use Docker 
docker model run hf.co/Naphula-Archives/Qliphoth-24B-v2-Prototypes-Q4_0-GGUF:Q4_0
Quick Links

YAML Metadata Warning:empty or missing yaml metadata in repo card

Check out the documentation for more information.

Qliphoth has diverged from the Cthulhu/Goetia line as of v2 into its branch. It is now also a custom merge method. merge_method: qliphoth is very experimental and builds upon the previous magic method's "Aikido Flip" concept.

v1 was broken but v2 is stable and is being optimized. They should all be fully uncensored.

Update: None of these are fully uncensored, there are some refusals. A new version is coming soon

Uploading prototypes of v2 as Q4_0 GGUF since these are fastest to quantize

  • v2a had an imbalanced yaml on accident, it might be too "loud"
  • v2b is quite balanced and creative
  • v2c was a somewhat unsuccesful attempt to boost intelligence and creativity
  • v2d is another attempt to allow higher deviations (novelty), it seems better than v2c
  • v2e is an expansion attempt to utilize twice as many donors. This version seems like the smartest but unfortunately has refusals (likely due to the readyart 2501 models).
  • v2f is the next attempt to find an optimal middle ground between v2d and v2e [IN PROGRESS]
Downloads last month
59
GGUF
Model size
24B params
Architecture
llama
Hardware compatibility
Log In to add your hardware

4-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support