Instructions to use YOYO-AI/Qwen2.5-14B-YOYO-super with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use YOYO-AI/Qwen2.5-14B-YOYO-super with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="YOYO-AI/Qwen2.5-14B-YOYO-super")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("YOYO-AI/Qwen2.5-14B-YOYO-super")
model = AutoModelForCausalLM.from_pretrained("YOYO-AI/Qwen2.5-14B-YOYO-super", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use YOYO-AI/Qwen2.5-14B-YOYO-super with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "YOYO-AI/Qwen2.5-14B-YOYO-super"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "YOYO-AI/Qwen2.5-14B-YOYO-super",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/YOYO-AI/Qwen2.5-14B-YOYO-super

SGLang

How to use YOYO-AI/Qwen2.5-14B-YOYO-super with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "YOYO-AI/Qwen2.5-14B-YOYO-super" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "YOYO-AI/Qwen2.5-14B-YOYO-super",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "YOYO-AI/Qwen2.5-14B-YOYO-super" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "YOYO-AI/Qwen2.5-14B-YOYO-super",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use YOYO-AI/Qwen2.5-14B-YOYO-super with Docker Model Runner:
```
docker model run hf.co/YOYO-AI/Qwen2.5-14B-YOYO-super
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

Achieve the Optimal Merged Model by Using One Basic Model and Two Fine-tuned Models!

What is the best way to merge one base model and two fine-tuned models?

This might be the best answer at the present stage!

Qwen2.5-7B-YOYO-super

Qwen2.5-14B-YOYO-super

This is not a whim release, but the optimal result of countless merging experiments!

Here is the formula for the previous generation:

models:  
  - model: Qwen/Qwen2.5-14B-Instruct  
    parameters:  
      density: 1  
      weight: 1  
      lambda: 0.9
  - model: Qwen/Qwen2.5-14B-Instruct-1M  
    parameters:  
      density: 1  
      weight: 1  
      lambda: 0.9  
merge_method: della  
base_model: Qwen/Qwen2.5-14B  
parameters:  
  density: 1  
  weight: 1  
  lambda: 0.9  
  normalize: true  
  int8_mask: true  
dtype: bfloat16  
tokenizer_source: base

It was widely used in the merging process of the previous generation of models.

However, there are some deficiencies:

1.There is relatively little retention of knowledge of the basic model.

2.The mathematical and coding abilities have declined.

And here is the formula for this generation:

models:
  - model: Qwen/Qwen2.5-14B-instruct
    parameters:
      density: 1 
      weight: 1
      lambda: 0.9
merge_method: della
base_model: Qwen/Qwen2.5-14B
parameters:
  density: 1
  weight: 1
  lambda: 0.9
  normalize: true
  int8_mask: true
dtype: float16
tokenizer_source: base
name: Qwen2.5-14B-della

models:
  - model: Qwen/Qwen2.5-14B-instruct-1M
    parameters:
      density: 1 
      weight: 1
      lambda: 0.9
merge_method: della
base_model: Qwen/Qwen2.5-14B
parameters:
  density: 1
  weight: 1
  lambda: 0.9
  normalize: true
  int8_mask: true
dtype: float16
tokenizer_source: base
name: Qwen2.5-14B-della-1M

models:
  - model: Qwen/Qwen2.5-14B-instruct
    parameters:
      density: 1 
      weight: 1
merge_method: ties
base_model: Qwen/Qwen2.5-14B
parameters:
  density: 1
  weight: 1
  normalize: true
  int8_mask: true
dtype: float16
tokenizer_source: base
name: Qwen2.5-14B-ties

models:
  - model: Qwen/Qwen2.5-14B-instruct-1M
    parameters:
      density: 1 
      weight: 1
merge_method: ties
base_model: Qwen/Qwen2.5-14B
parameters:
  density: 1
  weight: 1
  normalize: true
  int8_mask: true
dtype: float16
tokenizer_source: base
name: Qwen2.5-14B-ties-1M

merge_method: model_stock
base_model: Qwen/Qwen2.5-14B
models:
  - model: mergekit-community/Qwen2.5-14B-della
  - model: mergekit-community/Qwen2.5-14B-della-1M
  - model: mergekit-community/Qwen2.5-14B-ties
  - model: mergekit-community/Qwen2.5-14B-ties-1M
  - model: Qwen/Qwen2.5-14B-instruct-1M
  - model: Qwen/Qwen2.5-14B-instruct
tokenizer_source: base
int8_mask: true
normalize: true
dtype: float16

Except for a slight decrease in instruction following, significant improvements have been achieved in all other aspects.

This formula will also be used in the development of the next generation of YOYO models.

YOYO-AI not only releases merged models with excellent performance but also publishes a complete and high-quality model merging formula, hoping to promote the progress of model merging technology in the open-source community with this!