Instructions to use matt-hans93/ups-tools-qwen2.5-7b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use matt-hans93/ups-tools-qwen2.5-7b with PEFT:

from peft import PeftModel
from transformers import AutoModelForCausalLM

base_model = AutoModelForCausalLM.from_pretrained("models/ups-tools-v3-merged")
model = PeftModel.from_pretrained(base_model, "matt-hans93/ups-tools-qwen2.5-7b")

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use matt-hans93/ups-tools-qwen2.5-7b with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf matt-hans93/ups-tools-qwen2.5-7b:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf matt-hans93/ups-tools-qwen2.5-7b:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf matt-hans93/ups-tools-qwen2.5-7b:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf matt-hans93/ups-tools-qwen2.5-7b:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf matt-hans93/ups-tools-qwen2.5-7b:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf matt-hans93/ups-tools-qwen2.5-7b:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf matt-hans93/ups-tools-qwen2.5-7b:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf matt-hans93/ups-tools-qwen2.5-7b:Q4_K_M

Use Docker

docker model run hf.co/matt-hans93/ups-tools-qwen2.5-7b:Q4_K_M

LM Studio
Jan

vLLM

How to use matt-hans93/ups-tools-qwen2.5-7b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "matt-hans93/ups-tools-qwen2.5-7b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "matt-hans93/ups-tools-qwen2.5-7b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/matt-hans93/ups-tools-qwen2.5-7b:Q4_K_M

Ollama
How to use matt-hans93/ups-tools-qwen2.5-7b with Ollama:
```
ollama run hf.co/matt-hans93/ups-tools-qwen2.5-7b:Q4_K_M
```

Unsloth Studio

How to use matt-hans93/ups-tools-qwen2.5-7b with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for matt-hans93/ups-tools-qwen2.5-7b to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for matt-hans93/ups-tools-qwen2.5-7b to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for matt-hans93/ups-tools-qwen2.5-7b to start chatting

How to use matt-hans93/ups-tools-qwen2.5-7b with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf matt-hans93/ups-tools-qwen2.5-7b:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "matt-hans93/ups-tools-qwen2.5-7b:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use matt-hans93/ups-tools-qwen2.5-7b with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf matt-hans93/ups-tools-qwen2.5-7b:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default matt-hans93/ups-tools-qwen2.5-7b:Q4_K_M

Run Hermes

hermes

OpenClaw new

How to use matt-hans93/ups-tools-qwen2.5-7b with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf matt-hans93/ups-tools-qwen2.5-7b:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "matt-hans93/ups-tools-qwen2.5-7b:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Docker Model Runner
How to use matt-hans93/ups-tools-qwen2.5-7b with Docker Model Runner:
```
docker model run hf.co/matt-hans93/ups-tools-qwen2.5-7b:Q4_K_M
```

Lemonade

How to use matt-hans93/ups-tools-qwen2.5-7b with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull matt-hans93/ups-tools-qwen2.5-7b:Q4_K_M

Run and chat with the model

lemonade run user.ups-tools-qwen2.5-7b-Q4_K_M

List all available models

lemonade list

Atomic Chat

UPS Tool-Use Qwen2.5 7B (v5a)

UPS Tool-Use Qwen2.5 7B is a QLoRA fine-tune of Qwen2.5-7B-Instruct for generating structured Hermes function-calling payloads for a UPS MCP server. It maps natural-language shipping and logistics requests to 18 UPS MCP tools covering tracking, rating, shipment creation, pickup workflows, address validation, paperless documents, landed cost, and location lookup.

This is the v5a release, trained on top of the v3 champion with replay + targeted augmentation data. It achieves 100% held-out accuracy and improves all stress-test axes over the prior release.

This repository publishes the PEFT LoRA adapter at the repo root and a Q4_K_M GGUF export for local inference with llama.cpp-compatible runtimes such as Ollama.

This model is not affiliated with or endorsed by UPS.

Model Details

Developed by: Matthew Hans
Model type: Qwen2.5-7B-Instruct causal language model with LoRA adapter fine-tuning
Base model: Qwen/Qwen2.5-7B-Instruct
Training base: unsloth/Qwen2.5-7B-Instruct-bnb-4bit
Fine-tuning method: QLoRA with rank-stabilized LoRA (rsLoRA)
Language: English
Output format: Hermes FC v1 tool call blocks
Release: v5a (May 2026)

Intended Use

The model is intended to sit behind a UPS MCP server and produce one structured tool call for each user request. A downstream orchestrator should parse the <tool_call_> block, validate arguments against the MCP tool schema, and then decide whether to call the UPS API.

<tool_call_>
{"name": "track_package", "arguments": {"inquiryNumber": "1Z999AA10123456784"}}
</tool_call_>

The system prompt should include the complete set of UPS MCP tool definitions in Hermes-compatible JSON:

<tools>
[{"type": "function", "function": {"name": "track_package", "parameters": {...}}}]
</tools>

Supported UPS MCP Tools

Tool	Category
`track_package`	Tracking
`validate_address`	Address validation
`rate_shipment`	Rating
`create_shipment`	Shipping
`void_shipment`	Shipping
`recover_label`	Shipping
`get_time_in_transit`	Transit
`get_landed_cost_quote`	International landed cost
`upload_paperless_document`	Paperless documents
`push_document_to_shipment`	Paperless documents
`delete_paperless_document`	Paperless documents
`find_locations`	Locator
`rate_pickup`	Pickup
`schedule_pickup`	Pickup
`cancel_pickup`	Pickup
`get_pickup_status`	Pickup
`get_political_divisions`	Pickup metadata
`get_service_center_facilities`	Pickup metadata

Evaluation

Held-out accuracy (122 test cases, disjoint from training)

Metric	Score
Tool selection accuracy	100%
Parameter completeness	100%
JSON validity rate	100%
Format compliance	100%
Payload structural accuracy	99.06%

Stress-test suite (~2,000+ inference calls across 9 axes)

Four graders run in parallel on every case:

Shape grader — required keys present in the tool call
Value-grounded grader — argument values match what the prompt said
Nested-structural grader — deep paths populated correctly
Schema-typed grader — types and enums match the MCP schema

Confidence intervals are Wilson 95%.

Axis	n	Shape	Value-grounded	Silent hallucination
Strict replay (templated)	900	95.22%	93.78%	1.78%
OOD-lexicon (disjoint vocab)	360	99.17%	98.06%	1.11%
Paraphrase (voice/email/sms/typo)	360	96.67%	68.61%	28.06%
Sibling-tool ambiguity	120	88.33%	79.17%	15.00%
Natural-language (real phrasing)	100	95.00%	82.00%	13.00%
Negative / refusal	80	—	—	— (86.25% pass)
Multi-turn workflows	30	—	—	— (80.0% flow / 96.27% step)
Real-API (UPS CIE)	48	—	—	— (80% clean acceptance)
Sampling variance (5x at T=0.7)	500	—	—	— (97.8% avg pass)

Changes from previous release (v3 → v5a)

Axis	v3	v5a	Delta
Held-out accuracy	98.5%	100%	+1.5pp
Sibling shape	89.2%	88.33%	−0.9pp
Multi-turn flow	66.7%	80.0%	+13.3pp
Multi-turn step	93.8%	96.27%	+2.5pp
Negative pass	53.75%	86.25%	+32.5pp

Training Data

The dataset is synthetic, generated from UPS MCP server tool schemas.

Split	Samples
Train	~1,100
Eval	~120
Test	122
Total	~1,342

The v5a training data includes:

Full replay of the original training set
20 targeted augmentation examples focused on multi-turn workflow steps and negative/refusal boundaries
Oversampling of initially weak tools (3x): rate_shipment, get_time_in_transit, upload_paperless_document, get_service_center_facilities, get_landed_cost_quote

No UPS API credentials or private customer shipment data are included.

Training Procedure

Setting	Value
Method	QLoRA SFT
Base checkpoint for training	`models/ups-tools-v3-merged` (v3 champion, merged)
LoRA rank	16
LoRA alpha	16
rsLoRA	Enabled
Target modules	`q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj`
Trainable parameters	~40M
Learning rate	5e-5
Epochs	1
Effective batch size	8
Max sequence length	6,144
Optimizer	AdamW 8-bit paged
Training precision	FP16
Chat template	ChatML
Final train loss	0.200

Hardware and Software

Training was run locally on an NVIDIA GeForce RTX 3090 with 24 GB VRAM.

Component	Version
Python	3.12.3
PyTorch	2.10.0+cu128
Unsloth	2026.2.1
PEFT	0.18.1
TRL	0.24.0

Artifacts

Artifact	Path
LoRA adapter	Repo root: `adapter_config.json`, `adapter_model.safetensors`, tokenizer files
GGUF Q4_K_M export	`ups-tools-v5a-Q4_K_M.gguf`
Ollama Modelfile	`Modelfile`

Usage

Transformers and PEFT

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_id = "Qwen/Qwen2.5-7B-Instruct"
adapter_id = "matt-hans93/ups-tools-qwen2.5-7b"

base = AutoModelForCausalLM.from_pretrained(base_id, device_map="auto")
model = PeftModel.from_pretrained(base, adapter_id)
tokenizer = AutoTokenizer.from_pretrained(adapter_id)

Ollama

Download ups-tools-v5a-Q4_K_M.gguf and Modelfile, then create a local Ollama model:

ollama create ups-tools-qwen25 -f Modelfile

The Modelfile uses a low temperature and enough output budget for nested JSON payloads:

FROM ./ups-tools-v5a-Q4_K_M.gguf

PARAMETER temperature 0.1
PARAMETER top_p 0.9
PARAMETER num_predict 2048
PARAMETER stop "<|im_end|>"

Limitations and Risks

Paraphrase fragility. Value-grounded accuracy drops to 68.6% on paraphrased prompts (terse SMS-style is worst at ~57%). Free-form text interfaces will see higher error rates.
Sibling-tool confusion. On adversarial prompts with similar tools, shape accuracy is 88.3%. The most confused pairs are get_political_divisions vs find_locations and rate_shipment vs create_shipment.
Incomplete prompts. The model fabricates missing fields instead of asking for clarification. Production orchestrators must validate arguments against the MCP schema before execution.
rate_shipment cross-field combinations. The model can pair UPS service codes with incompatible packaging types. Validate service↔packaging compatibility before submitting.
Multi-step workflow cascading. Step accuracy is 96.3% but full-workflow accuracy is 80% — early errors cascade. Validate each turn in multi-turn flows.
Specialization to 18 UPS MCP tools. Do not expect generalization to unrelated APIs without additional evaluation.

Out-of-Scope Use

Do not use this model as an autonomous authority for shipping purchases, billing decisions, customs declarations, legal compliance, or cancellation actions. It should generate candidate MCP tool calls only; production systems should validate, log, and gate execution.

License

Apache 2.0. The base Qwen2.5 model is also released under Apache 2.0.

Downloads last month: 19

GGUF

Model size

8B params

Architecture

qwen2

Hardware compatibility

4-bit

Model tree for matt-hans93/ups-tools-qwen2.5-7b

Base model

Qwen/Qwen2.5-7B

Finetuned

Qwen/Qwen2.5-7B-Instruct

Adapter

(2557)

this model

Evaluation results

Tool selection accuracy on 122 held-out test cases (synthetic, disjoint from training)
self-reported

1.000
Parameter completeness on 122 held-out test cases (synthetic, disjoint from training)
self-reported

1.000
JSON validity on 122 held-out test cases (synthetic, disjoint from training)
self-reported

1.000
Format compliance on 122 held-out test cases (synthetic, disjoint from training)
self-reported

1.000
Shape on 900-case templated stress (50/tool x 18 tools)
self-reported

0.952
Value-grounded on 900-case templated stress (50/tool x 18 tools)
self-reported

0.938
Structural on 900-case templated stress (50/tool x 18 tools)
self-reported

0.951
JSON validity on 900-case templated stress (50/tool x 18 tools)
self-reported

0.968