Model Card for zeroproof-ecommerce-1b

Frontier-approaching e-commerce intent detection in a 1B model. Given a live customer conversation, zeroproof-ecommerce-1b returns a single JSON object with the customer's payment intent and its structured details. It runs inline on live traffic, at a cost and latency where calling a frontier model is not an option.

On a hard, held-out benchmark it reaches 77.5 macro-averaged intent-type accuracy, 96% of GPT-5's, at $0.18 per 1M output tokens and about a second per response.

Model Details

Model Description

• Developed by: ZeroProof
• Model type: E-commerce payment-intent classifier; structured JSON output over seven intent types
• Language: English
• License: Gemma (inherited from the base model)
• Finetuned from: google/gemma-3-1b-it, via 4-bit QLoRA

Model Sources

• Repository: https://huggingface.co/zero-proof-ai/zeroproof-ecommerce-1b
• Training data: https://huggingface.co/datasets/zero-proof-ai/ecommerce-intent

Uses

Direct Use

Drop-in intent detection for e-commerce and payments support. Feed the conversation so far; the model returns one JSON object per message: whether an actionable payment intent is present, which of seven types it is, and the extracted details. Cheap and fast enough to call on every turn of every conversation.

Out-of-Scope Use

English-language e-commerce and payments only. It is a narrow intent classifier, not a general assistant, and should not be the sole authority for executing a payment without a downstream verification step.

Bias, Risks, and Limitations

Trained and evaluated on synthetic role-play conversations from a multi-agent marketplace simulation, labeled in a separate model pass under a locked labeling policy; labels are model-assigned and carry residual noise, and live-traffic distribution may differ. Accuracy is currently strongest on refunds, exchanges, and transfers; checkout and bill-pay are the focus of the current data round.

Recommendations

Pair the model with a verification layer before any payment executes, and re-benchmark on your own traffic before relying on the reported numbers in production.

How to Get Started with the Model

from peft import AutoPeftModelForCausalLM
from transformers import AutoTokenizer

model = AutoPeftModelForCausalLM.from_pretrained("zero-proof-ai/zeroproof-ecommerce-1b")
tok   = AutoTokenizer.from_pretrained("zero-proof-ai/zeroproof-ecommerce-1b")

Prompt it with the system prompt shipped in prompt_template.txt plus the conversation so far, and decode greedily (temperature=0).

Output format

The model returns one JSON object per message:

{"intent_detected": true, "core_type": "reverse",
 "details": {"action": "refund", "merchant": "Amazon", "amount": 81.40, "reason_code": "scam"},
 "confidence": 0.85, "reason": "Customer wants a refund on a suspicious charge.",
 "source_message_seqs": [2]}

core_type is one of spend, send, exchange, recur, bill, reverse, or none, and each type carries its own details fields.

Training Details

Training Data

The differentiator is the data. ZeroProof builds e-commerce intent training data as randomized conversational simulations: role-played customers with independently sampled personas, tones, financial situations, life stages, devices, and behaviors, including adversarial actors, simulated turn by turn between two independently drawn models. Generation is label-blind (the generating models never see the intent schema), labels are assigned in a separate pass, and every candidate passes a structural data gate: deduplicated by message signature, with zero train/eval leakage.

Training Procedure

Training Hyperparameters

• Method: 4-bit QLoRA (via unsloth)
• Epochs: 1
• Effective batch size: 16
• Learning rate: 2e-4
• Sequence length: 12288 (long conversations fit)

Evaluation

Testing Data, Factors & Metrics

Held-out eval of 1,977 conversations, zero train/eval leakage, macro-averaged (equal weight per intent), scored deterministically against gold: intent type by exact match, order details as the fraction of gold fields reproduced (partial credit per field). The frontier panel is scored on a 412-row balanced subset of the same eval.

Results

• Fine-tuning takes intent-type accuracy from 12 to 77.5 (macro), landing at 96% of GPT-5's 80.2.
• On structured detail extraction it exceeds the full frontier panel: 64 against Sonnet 5's 56, Opus 4.8's 53, and GPT-5's 41.
• On the no-action boundary it errs toward inaction: 9.2% of true no-action conversations are marked actionable, with remaining errors concentrated in the safe direction (missed actions, not invented ones).

Technical Specifications

Model Architecture and Objective

A LoRA adapter over gemma-3-1b-it, trained response-only to emit one structured intent object per turn.

Compute Infrastructure

Served as an OpenAI-compatible endpoint (base + adapter) under vLLM. Measured on a Modal L4 GPU: $0.18 per 1M output tokens at batched capacity and ~1s per response (p50 984 ms, p95 1,191 ms), 55 to 140x below frontier list prices.

Model Card Contact

ZeroProof, https://huggingface.co/zero-proof-ai