NhZtw/Qwen3.5-2B-UCO-6to4-TruthFix-GGUF

Qwen3.5-2B-UCO-6to4-TruthFix

A Two-Stage Instruction-Tuned and Truthfulness-Aligned Language Model
Based on Qwen/Qwen3.5-2B · Published: 2026-04-11

---

Abstract

This model card documents Qwen3.5-2B-UCO-6to4-TruthFix, a fine-tuned derivative of the Qwen/Qwen3.5-2B base model produced through a deliberate two-stage supervised fine-tuning (SFT) pipeline.

In Stage 1, the model was aligned for general instruction-following and chain-of-thought reasoning using a weighted blend of the UltraChat 200k and Opus-4.6 Reasoning datasets (60/40 split). In Stage 2 (designated TruthFix), the Stage-1 checkpoint was further fine-tuned on the TruthfulQA dataset to suppress common factual hallucinations and improve model calibration on questions where large language models are known to produce misleading but confidently stated answers.

Evaluation on the standard Open LLM Leaderboard benchmark suite (ARC-Challenge, ARC-Easy, HellaSwag, TruthfulQA MC1/MC2, WinoGrande) demonstrates consistent improvements over the untuned base model across 8 of 9 metrics. The most notable gains are observed on HellaSwag acc_norm (+6.66%) and TruthfulQA MC1 (+5.43%), confirming that TruthFix successfully improves factual truthfulness without causing catastrophic forgetting on general language understanding tasks.

---

Table of Contents

1. Model Overview
2. Training Pipeline
3. Evaluation
4. Usage
5. Limitations, Risks, and Biases
6. Environmental Impact
7. Citation
8. References

---

1. Model Overview

1.1 Identification

Field	Value
Model ID	NhZtw/Qwen3.5-2B-UCO-6to4-TruthFix
Base Model	Qwen/Qwen3.5-2B
Model Family	Qwen3.5
Parameter Count	~2 Billion
Architecture	Transformer decoder (causal LM)
Training Type	Supervised Fine-Tuning (SFT), two stages
Language(s)	English (primary), Chinese (inherited from base)
License	Apache 2.0 (inherited from base model)
Release Date	2026-04-11

1.2 Model Lineage

Qwen/Qwen3.5-2B  (base, ~2B params)
        │
        ▼
┌───────────────────────────────────────────┐
│             Stage 1 Fine-Tune             │
│  60% HuggingFaceH4/ultrachat_200k         │
│  40% Crownelius/Opus-4.6-Reasoning-3300x  │
│  Objective: instruction-following +        │
│             chain-of-thought reasoning    │
└───────────────────────────────────────────┘
        │
        ▼  (Stage-1 checkpoint)
┌───────────────────────────────────────────┐
│       Stage 2 Fine-Tune  (TruthFix)       │
│  100% truthfulqa/truthful_qa              │
│  Objective: factual calibration,          │
│             hallucination suppression     │
└───────────────────────────────────────────┘
        │
        ▼
Qwen3.5-2B-UCO-6to4-TruthFix  ← this model

1.3 Intended Use

• General-purpose instruction-following in English and Chinese
• Chain-of-thought and multi-step reasoning tasks
• Applications requiring reduced hallucination and improved factual calibration
• Research on small-scale SFT pipelines and truthfulness alignment techniques
• Educational and experimental deployment on resource-constrained hardware

1.4 Out-of-Scope Use

• Safety-critical applications without additional RLHF or constitutional alignment
• Legal, medical, or financial advice in production settings
• Tasks requiring up-to-date factual knowledge beyond the training data cutoff
• High-volume production inference without profiling on target hardware

---

2. Training Pipeline

2.1 Design Philosophy

The model was produced via a sequential two-stage SFT pipeline applied to the weights of Qwen/Qwen3.5-2B. Each stage uses a distinct dataset composition targeting a specific behavioural objective.

The two-stage design is intentional. Stage 1 establishes broad instruction-following competence across diverse domains, while Stage 2 applies a targeted corrective signal to address the well-documented tendency of base language models to produce plausible but factually incorrect answers — sometimes called the sycophancy / hallucination problem. Applying Stage 2 after Stage 1 has converged ensures that the TruthFix signal does not disrupt the model's foundational instruction-following capability, mitigating the risk of catastrophic forgetting.

2.2 Dataset Composition

Stage	Dataset	Mix Ratio	Primary Objective
Stage 1	HuggingFaceH4/ultrachat_200k	60%	General instruction-following, multi-turn dialogue
Stage 1	Crownelius/Opus-4.6-Reasoning-3300x	40%	Chain-of-thought reasoning and logical inference
Stage 2	truthfulqa/truthful_qa	100%	Factual calibration, hallucination suppression

2.2.1 Stage 1 — Instruction & Reasoning Alignment

Dataset A: HuggingFaceH4/ultrachat_200k (60% weight)

UltraChat 200k is a large-scale, synthetically generated multi-turn dialogue corpus curated by HuggingFace. It covers a broad range of topics and is specifically structured to elicit instruction-following behavior. By assigning it the majority weight (60%), Stage 1 ensures the model develops:

• Robust alignment with human instructions across diverse task types
• A conversational register appropriate for assistant-style deployment
• Capability for summarisation, question-answering, creative writing, and structured output generation

Dataset B: Crownelius/Opus-4.6-Reasoning-3300x (40% weight)

This reasoning-focused dataset provides structured examples of multi-step problem decomposition and chain-of-thought (CoT) traces. Assigning it a 40% weight introduces a strong reasoning prior before the truthfulness correction stage, preventing Stage 2 from inadvertently suppressing long-form structured thinking while correcting factual errors.

2.2.2 Stage 2 — TruthFix (Truthfulness Correction)

Dataset: truthfulqa/truthful_qa (100% of Stage 2)

TruthfulQA (Lin et al., 2022) is a benchmark dataset of 817 questions specifically designed to probe false beliefs and misconceptions that humans commonly hold. The questions span 38 categories including health, law, finance, conspiracy theories, and everyday misconceptions.

By fine-tuning the Stage-1 checkpoint on TruthfulQA, Stage 2 exposes the model to the exact failure modes it is designed to mitigate — training the model to decline confidently asserting falsehoods and to appropriately qualify uncertain claims. This stage is designated TruthFix in the model identifier.

A critical design consideration is that Stage 2 is applied after Stage 1 has already converged. This ordering ensures that the model's general instruction-following capability is not disrupted by the relatively small and domain-specific TruthfulQA signal (817 questions vs. the much larger Stage 1 corpus). The risk of catastrophic forgetting is mitigated by the stable foundation established in Stage 1.

2.3 Training Configuration

Parameter	Value
Training Framework	Hugging Face Transformers
Hardware	2× GPU (Kaggle infrastructure)
Evaluation Framework	EleutherAI lm-evaluation-harness
Evaluation Timestamp	2026-04-11T07:34:33
Batch Size (eval)	auto (28)
Few-Shot Setting	0-shot (all benchmarks)
Precision	bfloat16 / auto

---

3. Evaluation

3.1 Benchmark Descriptions

ARC (AI2 Reasoning Challenge) — Clark et al., 2018.
A dataset of 7,787 genuine US grade-school science questions divided into an Easy and a Challenge partition. The Challenge set contains questions that require more complex reasoning not solvable by simple retrieval. Both raw accuracy (acc) and length-normalised accuracy (acc_norm) are reported. acc_norm is generally considered the more robust metric as it corrects for answer-length bias.

HellaSwag — Zellers et al., 2019.
A commonsense NLI benchmark of 70k+ sentence-completion questions testing physical situational reasoning. Adversarially filtered to be difficult for models while remaining easy for humans. acc_norm is the primary metric and the more meaningful signal for this benchmark.

TruthfulQA MC1 / MC2 — Lin et al., 2022.
A benchmark of 817 questions probing 38 categories of common human misconceptions. MC1 tests single best-answer selection (harder); MC2 tests multi-true/false scoring (softer). Both metrics reward the model for correctly identifying false but plausible claims — the exact behaviour targeted by Stage 2.

WinoGrande — Sakaguchi et al., 2021.
A large-scale Winograd-schema commonsense reasoning benchmark of 44k problems testing pronoun resolution grounded in everyday world knowledge. Adversarially filtered from the original Winograd Schema dataset to minimise solvability via spurious lexical cues.

3.2 Results: Fine-Tuned Model

The following table presents the raw benchmark results for this model (0-shot):

Benchmark	Metric	Value	Stderr
ARC-Challenge	acc ↑	0.3933	±0.0143
ARC-Challenge	acc_norm ↑	0.4224	±0.0144
ARC-Easy	acc ↑	0.6511	±0.0098
ARC-Easy	acc_norm ↑	0.5598	±0.0102
HellaSwag	acc ↑	0.4635	±0.0050
HellaSwag	acc_norm ↑	0.6004	±0.0049
TruthfulQA MC1	acc ↑	0.3317	±0.0165
TruthfulQA MC2	acc ↑	0.5099	±0.0149
WinoGrande	acc ↑	0.5975	±0.0138

3.3 Comparison with Base Model

All results use 0-shot evaluation. Percentage change is computed as (fine-tuned − base) / base × 100.

Benchmark	Metric	Base (Qwen3.5-2B)	This Model	Δ Change	Stderr (FT)
ARC-Challenge	acc	0.3857	0.3933	+1.97% ↑	±0.0143
ARC-Challenge	acc_norm	0.4027	0.4224	+4.89% ↑	±0.0144
ARC-Easy	acc	0.6578	0.6511	−1.02% ↓ ¹	±0.0098
ARC-Easy	acc_norm	0.5425	0.5598	+3.19% ↑	±0.0102
HellaSwag	acc	0.4471	0.4635	+3.67% ↑	±0.0050
HellaSwag	acc_norm	0.5629	0.6004	+6.66% ↑ ★	±0.0049
TruthfulQA MC1	acc	0.3146	0.3317	+5.43% ↑ ★	±0.0165
TruthfulQA MC2	acc	0.5043	0.5099	+1.11% ↑	±0.0149
WinoGrande	acc	0.5722	0.5975	+4.42% ↑	±0.0138

¹ The ARC-Easy acc decline of −1.02% falls within the reported standard error (±0.0098) and is not considered statistically significant.
★ Largest absolute improvements — see Key Findings below.

3.4 Key Findings

1. HellaSwag acc_norm registers the largest relative gain (+6.66%).
This improvement is attributed primarily to the Stage 1 Opus-4.6-Reasoning dataset, which introduces structured chain-of-thought traces that improve the model's ability to anticipate physically plausible event continuations — the core competency tested by HellaSwag.

2. TruthfulQA MC1 improves by +5.43%, directly validating the TruthFix objective.
MC1's single-best-answer format is the harder and more discriminative of the two TruthfulQA metrics. A +5.43% improvement on this metric provides direct evidence that Stage 2 fine-tuning successfully improves the model's ability to identify and reject false but plausible claims.

3. ARC-Challenge acc_norm improves by +4.89%.
The challenge partition of ARC requires more complex scientific reasoning than simple retrieval. The improvement here suggests that the reasoning-focused Stage 1 training generalises positively to science-domain multi-hop inference.

4. WinoGrande improves by +4.42%.
WinoGrande tests commonsense pronoun resolution under adversarial filtering. The improvement is consistent with the hypothesis that instruction-following fine-tuning strengthens the model's pragmatic language understanding.

5. No catastrophic forgetting is observed.
All general-reasoning benchmarks (ARC, HellaSwag, WinoGrande) maintained or improved performance, confirming that the Stage 2 TruthFix signal did not negatively impact the model's foundational language understanding established in Stage 1. This validates the sequential two-stage design choice.

6. ARC-Easy acc marginally declines (−1.02%), within noise.
This is the only metric to show a decline. Given that the delta falls within the standard error of ±0.0098, this result is not statistically significant and does not represent a genuine degradation in model capability.

3.5 Evaluation Reproducibility

Evaluation was conducted using EleutherAI/lm-evaluation-harness on Kaggle infrastructure with 2 GPUs. Full results JSON is preserved in the model repository.

lm_eval \
  --model hf \
  --model_args pretrained=NhZtw/Qwen3.5-2B-UCO-6to4-TruthFix,trust_remote_code=True \
  --tasks arc_challenge,arc_easy,hellaswag,truthfulqa_mc1,truthfulqa_mc2,winogrande \
  --device cuda \
  --batch_size auto:28 \
  --num_fewshot 0 \
  --output_path ./results

---

4. Usage

4.1 Loading the Model

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "NhZtw/Qwen3.5-2B-UCO-6to4-TruthFix"

tokenizer = AutoTokenizer.from_pretrained(
    model_id,
    trust_remote_code=True
)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype="auto",
    device_map="auto",
    trust_remote_code=True
)

4.2 Inference Example

messages = [
    {"role": "user", "content": "What is the boiling point of water on Mount Everest?"}
]

text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)
inputs = tokenizer([text], return_tensors="pt").to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=512,
    temperature=0.7,
    do_sample=True,
    repetition_penalty=1.1
)

response = tokenizer.decode(
    outputs[0][inputs.input_ids.shape[-1]:],
    skip_special_tokens=True
)
print(response)

4.3 Hardware Requirements

Precision	Min VRAM	Recommended Setup
float32	~8 GB	16 GB+ for comfortable inference
bfloat16 / float16	~4 GB	8 GB (e.g. RTX 3060 12GB)
int8 (bitsandbytes)	~2.5 GB	4–6 GB
int4 (GPTQ / AWQ)	~1.5 GB	3–4 GB — runs on consumer GPU

4.4 Quantisation Example (int4 via bitsandbytes)

from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
import torch

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_compute_dtype=torch.bfloat16,
    bnb_4bit_use_double_quant=True,
    bnb_4bit_quant_type="nf4"
)

model = AutoModelForCausalLM.from_pretrained(
    "NhZtw/Qwen3.5-2B-UCO-6to4-TruthFix",
    quantization_config=bnb_config,
    device_map="auto",
    trust_remote_code=True
)

---

5. Limitations, Risks, and Biases

5.1 Model-Level Limitations

• Scale. At ~2 billion parameters, the model has substantially lower capacity than frontier models (70B+ class). Complex multi-step reasoning, rare-domain knowledge, and long-context coherence will be weaker.
• Knowledge cutoff. World knowledge is bounded by the training data of the base Qwen3.5-2B model and the fine-tuning datasets. The model will not have reliable information about events after its knowledge cutoff.
• Residual hallucination. TruthFix reduces, but does not eliminate, hallucination. The model may still produce plausible-sounding but factually incorrect statements, particularly on obscure or specialised topics.
• Calibration. The model does not have explicit uncertainty quantification. Expressed confidence in outputs may not correlate reliably with factual accuracy.

5.2 Dataset-Level Biases

• UltraChat 200k is synthetically generated, which may introduce systematic biases from the generator model's priors, including overconfidence and Western-centric perspectives.
• TruthfulQA targets English-language misconceptions primarily from Western cultural contexts. Truthfulness improvements may not generalise equally to other languages or cultural knowledge systems.
• Opus-4.6 Reasoning reflects the reasoning style of a specific model family; this may induce stylistic reasoning biases not present in the base model.

5.3 Deployment Risks

• This model should not be used as the sole decision-maker in safety-critical, legal, medical, or financial applications.
• Without additional safety alignment (e.g. RLHF, constitutional AI), the model may produce harmful, biased, or offensive outputs under adversarial prompting conditions.
• Users deploying this model in production should conduct domain-specific safety evaluations appropriate to their use case prior to deployment.

---

6. Environmental Impact

Fine-tuning was conducted on 2 GPUs via Kaggle's cloud infrastructure. Exact compute hours and carbon emissions were not instrumented during training. As a reference point, fine-tuning 2B-parameter models typically requires on the order of a few GPU-hours for convergence on datasets of this scale — substantially lower than pre-training from scratch. Carbon-equivalent estimates can be computed via the ML CO₂ Impact calculator (Lacoste et al., 2019).

---

7. Citation

If you use this model in published research, please cite the model and the relevant datasets:

@misc{qwen3_5_2b_truthfix_2026,
  title        = {Qwen3.5-2B-UCO-6to4-TruthFix},
  author       = {NhZtw},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/NhZtw/Qwen3.5-2B-UCO-6to4-TruthFix}},
  note         = {Two-stage SFT fine-tune of Qwen3.5-2B with TruthfulQA alignment}
}

@misc{qwen3_5_2b_base,
  title        = {Qwen3.5-2B},
  author       = {{Qwen Team, Alibaba Cloud}},
  howpublished = {\url{https://huggingface.co/Qwen/Qwen3.5-2B}}
}

@inproceedings{lin2022truthfulqa,
  title     = {TruthfulQA: Measuring How Models Mimic Human Falsehoods},
  author    = {Stephanie Lin and Jacob Hilton and Owain Evans},
  booktitle = {Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (ACL)},
  year      = {2022}
}

@article{zellers2019hellaswag,
  title   = {HellaSwag: Can a Machine Really Finish Your Sentence?},
  author  = {Rowan Zellers and Ari Holtzman and Yonatan Bisk and Ali Farhadi and Yejin Choi},
  journal = {arXiv:1905.07830},
  year    = {2019}
}

@article{clark2018arc,
  title   = {Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge},
  author  = {Peter Clark and Isaac Cowhey and Oren Etzioni and Tushar Khot and Ashish Sabharwal and Carissa Schoenick and Oyvind Tafjord},
  journal = {arXiv:1803.05457},
  year    = {2018}
}

@inproceedings{sakaguchi2021winogrande,
  title     = {WinoGrande: An Adversarial Winograd Schema Challenge at Scale},
  author    = {Keisuke Sakaguchi and Ronan Le Bras and Chandra Bhagavatula and Yejin Choi},
  booktitle = {AAAI 2021},
  year      = {2021}
}

---

8. References

• Clark et al. (2018). Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge. arXiv:1803.05457.
• Zellers et al. (2019). HellaSwag: Can a Machine Really Finish Your Sentence? ACL 2019.
• Lin, Hilton & Evans (2022). TruthfulQA: Measuring How Models Mimic Human Falsehoods. ACL 2022.
• Sakaguchi et al. (2021). WinoGrande: An Adversarial Winograd Schema Challenge at Scale. AAAI 2021.
• Lacoste et al. (2019). Quantifying the Carbon Emissions of Machine Learning. NeurIPS 2019 Workshop on Climate Change AI.
• Gao et al. (2021). A Framework for Few-Shot Language Model Evaluation (lm-evaluation-harness). GitHub.