Instructions to use rikunarita/Qwen3.5-2B-Base-FP16-FP32Norm with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use rikunarita/Qwen3.5-2B-Base-FP16-FP32Norm with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="rikunarita/Qwen3.5-2B-Base-FP16-FP32Norm")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("rikunarita/Qwen3.5-2B-Base-FP16-FP32Norm")
model = AutoModelForMultimodalLM.from_pretrained("rikunarita/Qwen3.5-2B-Base-FP16-FP32Norm", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use rikunarita/Qwen3.5-2B-Base-FP16-FP32Norm with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "rikunarita/Qwen3.5-2B-Base-FP16-FP32Norm"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "rikunarita/Qwen3.5-2B-Base-FP16-FP32Norm",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/rikunarita/Qwen3.5-2B-Base-FP16-FP32Norm

SGLang

How to use rikunarita/Qwen3.5-2B-Base-FP16-FP32Norm with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "rikunarita/Qwen3.5-2B-Base-FP16-FP32Norm" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "rikunarita/Qwen3.5-2B-Base-FP16-FP32Norm",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "rikunarita/Qwen3.5-2B-Base-FP16-FP32Norm" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "rikunarita/Qwen3.5-2B-Base-FP16-FP32Norm",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use rikunarita/Qwen3.5-2B-Base-FP16-FP32Norm with Docker Model Runner:
```
docker model run hf.co/rikunarita/Qwen3.5-2B-Base-FP16-FP32Norm
```

🇬🇧 English

🚀 Qwen3.5-2B-Base FP16 (Colab T4 Ready, Norm FP32 Stable)

A lightweight FP16-converted version of Qwen3.5-2B-Base, specifically optimized for Google Colab T4 GPUs, with critical normalization layers kept in FP32 for numerical stability.

✨ Why this model?

The original model uses BF16, which is not fully supported on NVIDIA T4 GPUs.

This repository provides a carefully engineered FP16 conversion, while preserving Norm layers in FP32, resulting in:

✅ Stable training on Colab (T4)
✅ Improved numerical stability vs pure FP16
✅ Reduced NaNs / training divergence
✅ Drop-in compatibility with standard pipelines

⚡ Key Features

🔄 BF16 → FP16 conversion
🧠 Norm layers kept in FP32 (critical for stability)
⚙️ Fully compatible with Hugging Face ecosystem
🚀 Optimized for T4 (16GB VRAM)
🔧 Ready for LoRA / QLoRA fine-tuning

🧠 Why FP32 Norm Matters

Pure FP16 conversion can introduce:

Gradient instability
Loss spikes
Training divergence

By keeping normalization layers in FP32:

✔ More stable forward/backward passes
✔ Better convergence during fine-tuning
✔ Reduced precision-related artifacts

This mirrors best practices used in modern LLM training pipelines.

🧪 Verified Environment

GPU: NVIDIA T4
Platform: Google Colab
Precision: FP16 (Norm: FP32)
Framework: PyTorch + Transformers

⚠️ Notes

This is a precision-converted model, not retrained
Norm layers are intentionally kept in FP32
Minor differences from BF16 may exist
All original weights and architecture are preserved

💡 Recommended Use Cases

Colab-based prototyping
Fine-tuning pipelines (LoRA / QLoRA)
Educational experiments
Low-resource environments

🤝 Acknowledgements

Original model: Qwen/Qwen3.5-2B-Base
Conversion & stabilization: This repository

⭐ Support

If this model helps your workflow, consider liking ❤️ the model or following me!

🇯🇵 日本語

🚀 Qwen3.5-2B-Base FP16 (Colab T4 対応、重要な正規化レイヤーは数値的安定性確保のため FP32 維持)

Qwen3.5-2B-Base を FP16 に変換した軽量版であり、Google Colab T4 GPU向けに特別に最適化されています。また、数値的安定性を確保するため、重要な正規化レイヤーは FP32 のまま維持されています。

✨ このモデルを選ぶ理由

元のモデルはBF16を使用していますが、これはNVIDIA T4 GPUでは完全にはサポートされていません。

このリポジトリでは、Norm層をFP32のまま維持しつつ、入念に設計されたFP16変換を提供しており、その結果として以下の利点があります：

✅ Colab (T4) での安定したトレーニング
✅ 純粋なFP16と比較して数値的安定性が向上
✅ NaNの発生やトレーニングの収束不良が減少
✅ 標準的なパイプラインとのドロップイン互換性

⚡ 主な機能

🔄 BF16 → FP16変換
🧠 Norm層をFP32で保持（安定性にとって重要）
⚙️ Hugging Faceエコシステムと完全互換
🚀 T4（16GB VRAM）向けに最適化
🔧 LoRA / QLoRAによる微調整に対応

🧠 なぜFP32でNorm層のみ保存することが重要なのか

純粋なFP16変換では、以下の問題が発生する可能性があります：

勾配の不安定化
損失の急上昇
トレーニングの収束不良

Norm層をFP32で保持することで：

✔ フォワード/バックワードパスの安定性向上
✔ 微調整時の収束性の向上
✔ 精度に関連するアーティファクトの低減

これは、最新のLLMトレーニングパイプラインで採用されているベストプラクティスを反映しています。

🧪 検証環境

GPU: NVIDIA T4
プラットフォーム: Google Colab
精度: FP16 (Norm層正規化: FP32)
フレームワーク: PyTorch + Transformers

⚠️ 注意事項

これは再学習されたものではなく、精度変換されたモデルです
Norm層は意図的に FP32 のままにしています
BF16 との間に些細な違いが存在する可能性があります
オリジナルの重みとアーキテクチャはすべて保持されています

💡 推奨ユースケース

Colab ベースのプロトタイピング
微調整パイプライン (LoRA / QLoRA)
教育目的の実験
リソースが限られた環境

🤝 謝辞

オリジナルモデル：Qwen/Qwen3.5-2B-Base
変換および安定化：本リポジトリ

⭐ サポート

このモデルがあなたのお役に立った場合は、モデルに❤️、または私をフォローしていただけると嬉しいです！

Citation(引用)

@misc{qwen3.5,
    title  = {{Qwen3.5}: Towards Native Multimodal Agents},
    author = {{Qwen Team}},
    month  = {February},
    year   = {2026},
    url    = {https://qwen.ai/blog?id=qwen3.5}
}