Qwen3-0.6B: India Geopolitics & Policy Fine-Tune

This model is a LoRA fine-tune of Qwen/Qwen3-0.6B, specialized in Indian geopolitics, domestic policies, economy, and strategic affairs.

It was trained locally on an NVIDIA GTX 1650 (4GB VRAM) across two distinct continuous learning phases.

📚 Training Data

The model was fine-tuned on a custom curated dataset covering approximately 12,000 words (equivalent to 25+ pages) of highly focused strategic and policy content.

Phase 1 (Geopolitics):

Non-Alignment & Strategic Autonomy
Line of Actual Control (LAC) & Galwan Valley clash
Quad, Indo-Pacific strategy, and maritime security
India's Nuclear Doctrine (No First Use)

Phase 2 (Extended Policy & Strategy Corpus):

Governance & Law: India's Constitution, Judicial System, Internal Security (Northeast)
Economy: 1991 Economic Reforms, Trade Policy, Urban Development, Agricultural Policy
Strategic Sectors: Space Program (ISRO, Chandrayaan), Energy Policy, AI & Technology Policy, Cybersecurity
Foreign Policy: India-Africa Relations, India-Gulf Relations (CEPA, I2U2), ASEAN Relations
Social Development: National Education Policy (NEP) 2020, Ayushman Bharat Healthcare, Demographic Dividend (Viksit Bharat 2047)
Soft Power: Cultural Diplomacy (Yoga, Diaspora, Cinema)

🛠️ Training Configuration

Trained on Windows 11 using a custom PyTorch/GradientScaler loop to bypass Windows App Control restrictions.

Base Model: Qwen/Qwen3-0.6B
Method: PEFT / LoRA (r=16, alpha=32)
Precision: float16 with Autocast
Learning Rate: 3e-5 (Cosine Annealing)
Batch Size: 1 (with Gradient Accumulation steps = 8)
Max Length: 256 tokens
Epochs: 5 per phase

🚀 Usage

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
import torch

base_model = "Qwen/Qwen3-0.6B"
adapter_model = "ProfRutPatel/qwen3-india-geopolitics"

tokenizer = AutoTokenizer.from_pretrained(base_model, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    base_model, 
    dtype=torch.float16, 
    trust_remote_code=True
).cuda()

# Load the fine-tuned LoRA weights
model = PeftModel.from_pretrained(model, adapter_model)

prompt = "### Instruction:\nWhat is the strategic significance of India's demographic dividend for its 'Viksit Bharat' 2047 vision?\n\n### Response:\n"
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")

with torch.no_grad():
    outputs = model.generate(**inputs, max_new_tokens=150)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

⚠️ Limitations

As this is a 0.6B parameter model, it is highly efficient and runs on almost any hardware, but it is prone to hallucinations regarding specific names or dates. It successfully captures the structural tone, core concepts, and geopolitical themes present in the training data.

Downloads last month: -; Downloads are not tracked for this model. How to track