Aaqisher667/qwen05b-cyber-finetuned

Qwen2.5-0.5B Cybersecurity Fine-Tuned

A fine-tuned version of Qwen/Qwen2.5-0.5B-Instruct trained on cybersecurity instruction-response data using LoRA (Low-Rank Adaptation).

Model Details

Property	Value
Base model	Qwen/Qwen2.5-0.5B-Instruct
Model type	Causal Language Model
Parameters	0.5B
Fine-tuning method	LoRA (PEFT)
Training framework	Unsloth + TRL
Language	English
License	Apache-2.0
Domain	Cybersecurity

Training Details

Setting	Value
Dataset	AlicanKiraz0/Cybersecurity-Dataset-v1
Training examples	~2,500
Epochs	3
LoRA rank (r)	16
LoRA alpha	16
Learning rate	2e-4
Batch size	2
Gradient accumulation	4
Max sequence length	512
Quantization	4-bit (NF4)
Hardware	Tesla T4 GPU

How to Load

With Unsloth (recommended):

from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name   = "Aaqisher667/qwen05b-cyber-finetuned",
    max_seq_length = 512,
    dtype          = None,
    load_in_4bit   = True,
)

FastLanguageModel.for_inference(model)

Example usage:

messages = [
    {"role": "system", "content": "You are a cybersecurity expert assistant."},
    {"role": "user",   "content": "What is a SQL injection attack?"},
]

chat = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)

inputs = tokenizer(chat, return_tensors="pt").to(model.device)

import torch
with torch.no_grad():
    out = model.generate(
        **inputs,
        max_new_tokens = 256,
        temperature    = 0.7,
        do_sample      = True,
    )

response = tokenizer.decode(
    out[0][inputs["input_ids"].shape[1]:],
    skip_special_tokens=True
)
print(response)

Intended Use

This model is intended for:

• Cybersecurity question answering
• Security concept explanation
• Vulnerability analysis assistance
• On-device / edge deployment scenarios requiring a lightweight cybersecurity AI assistant

Limitations

• Small model size (0.5B) — responses may lack depth compared to larger models
• Fine-tuned on a limited dataset (~2,500 examples)
• Not suitable for generating exploit code or offensive security tooling
• Always verify cybersecurity advice from authoritative sources

Hardware Requirements

Setting	Minimum
VRAM (4-bit)	~2 GB
VRAM (full)	~4 GB
RAM	8 GB

Runs comfortably on a free Google Colab T4 GPU.

Training Data

This model was fine-tuned on AlicanKiraz0/Cybersecurity-Dataset-v1.

Property	Detail
Dataset	AlicanKiraz0/Cybersecurity-Dataset-v1
Size	~2,500 instruction-response pairs
Domain	Cybersecurity — defensive concepts, vulnerability analysis, security best practices
Format	System / User / Assistant instruction tuning format
Source	Publicly available on HuggingFace