🧠 Model Card: pranjalsingh/alpaca-Llama-3.1-70B-Instruct-chat

A LoRA fine-tuned version of the meta-llama/Llama-3.1-70B-Instruct model on the Alpaca dataset, optimized using PEFT and accelerated on Intel Gaudi3 HPU hardware.

---

📝 Model Summary

This model is a fine-tuned variant of LLaMA 3.1 70B Instruct, trained on the Alpaca dataset using Parameter-Efficient Fine-Tuning (PEFT) via LoRA. The goal of this fine-tuning was to improve instruction-following performance on lightweight resources, leveraging Intel’s Gaudi3 HPU for efficient training.

---

📄 Model Details

• Base Model: meta-llama/Llama-3.1-70B-Instruct
• Fine-tuned Model: pranjalsingh/alpaca-Llama-3.1-70B-Instruct-chat
• Fine-tuned By: Pranjal Singh Thakur
• Dataset: Stanford Alpaca dataset
• PEFT Library: PEFT v0.12.0
• Fine-tuning Technique: LoRA
• Epochs: 2
• Training Hardware: 1 Node with 8× Intel Gaudi3 HPUs
• Language(s): English
• License: Same as base model (LLaMA 3)
• Credit: Intel for providing Gaudi3 HPU infrastructure

---

🚀 Usage

Direct Use

Use the model as an instruction-following chatbot or in downstream applications requiring LLM completion with lightweight deployment using LoRA adapters.

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-3.1-70B-Instruct")
tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-3.1-70B-Instruct")

model = PeftModel.from_pretrained(base_model, "pranjalsingh/alpaca-Llama-3.1-70B-Instruct-chat")

inputs = tokenizer("### Instruction: Explain quantum computing in simple terms.", return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

---

📊 Evaluation Results

Metric	Value
Eval Accuracy	73.27%
Eval Loss	1.02
Perplexity	2.79
Evaluation Runtime	20.97s
Samples Evaluated	101
Samples/Sec	4.82
Max Memory Used (GB)	126.2
Total Available Memory	126.54 GB
Memory Allocated (GB)	41.06

---

🛠 Training Configuration

• Epochs: 2
• Precision: Likely mixed precision (bf16/fp16 on Gaudi3)
• Hardware: Intel Gaudi3 HPU (8 cards, 1 node)
• Frameworks: PEFT, Hugging Face Transformers
• Batching & Tokenization: Not explicitly provided

---

📦 Model Sources

• Repository: Hugging Face Model Card
• Dataset: Stanford Alpaca
• Base Model: meta-llama/Llama-3.1-70B-Instruct

---

⚠️ Limitations & Risks

• Not suitable for multilingual tasks (trained only on English data).
• May reflect biases present in the Alpaca dataset.
• Not recommended for sensitive or safety-critical applications.
• Fine-tuning was conducted for instruction tasks — may not generalize to other domains.

---

♻️ Environmental Impact

Parameter	Value
Compute Platform	Intel Gaudi3
Cards Used	8× HPU
Training Duration	~2 Epochs
Region	[More info needed]
Emission Estimate	[Use MLCO2 to calculate]

---

👨‍💻 Author & Acknowledgment

• Author: Pranjal Singh Thakur
• Credit: Intel (for compute resources using Gaudi3 HPU)

---

🔖 Citation

Coming soon.