advaitshewale/onyx-legal-70b-adapter

⚖️ Onyx Legal AI — 70B Legal Document Interpreter

Onyx is a fine-tuned LoRA adapter for Llama 3.3 70B Instruct, trained to analyze legal documents, court records, medical bills, insurance documents, and contracts with precision and clarity.

Built by Advait Shewale @ ExcavatedStudios.

---

🧠 What Onyx Does

Paste in any legal document and Onyx will:

• Identify document type, key clauses, and obligations
• Flag risks, ambiguities, and unfavorable terms
• Cite relevant statutes (USC, CFR, ECHR, ICD-10, CPT)
• Recommend actionable next steps
• Summarize complex legal language in plain English

Supported Document Types

Category	Examples
🏛️ Court Records	SCOTUS opinions, federal court decisions, case holdings
📄 Contracts	Employment, NDA, license, merger, supply, indemnification agreements
🏥 Medical Bills	CPT/ICD-10 coded bills, EOBs, dispute letters
🛡️ Insurance	Policy analysis, denial letters, coverage disputes, appeals
🌍 Human Rights	ECtHR cases, ECHR Article analysis
🇪🇺 EU Regulations	EurLex directives, regulatory compliance
🏛️ Legislation	US Congressional bills, federal regulations
🔒 Privacy / ToS	GDPR/CCPA compliance, terms of service red flags

---

🏗️ Model Details

Property	Value
Base Model	meta-llama/Llama-3.3-70B-Instruct
Adapter Type	LoRA (PEFT)
LoRA Rank	r=64, alpha=128
Target Modules	q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Quantization (Training)	4-bit NF4 (bitsandbytes)
Training Hardware	NVIDIA GH200 96GB HBM3e
Max Sequence Length	4,096 tokens
Eval Loss	0.7236
Accuracy	83.2%
Training Epochs	2 (Phase 2)
Training Steps	260
Optimizer	paged_adamw_8bit

---

📚 Training Data

Onyx was trained in two phases:

Phase 1 — Synthetic Legal Data

• Custom synthetic dataset of legal Q&A scenarios
• Covers: contract disputes, medical billing errors, insurance denials, court procedure, regulatory compliance
• Realistic CPT/ICD-10 medical codes, USC/CFR statute references

Phase 2 — Real World Legal Documents (4,600 examples)

Source	Type	Count
LexGLUE SCOTUS	U.S. Supreme Court opinions	1,800
LexGLUE LEDGAR	Real SEC-filed contract clauses	1,500
LexGLUE ECtHR	European Court of Human Rights cases	500
LexGLUE EurLex	EU regulatory documents	300
InsuranceQA	Insurance questions & answers	300
CMS Medical Billing	Real CPT/ICD-10 dispute scenarios	200

---

🚀 How to Use

Requirements

pip install transformers peft bitsandbytes accelerate torch

Load & Run Inference

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel

BASE_MODEL   = "meta-llama/Llama-3.3-70B-Instruct"
ADAPTER_PATH = "advaitshewale/onyx-legal-70b-adapter"

SYSTEM = (
    "You are Onyx, an expert AI legal analyst. You analyze legal documents, "
    "court records, medical bills, insurance documents, and contracts with "
    "precision and clarity. Identify key clauses, obligations, risks, "
    "relevant statutes, and provide actionable insights. If you do not recognize"
    "something in the analyzed text, simply reply I do not know. Do not make things up"
)

# Load tokenizer
tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL)
tokenizer.pad_token = tokenizer.eos_token

# Load base model in 4-bit
bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
    bnb_4bit_use_double_quant=True,
)
model = AutoModelForCausalLM.from_pretrained(
    BASE_MODEL,
    quantization_config=bnb_config,
    device_map="auto",
    torch_dtype=torch.bfloat16,
)

# Load Onyx adapter
model = PeftModel.from_pretrained(model, ADAPTER_PATH)
model.eval()

# Run inference
messages = [
    {"role": "system", "content": SYSTEM},
    {"role": "user",   "content": "Analyze this contract clause:\n\nThe party hereby waives all rights to appeal under Section 14(b), provided that the indemnification clause in Exhibit A remains in full force."}
]

text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)

with torch.no_grad():
    output = model.generate(
        **inputs,
        max_new_tokens=1024,
        temperature=0.2,
        top_p=0.9,
        do_sample=True,
        repetition_penalty=1.1,
    )

response = tokenizer.decode(output[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
print(response)

Minimum Hardware Requirements

Setup	VRAM	Notes
4-bit NF4 (recommended)	40GB+	A100 40GB, 2x A40, GH200
4-bit NF4 + CPU offload	24GB+ GPU	Slower, uses system RAM
AWQ base model	40GB+	Use hugging-quants/Meta-Llama-3.3-70B-Instruct-AWQ-INT4

---

📊 Training Configuration

model:
  name: meta-llama/Llama-3.3-70B-Instruct
  max_seq_length: 4096

quantization:
  load_in_4bit: true
  bnb_4bit_quant_type: nf4
  bnb_4bit_compute_dtype: bfloat16
  bnb_4bit_use_double_quant: true

lora:
  r: 64
  lora_alpha: 128
  target_modules: [q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj]

training:
  num_train_epochs: 2
  per_device_train_batch_size: 2
  gradient_accumulation_steps: 16
  learning_rate: 2.0e-5
  gradient_checkpointing: true
  optim: paged_adamw_8bit

---

⚠️ Limitations & Disclaimer

• Not a lawyer. Onyx is an AI tool and does not provide legal advice. Always consult a licensed attorney for legal matters.
• 83.2% accuracy — solid for document analysis, but not infallible. Verify critical findings independently.
• Training data cutoff — legal statutes and case law change. Always verify current law.
• Jurisdiction — primarily trained on US federal law and EU regulations. Coverage of state law and other jurisdictions is limited.

---

🗺️ Roadmap

• v2: Fix warm-start Phase 1 → Phase 2 weight transfer
• v2: 33,500 example expanded dataset (CUAD, CourtListener, MultiLexSum, BillSum, ContractNLI)
• v2: 90%+ accuracy target
• v3: Add user's own case documents for domain-specific fine-tuning
• Merge + GGUF export for llama.cpp deployment

---

📜 License

This adapter inherits the Llama 3.3 Community License. For commercial use, review Meta's license terms.

---

Built with ❤️ by Advait Shewale -- ExcavatedStudios