ysong21/entropy-v1-lora

Try this model on Entropy Studio

---

Entropy LoRA for Gemma 3 27B IT

This repository contains a PEFT LoRA adapter for google/gemma-3-27b-it that rewrites polished AI text into more human, varied prose while aiming to preserve meaning.

This is an adapter, not a standalone model. You must load it on top of the base Gemma 3 27B IT weights.

Intended Use

• Post-processing AI-written drafts to restore voice, texture, and natural variation.
• Professional and general-audience rewriting where you want fewer “AI markers” without changing meaning.

Not intended for deception, academic dishonesty, or other misleading uses.

Prompting (Training Trigger)

The fine-tuning data uses short “rewrite-to-human” triggers. For best results, start your prompt with a similar trigger. A canonical trigger from the training set is:

Polish this AI passage to feel more human:
{PASTE_TEXT_HERE}

The training data also contains close variants like:

• Rephrase this AI passage to feel more human:
• Convert this AI passage into a more human-sounding version:

Usage

vLLM (Runtime LoRA)

This adapter is rank 64, so vLLM must be started with --max-lora-rank 64 (or higher).

vllm serve google/gemma-3-27b-it \
  --served-model-name google/gemma-3-27b-it \
  --dtype bfloat16 \
  --enable-lora \
  --max-lora-rank 64 \
  --lora-modules entropy-v1=ysong21/entropy-v1-lora

Then, in OpenAI-compatible clients, select the adapter by setting model to the served LoRA name (for the example above, entropy-v1).

Transformers + PEFT

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_id = "google/gemma-3-27b-it"
adapter_id = "ysong21/entropy-v1-lora"

tok = AutoTokenizer.from_pretrained(base_id)
base = AutoModelForCausalLM.from_pretrained(base_id, device_map="auto")
model = PeftModel.from_pretrained(base, adapter_id)

prompt = "Polish this AI passage to feel more human:\n" + "..."
inputs = tok(prompt, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=512)
print(tok.decode(out[0], skip_special_tokens=True))

Training

• Base model: google/gemma-3-27b-it
• Dataset: N8Programs/unslop-good (1000 rewrite pairs)
• LoRA: r=64, alpha=128, dropout=0.05
• Target modules: q_proj,k_proj,v_proj,o_proj,gate_proj,up_proj,down_proj

Evaluation

We ran an offline validation study on 70 Project Gutenberg passages that do not overlap with the passages from unslop-good.

Metric: Conditional Bits Per Character

We evaluate conditional negative log-likelihood on the target (human) text only, given the “slopped/polished” input. We then normalize by character count to make results more comparable across tokenizers:

• Let NLL be total negative log-likelihood of the target tokens in nats.
• Let C be the number of characters in the target text.
• nats_per_char = NLL / C
• bits_per_char = nats_per_char / ln(2)

Lower bits_per_char is better.

Results (70 Gutenberg Examples)

System	bits_per_char (↓)	Rel. vs Unslopper
N8Programs/Unslopper-30B-A3B-bf16 (baseline)	0.37522
ysong21/entropy-v1-lora	0.35877	+4.38%
Base google/gemma-3-27b-it	0.99565	-165.35%

Notes:

• Token-level perplexity depends on tokenizer, so bits_per_char is the primary cross-model comparison here.
• This evaluation is teacher-forced likelihood scoring (not a generation quality benchmark).
• The validation distribution is Gutenberg-derived, so results may not fully transfer to modern web/business writing without additional data.