Add inference script with post-processing and update README

The inference.py script loads the fine-tuned model and generates
recipes with automatic cleanup of generation artifacts (trailing
comments, empty bullets, malformed lines, truncated text).

Co-Authored-By: Claude Opus 4.6 <[email protected]>

README.md

@@ -93,45 +93,58 @@ After a full run:
 - `./processed_data/val/` — tokenized validation split (Arrow format)
 - `./processed_data/lora_adapter/` — trained LoRA adapter weights
 
-## Testing the Trained Model
+## Inference
 
-### Load from local adapter
+A standalone inference script with built-in post-processing is provided. It removes common generation artifacts (trailing comments, empty bullets, malformed lines, truncated text).
 
-```python
-from peft import PeftModel
-from transformers import AutoModelForCausalLM, AutoTokenizer
-import torch
+### Quick start
 
-# Load base model + LoRA adapter
-base_model = AutoModelForCausalLM.from_pretrained("google/gemma-2b")
-model = PeftModel.from_pretrained(base_model, "./processed_data/lora_adapter")
-tokenizer = AutoTokenizer.from_pretrained("./processed_data/lora_adapter")
-model.eval()
+```bash
+python inference.py --prompt "Recipe for chocolate chip cookies:"
+```
 
-# Generate a recipe
-prompt = "Recipe for chocolate chip cookies:\n"
-inputs = tokenizer(prompt, return_tensors="pt")
-with torch.no_grad():
-    outputs = model.generate(
-        **inputs,
-        max_new_tokens=256,
-        temperature=0.7,
-        top_p=0.9,
-        do_sample=True,
-        repetition_penalty=1.2,
-    )
-print(tokenizer.decode(outputs[0], skip_special_tokens=True))
+### Options
+
+| Flag | Default | Description |
+|---|---|---|
+| `--prompt` | `"Recipe for chocolate chip cookies:"` | Prompt for recipe generation |
+| `--adapter` | `./processed_data/lora_adapter` | Path to LoRA adapter (local or HuggingFace Hub ID) |
+| `--model` | `google/gemma-2b` | Base model name |
+| `--max-tokens` | `256` | Maximum new tokens to generate |
+| `--temperature` | `0.7` | Sampling temperature |
+| `--raw` | off | Show raw output without post-processing |
+| `--save` | none | Save output to file |
+
+### Examples
+
+```bash
+# Generate with post-processing (default)
+python inference.py --prompt "Recipe for pasta carbonara:"
+
+# Compare raw vs cleaned output
+python inference.py --prompt "Recipe for tomato soup:" --raw
+
+# Save to file
+python inference.py --prompt "Recipe for banana bread:" --save output.txt
+
+# Use adapter from HuggingFace Hub
+python inference.py --adapter ClaireLee2429/gemma-2b-recipes-lora --prompt "Recipe for chicken stir fry:"
 ```
 
-### Load from HuggingFace Hub
+### Using the model directly in Python
 
 ```python
 from peft import PeftModel
 from transformers import AutoModelForCausalLM, AutoTokenizer
 
+# From local adapter
 base_model = AutoModelForCausalLM.from_pretrained("google/gemma-2b")
-model = PeftModel.from_pretrained(base_model, "ClaireLee2429/gemma-2b-recipes-lora")
-tokenizer = AutoTokenizer.from_pretrained("ClaireLee2429/gemma-2b-recipes-lora")
+model = PeftModel.from_pretrained(base_model, "./processed_data/lora_adapter")
+tokenizer = AutoTokenizer.from_pretrained("./processed_data/lora_adapter")
+
+# Or from HuggingFace Hub
+# model = PeftModel.from_pretrained(base_model, "ClaireLee2429/gemma-2b-recipes-lora")
+# tokenizer = AutoTokenizer.from_pretrained("ClaireLee2429/gemma-2b-recipes-lora")
 ```
 
 ### Sample output

inference.py

@@ -0,0 +1,213 @@
+"""
+Standalone inference script for the fine-tuned recipe generation model.
+
+Usage:
+    python inference.py --prompt "Recipe for chocolate chip cookies:"
+    python inference.py --prompt "Recipe for pasta carbonara:" --save output.txt
+    python inference.py --prompt "Recipe for banana bread:" --raw
+    python inference.py --adapter ClaireLee2429/gemma-2b-recipes-lora --prompt "Recipe for soup:"
+"""
+
+import argparse
+import re
+
+import torch
+from peft import PeftModel
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+
+def clean_recipe(text: str) -> str:
+    """Post-process generated recipe text to remove artifacts."""
+    lines = text.split("\n")
+    cleaned = []
+
+    for line in lines:
+        stripped = line.strip()
+
+        # Remove empty or malformed bullet lines (e.g., "- ", "- .", "- ,", "- AZ")
+        if re.match(r"^-\s*[.,;:]*\s*$", stripped):
+            continue
+
+        # Remove short junk bullets (single word/number fragments like "- AZ", "- 12-07-02.")
+        if re.match(r"^-\s+\S{1,10}$", stripped) and not re.match(r"^-\s+\d+", stripped):
+            # Allow numeric items like "- 1 cup" but skip junk like "- AZ"
+            words_after_dash = stripped[2:].strip()
+            if len(words_after_dash.split()) <= 1 and not any(
+                c.islower() for c in words_after_dash
+            ):
+                continue
+
+        # Stop at trailing commentary sections
+        if re.match(r"^-?\s*Notes?:", stripped, re.IGNORECASE):
+            break
+        if re.match(r"^-?\s*Recipe (from|by|submitted)", stripped, re.IGNORECASE):
+            break
+        if re.match(r"^-?\s*Source:", stripped, re.IGNORECASE):
+            break
+        if re.match(
+            r"^-\s+(I |My |This is |You can |That |He |She |We |It |Visit )",
+            stripped,
+        ):
+            break
+        if re.match(r"^-\s+Bon App", stripped):
+            break
+
+        cleaned.append(line)
+
+    # Remove duplicate consecutive lines
+    deduped = []
+    for line in cleaned:
+        if not deduped or line.strip() != deduped[-1].strip():
+            deduped.append(line)
+
+    # Trim trailing incomplete line (doesn't end with punctuation)
+    while deduped:
+        last = deduped[-1].strip()
+        if not last:
+            deduped.pop()
+            continue
+        if last and last[-1] not in ".!?)\":;":
+            deduped.pop()
+        else:
+            break
+
+    # Remove trailing blank lines
+    while deduped and not deduped[-1].strip():
+        deduped.pop()
+
+    return "\n".join(deduped)
+
+
+def load_model(model_name: str, adapter_path: str):
+    """Load the base model with LoRA adapter."""
+    use_cuda = torch.cuda.is_available()
+    use_mps = torch.backends.mps.is_available()
+
+    dtype = torch.bfloat16 if use_cuda else torch.float32
+    if use_cuda:
+        device_map = "auto"
+    elif use_mps:
+        device_map = {"": "mps"}
+    else:
+        device_map = {"": "cpu"}
+
+    device_name = "CUDA" if use_cuda else ("MPS" if use_mps else "CPU")
+    print(f"Loading base model ({device_name})...")
+
+    base_model = AutoModelForCausalLM.from_pretrained(
+        model_name, torch_dtype=dtype, device_map=device_map
+    )
+
+    print(f"Loading LoRA adapter from {adapter_path}...")
+    model = PeftModel.from_pretrained(base_model, adapter_path)
+    model.eval()
+
+    tokenizer = AutoTokenizer.from_pretrained(adapter_path)
+
+    device = "cuda" if use_cuda else ("mps" if use_mps else "cpu")
+    return model, tokenizer, device
+
+
+def generate_recipe(
+    model,
+    tokenizer,
+    device: str,
+    prompt: str,
+    max_new_tokens: int = 256,
+    temperature: float = 0.7,
+    raw: bool = False,
+) -> str:
+    """Generate a recipe from a prompt and optionally post-process."""
+    inputs = tokenizer(prompt, return_tensors="pt").to(device)
+
+    with torch.no_grad():
+        outputs = model.generate(
+            **inputs,
+            max_new_tokens=max_new_tokens,
+            temperature=temperature,
+            top_p=0.9,
+            do_sample=True,
+            repetition_penalty=1.2,
+        )
+
+    text = tokenizer.decode(outputs[0], skip_special_tokens=True)
+
+    if raw:
+        return text
+    return clean_recipe(text)
+
+
+def main():
+    parser = argparse.ArgumentParser(description="Generate recipes with the fine-tuned model")
+    parser.add_argument(
+        "--prompt",
+        type=str,
+        default="Recipe for chocolate chip cookies:\n",
+        help="Prompt for recipe generation",
+    )
+    parser.add_argument(
+        "--adapter",
+        type=str,
+        default="./processed_data/lora_adapter",
+        help="Path to LoRA adapter (local or HuggingFace Hub ID)",
+    )
+    parser.add_argument(
+        "--model",
+        type=str,
+        default="google/gemma-2b",
+        help="Base model name",
+    )
+    parser.add_argument(
+        "--max-tokens",
+        type=int,
+        default=256,
+        help="Maximum new tokens to generate",
+    )
+    parser.add_argument(
+        "--temperature",
+        type=float,
+        default=0.7,
+        help="Sampling temperature",
+    )
+    parser.add_argument(
+        "--raw",
+        action="store_true",
+        help="Show raw output without post-processing",
+    )
+    parser.add_argument(
+        "--save",
+        type=str,
+        default=None,
+        help="Save output to file",
+    )
+
+    args = parser.parse_args()
+
+    # Ensure prompt ends with newline
+    prompt = args.prompt if args.prompt.endswith("\n") else args.prompt + "\n"
+
+    model, tokenizer, device = load_model(args.model, args.adapter)
+
+    print(f"\nPrompt: {prompt.strip()}")
+    print("-" * 40)
+
+    result = generate_recipe(
+        model,
+        tokenizer,
+        device,
+        prompt,
+        max_new_tokens=args.max_tokens,
+        temperature=args.temperature,
+        raw=args.raw,
+    )
+
+    print(result)
+
+    if args.save:
+        with open(args.save, "w") as f:
+            f.write(result + "\n")
+        print(f"\nSaved to {args.save}")
+
+
+if __name__ == "__main__":
+    main()

inference_results.txt

@@ -0,0 +1,194 @@
+============================================================
+PROMPT: Recipe for chocolate chip cookies:
+------------------------------------------------------------
+Recipe for chocolate chip cookies:
+Ingredients:
+- 1/2 cup butter
+- 1/4 cup sugar
+- 1/4 cup packed brown sugar
+- 3/4 cup flour
+- 1/2 teaspoon baking soda
+- 1/2 teaspoon salt
+- 1 egg
+- 1 teaspoon vanilla
+- 1/2 cup chocolate chips
+
+Directions:
+- In a medium bowl, cream together the butter and sugars.
+- Add in the egg and vanilla, mixing until combined.
+- In another bowl, whisk together the flour, baking soda and salt.
+- Add to the creamed mixture alternately with the chocolate chips, ending with the dry ingredients.
+- Stir in gently using a rubber spatula.
+- Drop by rounded teaspoonfuls onto ungreased cookie sheets.
+- Bake at 350 degrees F for 9 minutes.
+- Cool on wire racks before serving.
+- Yield: 2 dozen cookies.
+- Notes:
+- I use this recipe all the time.
+- The only thing that's changed is adding raisins!
+- It makes it more like a raisin cookie, but still delicious.
+- Recipe may be doubled or tripled if desired.
+- If you make them small
+
+============================================================
+PROMPT: Recipe for pasta carbonara:
+------------------------------------------------------------
+Recipe for pasta carbonara:
+Ingredients:
+- 100 g spaghetti
+- 100 g smoked bacon or pancetta
+- 250 g mushrooms
+- 2 eggs
+- 1 tablespoon olive oil
+- 1 tablespoon white wine
+- 1 teaspoon freshly grated parmesan cheese
+
+Directions:
+- Cook the pasta according to package instructions.
+- Meanwhile, brown the bacon in a frying pan with some olive oil.
+- Add the mushrooms and cook them until they are tender (about 10 minutes).
+- Add the cooked pasta to the mushroom mixture along with the eggs and stir well.
+- Stir in the wine and then sprinkle over the grated parmesan cheese.
+- Season with salt and pepper and serve immediately.
+- Enjoy!
+- Notes:
+- The recipe is based on a traditional Italian dish called pasta carbonara.
+- It's usually made with bacon, but you can also use pancetta or even ham.
+- I like to add some mushrooms too, but it's not necessary.
+- If you don't have any fresh mushrooms, you can use frozen ones instead.
+- Be sure to cook the pasta al dente (just shy of being fully cooked) so that it's still nice
+
+============================================================
+PROMPT: Recipe for banana bread:
+------------------------------------------------------------
+Recipe for banana bread:
+Ingredients:
+- 2 c. sugar
+- 1/2 c. oil
+- 3 eggs
+- 1 c. mashed banana
+- 3 c. flour
+- 2 tsp. baking soda
+- 1 tsp. cinnamon
+- 1 tsp. salt
+- 1 tsp. vanilla
+- 1 c. chopped nuts
+
+Directions:
+- Mix together first four ingredients.
+- Mix the next five ingredients and add to previous mixture.
+- Stir well.
+- Add nuts on top of batter.
+- Bake at 400° for 30 minutes or until toothpick comes out clean.
+- Makes about 9 to 8 loaves.
+- Serves 12 to 6 servings.
+- Recipe from my mother, who was born in 1898.
+- This is a great recipe!
+- My mother had no electric mixer; she used an old fashioned wooden spoon when mixing.
+- It works fine!
+- (This recipe can be halved.)
+- I have also made this with applesauce instead of bananas.
+- The loaf will rise more than if you use bananas.
+- If you want a denser loaf, substitute butter for the oil
+
+============================================================
+PROMPT: Recipe for chicken stir fry:
+------------------------------------------------------------
+Recipe for chicken stir fry:
+Ingredients:
+- 1 lb. boneless chicken breasts, cut into bite size pieces
+- 1/2 c. soy sauce
+- 1/4 c. brown sugar
+- 1 tsp. cornstarch
+- 3/4 c. water
+- 2 Tbsp. oil
+
+Directions:
+- Mix all ingredients and place in microwaveable dish.
+- Cover with plastic wrap and cook on high for 15 minutes.
+- Stir and continue cooking until desired doneness is reached.
+- May also be cooked in a skillet over medium heat using the same method.
+- Serves 4 to 6 people.
+- Note: If you want it spicier, add more chili paste.
+- For extra flavor, try adding some minced ginger.
+- This can also be done in an oven at 350° for about 30 minutes or so.
+- (Serves 4 to 8).
+- Recipe by: Judy G., Mesa, Ariz.
+- .
+- ,
+- AZ
+- 12-07-02.
+- .
+- .
+- .
+- .
+- .
+- .
+- .
+- .
+- .
+-
+
+============================================================
+PROMPT: Recipe for tomato soup:
+------------------------------------------------------------
+Recipe for tomato soup:
+Ingredients:
+- 1 (28 ounce) can crushed tomatoes
+- 1 (4 ounce) can tomato paste
+- 1 tablespoon dried basil
+- 1/2 tablespoon salt
+- 1/2 tablespoon sugar
+- 1/2 teaspoon garlic powder
+- 1/4 teaspoon red pepper flakes
+
+Directions:
+- Combine all ingredients in a large pot and bring to a boil.
+- Reduce heat, cover and simmer until thickened, about 1 hour.
+- Serve over grilled cheese sandwiches or with tortilla chips.
+- Enjoy!
+- Recipe submitted by:
+- Anonymous
+- Source:
+- Food.com
+- 
+- 
+- 
+- 
+- 
+- 
+- 
+- 
+- 
+- 
+- 
+- 
+- 
+- 
+- 
+- 
+- 
+- 
+- 
+- 
+- 
+- 
+- 
+- 
+- 
+- 
+- 
+- 
+- 
+- 
+- 
+- 
+- 
+- 
+- 
+- 
+- 
+- 
+- 
+- 
+