| <!DOCTYPE html> |
| <html lang="en"> |
| <head> |
| <meta charset="UTF-8"> |
| <title>AI Code Maintainability Scoring & Refactoring Engine — Revision Notes</title> |
| <style> |
| :root { |
| --bg: #0d1117; |
| --card: #161b22; |
| --border: #30363d; |
| --text: #e6edf3; |
| --muted: #8b949e; |
| --accent: #58a6ff; |
| --green: #3fb950; |
| --orange: #d29922; |
| --red: #f85149; |
| --purple: #bc8cff; |
| } |
| * { box-sizing: border-box; } |
| body { |
| background: var(--bg); |
| color: var(--text); |
| font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; |
| line-height: 1.6; |
| margin: 0; |
| padding: 0; |
| } |
| .wrap { max-width: 880px; margin: 0 auto; padding: 32px 20px 80px; } |
| h1 { |
| font-size: 1.7em; |
| border-bottom: 2px solid var(--accent); |
| padding-bottom: 12px; |
| margin-bottom: 4px; |
| } |
| .subtitle { color: var(--muted); margin-bottom: 28px; font-size: 0.95em; } |
| h2 { |
| color: var(--accent); |
| font-size: 1.25em; |
| margin-top: 42px; |
| border-left: 4px solid var(--accent); |
| padding-left: 12px; |
| } |
| h3 { |
| color: var(--purple); |
| font-size: 1.05em; |
| margin-top: 26px; |
| } |
| .card { |
| background: var(--card); |
| border: 1px solid var(--border); |
| border-radius: 8px; |
| padding: 16px 20px; |
| margin: 14px 0; |
| } |
| .pill { |
| display: inline-block; |
| background: rgba(88,166,255,0.15); |
| color: var(--accent); |
| border-radius: 5px; |
| padding: 2px 8px; |
| font-size: 0.85em; |
| margin-right: 6px; |
| } |
| table { width: 100%; border-collapse: collapse; margin: 14px 0; } |
| th, td { |
| border: 1px solid var(--border); |
| padding: 8px 12px; |
| text-align: left; |
| font-size: 0.95em; |
| } |
| th { background: #1c2229; color: var(--accent); } |
| tr:nth-child(even) { background: rgba(255,255,255,0.02); } |
| code, pre { |
| background: #010409; |
| color: #79c0ff; |
| border-radius: 6px; |
| font-family: "SF Mono", Consolas, monospace; |
| } |
| code { padding: 2px 6px; font-size: 0.9em; } |
| pre { |
| padding: 14px 16px; |
| overflow-x: auto; |
| border: 1px solid var(--border); |
| line-height: 1.5; |
| font-size: 0.88em; |
| } |
| .flow { |
| background: var(--card); |
| border: 1px solid var(--border); |
| border-radius: 8px; |
| padding: 18px 22px; |
| font-family: Consolas, monospace; |
| color: var(--green); |
| white-space: pre; |
| font-size: 0.9em; |
| } |
| .q { |
| color: var(--orange); |
| font-weight: 600; |
| margin-top: 18px; |
| } |
| .summary-box { |
| background: linear-gradient(135deg, rgba(88,166,255,0.1), rgba(188,140,255,0.1)); |
| border: 1px solid var(--accent); |
| border-radius: 8px; |
| padding: 18px 22px; |
| font-style: italic; |
| margin: 16px 0; |
| } |
| .warn { |
| background: rgba(248,81,73,0.1); |
| border: 1px solid var(--red); |
| border-radius: 8px; |
| padding: 12px 18px; |
| color: #ffb3ac; |
| font-size: 0.92em; |
| margin: 14px 0; |
| } |
| .num-table td:first-child { color: var(--muted); } |
| .num-table td:last-child { color: var(--green); font-weight: 600; } |
| ul { padding-left: 22px; } |
| li { margin: 4px 0; } |
| </style> |
| </head> |
| <body> |
| <div class="wrap"> |
|
|
| <h1>AI Code Maintainability Scoring & Refactoring Engine</h1> |
| <div class="subtitle">Revision Notes — Light-touch interview prep (2–3 sentences + one technical detail per topic)</div> |
|
|
| <div class="warn"> |
| <strong>Scope reminder:</strong> this project was deliberately scoped as light-touch, not a deep-dive like Urdu Sentiment or Medical Image. Use the "One-Line Summary" and "Quick Answers" sections as your primary prep. The deeper sections exist so you're not caught flat-footed on a follow-up, not so you memorize all of it. |
| </div> |
|
|
| <h2>What Is This Project? (Elevator Pitch)</h2> |
| <div class="card"> |
| "I built a two-phase AI system that scores the structural quality of Python code and then autonomously refactors risky code to improve it. Phase 1 uses AST parsing + a Random Forest classifier to predict a maintainability risk score. Phase 2 uses a CodeT5 deep learning model to generate and iteratively select better versions of risky code." |
| </div> |
|
|
| <h2>The Problem It Solves</h2> |
| <div class="card"> |
| Traditional linters (PyLint, Flake8) use rigid, hand-written rules. This engine instead <strong>learns</strong> what "risky" structure looks like from data — deep nesting, high complexity, large functions — and then goes a step further than any linter by actually generating improved code, not just flagging problems. |
| </div> |
|
|
| <h2>Full Pipeline Flow</h2> |
| <div class="flow">Input Code |
| │ |
| ▼ |
| PHASE 1 — Evaluator |
| AST Parsing → Feature Extraction → ML Scoring → Explanation |
| │ |
| ▼ |
| Risk Score (0–100) + Top 3 Reasons |
| │ |
| ▼ |
| PHASE 2 — Refactorer (only if risky) |
| Generate Candidates (CodeT5) → Validate → Re-score via Phase 1 → Select Best |
| │ (loop until target score or max iterations) |
| ▼ |
| Improved Code</div> |
|
|
| <h2>Phase 1 — The Evaluator</h2> |
|
|
| <h3>1. AST Analyzer — 11 Structural Features</h3> |
| <div class="card"> |
| Parses code into an Abstract Syntax Tree (not line-by-line text reading — actual structural traversal via <code>ast.walk()</code>) and extracts 11 signals: |
| <table> |
| <tr><th>Feature</th><th>What it measures</th></tr> |
| <tr><td>max_nesting_depth</td><td><strong>Strongest predictor</strong> — deepest if/for/while/try nesting</td></tr> |
| <tr><td>cyclomatic_complexity</td><td>1 + every decision point (if/for/while/except/with/assert/bool-op)</td></tr> |
| <tr><td>avg_function_length</td><td>Mean lines per function</td></tr> |
| <tr><td>num_functions, num_loops, num_if, num_try_except, num_return</td><td>Raw structural counts</td></tr> |
| <tr><td>line_count</td><td>Total file size</td></tr> |
| <tr><td>recursion_flag</td><td>1 if any function calls itself, else 0</td></tr> |
| <tr><td>global_variable_count</td><td>Count of <code>global</code> declarations — hidden state / tight coupling</td></tr> |
| </table> |
| <strong>One technical detail worth knowing cold:</strong> nesting depth is computed via a recursive traversal that increments depth only when it enters a defined set of "nesting nodes" (<code>If, For, While, With, Try, FunctionDef, AsyncFunctionDef, ClassDef</code>) — everything else keeps the current depth. |
| </div> |
|
|
| <h3>2. Dataset Generation</h3> |
| <div class="card"> |
| Synthetic dataset generated from code templates, not scraped real-world code. <strong>220 total samples, perfectly balanced: 110 labeled Clean (0), 110 labeled Risky (1).</strong> Reproducible via <code>random.seed(42)</code> — same dataset every run. |
| </div> |
|
|
| <h3>3. Feature Pipeline</h3> |
| <div class="card"> |
| Converts the feature dictionary into a fixed-order numeric vector (<code>FEATURE_SCHEMA</code> defines the order — this order must stay identical between training and inference or predictions break silently). Scales with <code>StandardScaler</code> (mean=0, std=1) so no single feature like <code>line_count</code> dominates just because its raw numbers are bigger. Scaler is pickled and reused at inference time — never re-fit on new data. |
| </div> |
|
|
| <h3>4. Model Training</h3> |
| <div class="card"> |
| Two models trained side by side on an 80/20 stratified split (176 train / 44 test): |
| <table> |
| <tr><th>Model</th><th>Key hyperparameters</th></tr> |
| <tr><td>Random Forest <span class="pill">primary / production</span></td><td>200 trees, max_depth=10, min_samples_split=4, class_weight="balanced"</td></tr> |
| <tr><td>XGBoost <span class="pill">comparison only</span></td><td>200 estimators, max_depth=6, learning_rate=0.1, subsample=0.8</td></tr> |
| </table> |
| <strong>Random Forest is the one actually used in the live Scoring API</strong> — XGBoost is trained and evaluated for comparison but not deployed. |
| </div> |
| <div class="warn"> |
| No saved accuracy/F1 number exists in the repo — the evaluation function prints it live but doesn't persist it to a file. Run <code>python model_trainer.py</code> before your interview and note the actual number rather than guessing one. |
| </div> |
|
|
| <h3>5. Explanation Engine</h3> |
| <div class="card"> |
| Answers "why is this risky" without recomputing anything new. Logic: for each of the 11 features, check if the code's actual value exceeds a fixed threshold (e.g. <code>max_nesting_depth > 3</code>, <code>cyclomatic_complexity > 5</code>, <code>global_variable_count > 1</code>). Of the features that exceed threshold, rank by the model's feature importance and return the <strong>top 3</strong> as plain-English sentences (e.g. "Deep nesting detected (depth: 7)"). |
| </div> |
|
|
| <h3>6. Scoring API — Final Output</h3> |
| <div class="card"> |
| Single entry point: <code>evaluate(code)</code>. Runs the model's predicted probability of the "risky" class × 100 as the risk score. |
| <table> |
| <tr><th>Risk Score</th><th>Level</th></tr> |
| <tr><td>0–30</td><td style="color:var(--green)">Low</td></tr> |
| <tr><td>31–60</td><td style="color:var(--orange)">Medium</td></tr> |
| <tr><td>61–100</td><td style="color:var(--red)">High</td></tr> |
| </table> |
| <pre>{ |
| "risk_score": 82, |
| "risk_level": "High", |
| "confidence": 0.82, |
| "top_risk_factors": ["Deep nesting detected (depth: 7)", ...] |
| }</pre> |
| Properties worth naming if asked: deterministic, stateless, fast (no training happens at inference time). |
| </div> |
|
|
| <h2>Phase 2 — The Refactorer</h2> |
|
|
| <div class="card"> |
| Only runs on code Phase 1 flagged as risky. Uses <strong>CodeT5</strong> (a deep learning code-generation model) to produce multiple refactored candidates, then uses Phase 1's own <code>evaluate()</code> as a reward/ranking function to pick the best one. |
| </div> |
|
|
| <h3>1. Candidate Generation — 3 Strategies</h3> |
| <div class="card"> |
| Generates one candidate per strategy, each with a different prompt and temperature: |
| <table> |
| <tr><th>Strategy</th><th>Temperature</th></tr> |
| <tr><td>Improve readability and clarity</td><td>0.5 (conservative)</td></tr> |
| <tr><td>Reduce nesting and simplify logic</td><td>0.7</td></tr> |
| <tr><td>Refactor for strict maintainability best practices</td><td>0.85 (more creative)</td></tr> |
| </table> |
| <strong>One technical detail:</strong> temperature controls how much the model deviates from the "safe" rewrite — low temperature stays close to minimal edits, high temperature takes bigger structural risks. |
| </div> |
|
|
| <h3>2. Validation → Selection → Iteration</h3> |
| <div class="card"> |
| Each candidate is checked to <strong>actually compile</strong> (no syntax errors) before it's even considered. Valid candidates are re-scored by feeding them back through the Phase 1 <code>evaluate()</code> API, and the lowest risk score wins that round. |
| <br><br> |
| <strong>Iterative loop defaults:</strong> <code>target_score=20</code>, <code>max_iterations=3</code>. Stops early if the target is hit, or after 3 rounds regardless. Starts by initializing the best-known score to <code>float('inf')</code> — a simple trick meaning "anything found is automatically an improvement over nothing." |
| </div> |
|
|
| <h2>Tech Stack</h2> |
| <div class="card"> |
| <table> |
| <tr><th>Component</th><th>Technology</th></tr> |
| <tr><td>Structural Analysis</td><td>Python <code>ast</code> module</td></tr> |
| <tr><td>Risk Prediction</td><td>Random Forest (primary), XGBoost (comparison) — scikit-learn</td></tr> |
| <tr><td>Refactoring</td><td>CodeT5 (Transformers / PyTorch)</td></tr> |
| <tr><td>API</td><td>Unified <code>main.py</code> entry point + <code>api_server.py</code></td></tr> |
| <tr><td>Deployment</td><td>Live at ai-code-maintainability.hmuhammadusman.com</td></tr> |
| </table> |
| </div> |
|
|
| <h2>Numbers to Remember</h2> |
| <div class="card"> |
| <table class="num-table"> |
| <tr><td>Structural features extracted</td><td>11</td></tr> |
| <tr><td>Total dataset samples</td><td>220 (110 Clean / 110 Risky — balanced)</td></tr> |
| <tr><td>Train / test split</td><td>176 / 44 (80/20, stratified)</td></tr> |
| <tr><td>Random Forest trees</td><td>200 (max_depth=10)</td></tr> |
| <tr><td>Risk score range</td><td>0–100</td></tr> |
| <tr><td>Top risk factors surfaced</td><td>3</td></tr> |
| <tr><td>Refactor candidate strategies</td><td>3 (temps 0.5 / 0.7 / 0.85)</td></tr> |
| <tr><td>Default optimizer target / max iterations</td><td>20 / 3</td></tr> |
| <tr><td>Reproducibility seed</td><td>42</td></tr> |
| </table> |
| </div> |
|
|
| <h2>Quick Answers — Likely Questions</h2> |
|
|
| <div class="q">"Why Random Forest over a simpler rule-based linter?"</div> |
| <div class="card">Rule-based linters need every threshold hand-tuned per rule. A trained model learns which combinations of features actually correlate with risk from data, and can weigh 11 signals together instead of checking them independently.</div> |
|
|
| <div class="q">"Why two models (RF + XGBoost) if only one is deployed?"</div> |
| <div class="card">Comparison during development — training both and comparing accuracy tells you whether the extra complexity of boosting is worth it on this dataset size before committing to one in production.</div> |
|
|
| <div class="q">"Isn't 220 samples very small for ML?"</div> |
| <div class="card">Yes — be upfront about this if asked. It's a synthetic, template-generated dataset, not real-world code, which is both the honest limitation and a natural "what I'd improve next" answer (train on real open-source repos labeled by actual maintainability metrics).</div> |
|
|
| <div class="q">"How does Phase 2 know a candidate is actually better, not just different?"</div> |
| <div class="card">It doesn't trust the language model's own judgment — every candidate gets re-scored through the exact same Phase 1 <code>evaluate()</code> pipeline used on the original code, so "better" is measured by the same objective risk score, not by how the refactor looks.</div> |
|
|
| <div class="q">"What would you improve?"</div> |
| <div class="card">Real-world training data instead of synthetic templates; persisting evaluation metrics instead of only printing them; and multi-file / cross-function analysis, since right now every file is scored in isolation.</div> |
|
|
| <h2>One-Line Summary to Open With</h2> |
| <div class="summary-box"> |
| "I built a two-phase AI system — Phase 1 uses AST parsing and a Random Forest model to score Python code's maintainability risk out of 100 and explain why, and Phase 2 uses a CodeT5 deep learning model to iteratively generate and select better refactored versions of risky code, using the Phase 1 score itself as the judge of improvement." |
| </div> |
|
|
| </div> |
| </body> |
| </html> |
|
|
