AI Code Maintainability Scoring & Refactoring Engine
Revision Notes — Light-touch interview prep (2–3 sentences + one technical detail per topic)
Scope reminder: this project was deliberately scoped as light-touch, not a deep-dive like Urdu Sentiment or Medical Image. Use the "One-Line Summary" and "Quick Answers" sections as your primary prep. The deeper sections exist so you're not caught flat-footed on a follow-up, not so you memorize all of it.
What Is This Project? (Elevator Pitch)
"I built a two-phase AI system that scores the structural quality of Python code and then autonomously refactors risky code to improve it. Phase 1 uses AST parsing + a Random Forest classifier to predict a maintainability risk score. Phase 2 uses a CodeT5 deep learning model to generate and iteratively select better versions of risky code."
The Problem It Solves
Traditional linters (PyLint, Flake8) use rigid, hand-written rules. This engine instead learns what "risky" structure looks like from data — deep nesting, high complexity, large functions — and then goes a step further than any linter by actually generating improved code, not just flagging problems.
Full Pipeline Flow
Input Code
│
▼
PHASE 1 — Evaluator
AST Parsing → Feature Extraction → ML Scoring → Explanation
│
▼
Risk Score (0–100) + Top 3 Reasons
│
▼
PHASE 2 — Refactorer (only if risky)
Generate Candidates (CodeT5) → Validate → Re-score via Phase 1 → Select Best
│ (loop until target score or max iterations)
▼
Improved Code
Phase 1 — The Evaluator
1. AST Analyzer — 11 Structural Features
Parses code into an Abstract Syntax Tree (not line-by-line text reading — actual structural traversal via
ast.walk()) and extracts 11 signals:
| Feature | What it measures |
|---|
| max_nesting_depth | Strongest predictor — deepest if/for/while/try nesting |
| cyclomatic_complexity | 1 + every decision point (if/for/while/except/with/assert/bool-op) |
| avg_function_length | Mean lines per function |
| num_functions, num_loops, num_if, num_try_except, num_return | Raw structural counts |
| line_count | Total file size |
| recursion_flag | 1 if any function calls itself, else 0 |
| global_variable_count | Count of global declarations — hidden state / tight coupling |
One technical detail worth knowing cold: nesting depth is computed via a recursive traversal that increments depth only when it enters a defined set of "nesting nodes" (
If, For, While, With, Try, FunctionDef, AsyncFunctionDef, ClassDef) — everything else keeps the current depth.
2. Dataset Generation
Synthetic dataset generated from code templates, not scraped real-world code. 220 total samples, perfectly balanced: 110 labeled Clean (0), 110 labeled Risky (1). Reproducible via random.seed(42) — same dataset every run.
3. Feature Pipeline
Converts the feature dictionary into a fixed-order numeric vector (FEATURE_SCHEMA defines the order — this order must stay identical between training and inference or predictions break silently). Scales with StandardScaler (mean=0, std=1) so no single feature like line_count dominates just because its raw numbers are bigger. Scaler is pickled and reused at inference time — never re-fit on new data.
4. Model Training
Two models trained side by side on an 80/20 stratified split (176 train / 44 test):
| Model | Key hyperparameters |
|---|
| Random Forest primary / production | 200 trees, max_depth=10, min_samples_split=4, class_weight="balanced" |
| XGBoost comparison only | 200 estimators, max_depth=6, learning_rate=0.1, subsample=0.8 |
Random Forest is the one actually used in the live Scoring API — XGBoost is trained and evaluated for comparison but not deployed.
No saved accuracy/F1 number exists in the repo — the evaluation function prints it live but doesn't persist it to a file. Run python model_trainer.py before your interview and note the actual number rather than guessing one.
5. Explanation Engine
Answers "why is this risky" without recomputing anything new. Logic: for each of the 11 features, check if the code's actual value exceeds a fixed threshold (e.g. max_nesting_depth > 3, cyclomatic_complexity > 5, global_variable_count > 1). Of the features that exceed threshold, rank by the model's feature importance and return the top 3 as plain-English sentences (e.g. "Deep nesting detected (depth: 7)").
6. Scoring API — Final Output
Single entry point:
evaluate(code). Runs the model's predicted probability of the "risky" class × 100 as the risk score.
| Risk Score | Level |
|---|
| 0–30 | Low |
| 31–60 | Medium |
| 61–100 | High |
{
"risk_score": 82,
"risk_level": "High",
"confidence": 0.82,
"top_risk_factors": ["Deep nesting detected (depth: 7)", ...]
}
Properties worth naming if asked: deterministic, stateless, fast (no training happens at inference time).
Phase 2 — The Refactorer
Only runs on code Phase 1 flagged as risky. Uses CodeT5 (a deep learning code-generation model) to produce multiple refactored candidates, then uses Phase 1's own evaluate() as a reward/ranking function to pick the best one.
1. Candidate Generation — 3 Strategies
Generates one candidate per strategy, each with a different prompt and temperature:
| Strategy | Temperature |
|---|
| Improve readability and clarity | 0.5 (conservative) |
| Reduce nesting and simplify logic | 0.7 |
| Refactor for strict maintainability best practices | 0.85 (more creative) |
One technical detail: temperature controls how much the model deviates from the "safe" rewrite — low temperature stays close to minimal edits, high temperature takes bigger structural risks.
2. Validation → Selection → Iteration
Each candidate is checked to actually compile (no syntax errors) before it's even considered. Valid candidates are re-scored by feeding them back through the Phase 1 evaluate() API, and the lowest risk score wins that round.
Iterative loop defaults: target_score=20, max_iterations=3. Stops early if the target is hit, or after 3 rounds regardless. Starts by initializing the best-known score to float('inf') — a simple trick meaning "anything found is automatically an improvement over nothing."
Tech Stack
| Component | Technology |
|---|
| Structural Analysis | Python ast module |
| Risk Prediction | Random Forest (primary), XGBoost (comparison) — scikit-learn |
| Refactoring | CodeT5 (Transformers / PyTorch) |
| API | Unified main.py entry point + api_server.py |
| Deployment | Live at ai-code-maintainability.hmuhammadusman.com |
Numbers to Remember
| Structural features extracted | 11 |
| Total dataset samples | 220 (110 Clean / 110 Risky — balanced) |
| Train / test split | 176 / 44 (80/20, stratified) |
| Random Forest trees | 200 (max_depth=10) |
| Risk score range | 0–100 |
| Top risk factors surfaced | 3 |
| Refactor candidate strategies | 3 (temps 0.5 / 0.7 / 0.85) |
| Default optimizer target / max iterations | 20 / 3 |
| Reproducibility seed | 42 |
Quick Answers — Likely Questions
"Why Random Forest over a simpler rule-based linter?"
Rule-based linters need every threshold hand-tuned per rule. A trained model learns which combinations of features actually correlate with risk from data, and can weigh 11 signals together instead of checking them independently.
"Why two models (RF + XGBoost) if only one is deployed?"
Comparison during development — training both and comparing accuracy tells you whether the extra complexity of boosting is worth it on this dataset size before committing to one in production.
"Isn't 220 samples very small for ML?"
Yes — be upfront about this if asked. It's a synthetic, template-generated dataset, not real-world code, which is both the honest limitation and a natural "what I'd improve next" answer (train on real open-source repos labeled by actual maintainability metrics).
"How does Phase 2 know a candidate is actually better, not just different?"
It doesn't trust the language model's own judgment — every candidate gets re-scored through the exact same Phase 1 evaluate() pipeline used on the original code, so "better" is measured by the same objective risk score, not by how the refactor looks.
"What would you improve?"
Real-world training data instead of synthetic templates; persisting evaluation metrics instead of only printing them; and multi-file / cross-function analysis, since right now every file is scored in isolation.
One-Line Summary to Open With
"I built a two-phase AI system — Phase 1 uses AST parsing and a Random Forest model to score Python code's maintainability risk out of 100 and explain why, and Phase 2 uses a CodeT5 deep learning model to iteratively generate and select better refactored versions of risky code, using the Phase 1 score itself as the judge of improvement."