Initial upload

README.md

@@ -1,13 +1,308 @@
----
-title: MetaSearch
-emoji: 📚
-colorFrom: pink
-colorTo: green
-sdk: gradio
-sdk_version: 6.0.1
-app_file: app.py
-pinned: false
-license: mit
----
-
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+# 🔬 Automated Consensus Analysis API
+
+A comprehensive HuggingFace Spaces API for automated peer review consensus analysis using LLMs and search-augmented verification.
+
+## 🌟 Features
+
+- **Critique Extraction**: Extract structured critique points from peer reviews using Gemini 2.0
+- **Disagreement Detection**: Identify conflicts and disagreements between reviewers
+- **Search-Augmented Verification**: Retrieve supporting/contradicting evidence from academic sources
+- **Disagreement Resolution**: AI-powered resolution using DeepSeek-R1 with reasoning
+- **Meta-Review Generation**: Comprehensive meta-reviews synthesizing all analyses
+- **Rate Limiting**: 10 requests per minute per client
+- **Queue Management**: Up to 3 concurrent pipeline executions
+- **Progress Tracking**: Real-time status updates for long-running tasks
+
+## 🚀 Quick Start
+
+### Local Development
+
+1. **Clone and setup**
+
+```bash
+cd api
+python -m venv venv
+source venv/bin/activate  # On Windows: venv\Scripts\activate
+pip install -r requirements.txt
+```
+
+2. **Configure environment**
+
+```bash
+cp .env.example .env
+# Edit .env with your API keys
+```
+
+3. **Run the application**
+
+```bash
+python app.py
+```
+
+Visit `http://localhost:7860` to access the Gradio interface.
+
+### HuggingFace Spaces Deployment
+
+1. **Create a new Space**
+
+   - Go to [HuggingFace Spaces](https://huggingface.co/spaces)
+   - Click "Create new Space"
+   - Select "Gradio" as SDK
+
+2. **Upload files**
+
+   - Upload all files from the `api/` directory
+   - Ensure `requirements.txt` and `app.py` are in the root
+
+3. **Configure secrets**
+
+   - Go to Space Settings → Repository secrets
+   - Add the following secrets:
+     - `GEMINI_API_KEY`
+     - `OPENROUTER_API_KEY`
+     - `TAVILY_API_KEY`
+     - `SERPAPI_API_KEY`
+
+4. **Deploy**
+   - The Space will automatically build and deploy
+
+## 📚 API Endpoints
+
+### Full Pipeline
+
+**Endpoint**: `/api/full_pipeline`  
+**Method**: POST  
+**Description**: Run the complete consensus analysis pipeline
+
+**Request Body**:
+
+```json
+{
+  "paper_title": "Visual Correspondence Hallucination",
+  "paper_abstract": "This paper investigates...",
+  "reviews": [
+    "Review 1: The methodology is sound but...",
+    "Review 2: While the experiments are comprehensive..."
+  ]
+}
+```
+
+**Response**:
+
+```json
+{
+  "request_id": "req_123456789",
+  "paper_title": "...",
+  "critique_points": [...],
+  "disagreements": [...],
+  "search_results": {...},
+  "resolution": [...],
+  "meta_review": "..."
+}
+```
+
+### Individual Stages
+
+#### Critique Extraction
+
+**Endpoint**: `/api/critique_extraction`  
+**Method**: POST
+
+```json
+{
+  "reviews": ["Review 1 text...", "Review 2 text..."]
+}
+```
+
+#### Disagreement Detection
+
+**Endpoint**: `/api/disagreement_detection`  
+**Method**: POST
+
+```json
+{
+  "critiques": [
+    {"Methodology": [...], "Experiments": [...]},
+    {"Methodology": [...], "Experiments": [...]}
+  ]
+}
+```
+
+#### Search & Retrieval
+
+**Endpoint**: `/api/search_retrieval`  
+**Method**: POST
+
+```json
+{
+  "paper_title": "...",
+  "paper_abstract": "...",
+  "critiques": [...]
+}
+```
+
+#### Progress Tracking
+
+**Endpoint**: `/api/progress/{request_id}`  
+**Method**: GET
+
+**Response**:
+
+```json
+{
+  "stage": "search_retrieval",
+  "progress": 0.5,
+  "message": "Searching for relevant research...",
+  "timestamp": "2025-01-15T10:30:00"
+}
+```
+
+## 🔧 Configuration
+
+### Environment Variables
+
+| Variable                  | Description                    | Default  |
+| ------------------------- | ------------------------------ | -------- |
+| `GEMINI_API_KEY`          | Google Gemini API key          | Required |
+| `OPENROUTER_API_KEY`      | OpenRouter API key (DeepSeek)  | Required |
+| `TAVILY_API_KEY`          | Tavily Search API key          | Required |
+| `SERPAPI_API_KEY`         | SerpAPI key for Google Scholar | Optional |
+| `MAX_REQUESTS_PER_MINUTE` | Rate limit                     | 10       |
+| `MAX_CONCURRENT_TASKS`    | Max parallel executions        | 3        |
+| `MAX_RETRIES`             | Retry attempts on failure      | 5        |
+
+### Rate Limits
+
+- **10 requests per minute** per client IP
+- **Maximum 3 concurrent** pipeline executions
+- **Queue size**: 20 pending requests
+
+## 🏗️ Architecture
+
+```
+api/
+├── app.py                      # Main Gradio application
+├── config.py                   # Configuration management
+├── requirements.txt            # Python dependencies
+├── pipeline/                   # Pipeline modules
+│   ├── critique_extraction.py  # Gemini-based extraction
+│   ├── disagreement_detection.py
+│   ├── search_retrieval.py     # LangChain search agent
+│   ├── disagreement_resolution.py  # DeepSeek resolution
+│   └── meta_review.py
+└── utils/                      # Utility modules
+    ├── rate_limiter.py
+    ├── queue_manager.py
+    └── validators.py
+```
+
+## 🔍 Pipeline Stages
+
+1. **Critique Extraction** (Gemini 2.0)
+
+   - Extracts structured critique points
+   - Categories: Methodology, Experiments, Clarity, Significance, Novelty
+
+2. **Disagreement Detection** (Gemini 2.0)
+
+   - Compares all review pairs
+   - Assigns disagreement scores (0-1)
+   - Identifies specific conflict points
+
+3. **Search & Retrieval** (LangChain + Multi-Search)
+
+   - SoTA research discovery
+   - Evidence validation
+   - Sources: Semantic Scholar, arXiv, Google Scholar, Tavily
+
+4. **Disagreement Resolution** (DeepSeek-R1)
+
+   - Validates critique points
+   - Accepts/rejects based on evidence
+   - Provides resolution summaries
+
+5. **Meta-Review Generation** (DeepSeek-R1)
+   - Synthesizes all analyses
+   - Provides final verdict
+   - Offers actionable recommendations
+
+## 📊 Example Usage
+
+### Python
+
+```python
+import requests
+
+response = requests.post(
+    "https://your-space.hf.space/api/full_pipeline",
+    json={
+        "paper_title": "Novel Approach to X",
+        "paper_abstract": "We propose...",
+        "reviews": [
+            "Reviewer 1: Strong methodology...",
+            "Reviewer 2: Weak experimental validation..."
+        ]
+    }
+)
+
+result = response.json()
+print(result["meta_review"])
+```
+
+### cURL
+
+```bash
+curl -X POST https://your-space.hf.space/api/full_pipeline \
+  -H "Content-Type: application/json" \
+  -d '{
+    "paper_title": "Novel Approach to X",
+    "paper_abstract": "We propose...",
+    "reviews": ["Review 1...", "Review 2..."]
+  }'
+```
+
+## 🛠️ Development
+
+### Running Tests
+
+```bash
+pytest tests/
+```
+
+### Code Quality
+
+```bash
+# Format code
+black .
+
+# Type checking
+mypy .
+
+# Linting
+ruff check .
+```
+
+## 📝 License
+
+See the main project LICENSE file.
+
+## 🤝 Contributing
+
+Contributions welcome! Please:
+
+1. Fork the repository
+2. Create a feature branch
+3. Submit a pull request
+
+## 📧 Support
+
+For issues or questions:
+
+- Open an issue on GitHub
+- Contact: [Your contact info]
+
+## 🔗 Links
+
+- [HuggingFace Space](https://huggingface.co/spaces/your-username/consensus-analysis)
+- [Main Repository](https://github.com/your-username/automated-consensus-analysis)
+- [Documentation](https://your-docs-site.com)

app.py

@@ -0,0 +1,342 @@
+import gradio as gr
+import json
+import os
+from typing import Dict, List, Optional
+from datetime import datetime
+import asyncio
+from functools import wraps
+
+from pipeline.critique_extraction import extract_critiques
+from pipeline.disagreement_detection import detect_disagreements
+from pipeline.search_retrieval import search_and_retrieve
+from pipeline.disagreement_resolution import resolve_disagreements
+from pipeline.meta_review import generate_meta_review
+from utils.rate_limiter import RateLimiter
+from utils.queue_manager import QueueManager
+from utils.validators import validate_paper_input
+
+from dotenv import load_dotenv
+load_dotenv()
+print(os.getenv("GEMINI_API_KEY"))
+
+# Initialize rate limiter and queue manager
+rate_limiter = RateLimiter(max_requests_per_minute=10)
+queue_manager = QueueManager(max_concurrent=3)
+
+# Progress tracking
+progress_store = {}
+
+def update_progress(request_id: str, stage: str, progress: float, message: str):
+    """Update progress for a request"""
+    progress_store[request_id] = {
+        "stage": stage,
+        "progress": progress,
+        "message": message,
+        "timestamp": datetime.now().isoformat()
+    }
+
+async def full_pipeline(
+    paper_title: str,
+    paper_abstract: str,
+    reviews: List[str],
+    request_id: Optional[str] = None
+) -> Dict:
+    """
+    Run the complete consensus analysis pipeline
+    
+    Args:
+        paper_title: Title of the paper
+        paper_abstract: Abstract of the paper
+        reviews: List of review texts
+        request_id: Optional request ID for progress tracking
+    
+    Returns:
+        Complete pipeline results
+    """
+    if not request_id:
+        request_id = f"req_{datetime.now().timestamp()}"
+    
+    results = {
+        "request_id": request_id,
+        "paper_title": paper_title,
+        "paper_abstract": paper_abstract
+    }
+    
+    try:
+        # Stage 1: Critique Extraction
+        update_progress(request_id, "critique_extraction", 0.1, "Extracting critique points...")
+        critique_results = await extract_critiques(reviews)
+        results["critique_points"] = critique_results
+        
+        # Stage 2: Disagreement Detection
+        update_progress(request_id, "disagreement_detection", 0.3, "Detecting disagreements...")
+        disagreement_results = await detect_disagreements(critique_results)
+        results["disagreements"] = disagreement_results
+        
+        # Stage 3: Search & Retrieval
+        update_progress(request_id, "search_retrieval", 0.5, "Searching for relevant research...")
+        search_results = await search_and_retrieve(paper_title, paper_abstract, critique_results)
+        results["search_results"] = search_results
+        
+        # Stage 4: Disagreement Resolution
+        update_progress(request_id, "disagreement_resolution", 0.7, "Resolving disagreements...")
+        resolution_results = await resolve_disagreements(
+            paper_title,
+            paper_abstract,
+            disagreement_results,
+            critique_results,
+            search_results
+        )
+        results["resolution"] = resolution_results
+        
+        # Stage 5: Meta-Review Generation
+        update_progress(request_id, "meta_review", 0.9, "Generating meta-review...")
+        meta_review = await generate_meta_review(
+            paper_title,
+            paper_abstract,
+            resolution_results,
+            search_results
+        )
+        results["meta_review"] = meta_review
+        
+        update_progress(request_id, "complete", 1.0, "Pipeline complete!")
+        return results
+        
+    except Exception as e:
+        update_progress(request_id, "error", 0.0, f"Error: {str(e)}")
+        raise
+
+# Gradio Interface Functions
+def run_full_pipeline_ui(title: str, abstract: str, reviews_json: str) -> str:
+    """UI wrapper for full pipeline"""
+    try:
+        # Validate and parse input
+        reviews = json.loads(reviews_json)
+        if not isinstance(reviews, list):
+            return json.dumps({"error": "Reviews must be a list of strings"}, indent=2)
+        
+        # Check rate limit
+        if not rate_limiter.allow_request():
+            return json.dumps({"error": "Rate limit exceeded. Please try again later."}, indent=2)
+        
+        # Add to queue and run
+        request_id = f"ui_{datetime.now().timestamp()}"
+        result = asyncio.run(queue_manager.add_task(
+            full_pipeline(title, abstract, reviews, request_id)
+        ))
+        
+        return json.dumps(result, indent=2)
+        
+    except json.JSONDecodeError:
+        return json.dumps({"error": "Invalid JSON format for reviews"}, indent=2)
+    except Exception as e:
+        return json.dumps({"error": str(e)}, indent=2)
+
+def run_critique_extraction_ui(reviews_json: str) -> str:
+    """UI wrapper for critique extraction"""
+    try:
+        reviews = json.loads(reviews_json)
+        if not rate_limiter.allow_request():
+            return json.dumps({"error": "Rate limit exceeded"}, indent=2)
+        
+        result = asyncio.run(extract_critiques(reviews))
+        return json.dumps(result, indent=2)
+    except Exception as e:
+        return json.dumps({"error": str(e)}, indent=2)
+
+def run_disagreement_detection_ui(critiques_json: str) -> str:
+    """UI wrapper for disagreement detection"""
+    try:
+        critiques = json.loads(critiques_json)
+        if not rate_limiter.allow_request():
+            return json.dumps({"error": "Rate limit exceeded"}, indent=2)
+        
+        result = asyncio.run(detect_disagreements(critiques))
+        return json.dumps(result, indent=2)
+    except Exception as e:
+        return json.dumps({"error": str(e)}, indent=2)
+
+def run_search_retrieval_ui(title: str, abstract: str, critiques_json: str) -> str:
+    """UI wrapper for search retrieval"""
+    try:
+        critiques = json.loads(critiques_json)
+        if not rate_limiter.allow_request():
+            return json.dumps({"error": "Rate limit exceeded"}, indent=2)
+        
+        result = asyncio.run(search_and_retrieve(title, abstract, critiques))
+        return json.dumps(result, indent=2)
+    except Exception as e:
+        return json.dumps({"error": str(e)}, indent=2)
+
+def check_progress_ui(request_id: str) -> str:
+    """Check progress of a request"""
+    if request_id in progress_store:
+        return json.dumps(progress_store[request_id], indent=2)
+    return json.dumps({"error": "Request ID not found"}, indent=2)
+
+# Build Gradio Interface
+with gr.Blocks(title="Automated Consensus Analysis API", theme=gr.themes.Soft()) as demo:
+    gr.Markdown("""
+    # 🔬 Automated Consensus Analysis API
+    
+    This API provides automated peer review consensus analysis using LLMs and search-augmented verification.
+    
+    ## Features:
+    - **Critique Extraction**: Extract structured critique points from reviews
+    - **Disagreement Detection**: Identify conflicts between reviewers
+    - **Search Retrieval**: Find supporting/contradicting evidence
+    - **Resolution**: Resolve disagreements with evidence
+    - **Meta-Review**: Generate comprehensive meta-reviews
+    """)
+    
+    with gr.Tabs():
+        # Full Pipeline Tab
+        with gr.Tab("📋 Full Pipeline"):
+            gr.Markdown("### Run the complete analysis pipeline")
+            with gr.Row():
+                with gr.Column():
+                    full_title = gr.Textbox(label="Paper Title", placeholder="Enter paper title...")
+                    full_abstract = gr.Textbox(label="Paper Abstract", lines=5, placeholder="Enter paper abstract...")
+                    full_reviews = gr.Code(
+                        label="Reviews (JSON Array)",
+                        language="json",
+                        value='["Review 1 text...", "Review 2 text..."]'
+                    )
+                    full_submit = gr.Button("🚀 Run Full Pipeline", variant="primary")
+                with gr.Column():
+                    full_output = gr.Code(label="Results", language="json")
+            
+            full_submit.click(
+                fn=run_full_pipeline_ui,
+                inputs=[full_title, full_abstract, full_reviews],
+                outputs=full_output
+            )
+        
+        # Individual Stages
+        with gr.Tab("🔍 Critique Extraction"):
+            gr.Markdown("### Extract critique points from reviews")
+            critique_reviews = gr.Code(
+                label="Reviews (JSON Array)",
+                language="json",
+                value='["Review 1...", "Review 2..."]'
+            )
+            critique_submit = gr.Button("Extract Critiques")
+            critique_output = gr.Code(label="Extracted Critiques", language="json")
+            
+            critique_submit.click(
+                fn=run_critique_extraction_ui,
+                inputs=critique_reviews,
+                outputs=critique_output
+            )
+        
+        with gr.Tab("⚡ Disagreement Detection"):
+            gr.Markdown("### Detect disagreements between reviews")
+            disagree_critiques = gr.Code(
+                label="Critique Points (JSON)",
+                language="json"
+            )
+            disagree_submit = gr.Button("Detect Disagreements")
+            disagree_output = gr.Code(label="Disagreement Analysis", language="json")
+            
+            disagree_submit.click(
+                fn=run_disagreement_detection_ui,
+                inputs=disagree_critiques,
+                outputs=disagree_output
+            )
+        
+        with gr.Tab("🔎 Search & Retrieval"):
+            gr.Markdown("### Search for supporting evidence")
+            with gr.Row():
+                with gr.Column():
+                    search_title = gr.Textbox(label="Paper Title")
+                    search_abstract = gr.Textbox(label="Paper Abstract", lines=3)
+                    search_critiques = gr.Code(label="Critiques (JSON)", language="json")
+                    search_submit = gr.Button("Search Evidence")
+                with gr.Column():
+                    search_output = gr.Code(label="Search Results", language="json")
+            
+            search_submit.click(
+                fn=run_search_retrieval_ui,
+                inputs=[search_title, search_abstract, search_critiques],
+                outputs=search_output
+            )
+        
+        with gr.Tab("📊 Progress Tracking"):
+            gr.Markdown("### Check pipeline progress")
+            progress_id = gr.Textbox(label="Request ID", placeholder="Enter request ID...")
+            progress_check = gr.Button("Check Progress")
+            progress_output = gr.Code(label="Progress Status", language="json")
+            
+            progress_check.click(
+                fn=check_progress_ui,
+                inputs=progress_id,
+                outputs=progress_output
+            )
+        
+        with gr.Tab("📖 API Documentation"):
+            gr.Markdown("""
+            ## API Endpoints
+            
+            ### POST /api/full_pipeline
+            Run the complete consensus analysis pipeline.
+            
+            **Request Body:**
+            ```json
+            {
+                "paper_title": "string",
+                "paper_abstract": "string",
+                "reviews": ["review1", "review2", ...]
+            }
+            ```
+            
+            ### POST /api/critique_extraction
+            Extract critique points from reviews.
+            
+            **Request Body:**
+            ```json
+            {
+                "reviews": ["review1", "review2", ...]
+            }
+            ```
+            
+            ### POST /api/disagreement_detection
+            Detect disagreements in critique points.
+            
+            **Request Body:**
+            ```json
+            {
+                "critiques": [{"Methodology": [...], ...}, ...]
+            }
+            ```
+            
+            ### POST /api/search_retrieval
+            Search for supporting evidence.
+            
+            **Request Body:**
+            ```json
+            {
+                "paper_title": "string",
+                "paper_abstract": "string",
+                "critiques": [...]
+            }
+            ```
+            
+            ### GET /api/progress/{request_id}
+            Check progress of a pipeline execution.
+            
+            ## Rate Limits
+            - 10 requests per minute per IP
+            - Maximum 3 concurrent pipeline executions
+            
+            ## Authentication
+            API keys are managed through HuggingFace Spaces secrets.
+            """)
+
+# Launch the app
+if __name__ == "__main__":
+    demo.queue(max_size=20)  # Enable queuing
+    demo.launch(
+        server_name="0.0.0.0",
+        server_port=7860,
+        share=False
+    )

config.py

@@ -0,0 +1,78 @@
+import os
+from pathlib import Path
+
+# Base directory
+BASE_DIR = Path(__file__).parent
+
+# API Configuration
+API_TITLE = "Automated Consensus Analysis API"
+API_VERSION = "1.0.0"
+API_DESCRIPTION = """
+## Automated Consensus Analysis for Peer Reviews
+
+This API provides comprehensive analysis of peer review disagreements using:
+- **LLM-based critique extraction** (Gemini 2.0)
+- **Disagreement detection** between reviewers
+- **Search-augmented evidence retrieval** (Semantic Scholar, arXiv, Google Scholar, Tavily)
+- **AI-powered disagreement resolution** (DeepSeek-R1)
+- **Meta-review generation**
+
+### Features:
+- ✅ Full pipeline or individual stage execution
+- ✅ Rate limiting and queue management
+- ✅ Progress tracking
+- ✅ JSON and form data support
+"""
+
+# Rate Limiting
+MAX_REQUESTS_PER_MINUTE = int(os.getenv("MAX_REQUESTS_PER_MINUTE", "10"))
+MAX_CONCURRENT_TASKS = int(os.getenv("MAX_CONCURRENT_TASKS", "3"))
+QUEUE_MAX_SIZE = int(os.getenv("QUEUE_MAX_SIZE", "20"))
+
+# Model Configuration
+GEMINI_MODEL = os.getenv("GEMINI_MODEL", "gemini-2.0-flash")
+DEEPSEEK_MODEL = os.getenv("DEEPSEEK_MODEL", "deepseek/deepseek-r1")
+
+# API Keys (from HF Spaces secrets)
+GEMINI_API_KEY = os.getenv("GEMINI_API_KEY")
+OPENROUTER_API_KEY = os.getenv("OPENROUTER_API_KEY")
+TAVILY_API_KEY = os.getenv("TAVILY_API_KEY")
+SERPAPI_API_KEY = os.getenv("SERPAPI_API_KEY")
+
+# Retry Configuration
+MAX_RETRIES = int(os.getenv("MAX_RETRIES", "5"))
+BASE_RETRY_WAIT = int(os.getenv("BASE_RETRY_WAIT", "2"))
+
+# Timeout Configuration
+REQUEST_TIMEOUT = int(os.getenv("REQUEST_TIMEOUT", "300"))  # 5 minutes
+SEARCH_TIMEOUT = int(os.getenv("SEARCH_TIMEOUT", "60"))  # 1 minute
+
+# Logging
+LOG_LEVEL = os.getenv("LOG_LEVEL", "INFO")
+
+def validate_environment():
+    """
+    Validate that all required environment variables are set
+    
+    Raises:
+        ValueError: If required variables are missing
+    """
+    required_vars = {
+        "GEMINI_API_KEY": GEMINI_API_KEY,
+        "OPENROUTER_API_KEY": OPENROUTER_API_KEY,
+        "TAVILY_API_KEY": TAVILY_API_KEY,
+    }
+    
+    missing = [var for var, value in required_vars.items() if not value]
+    
+    if missing:
+        raise ValueError(
+            f"Missing required environment variables: {', '.join(missing)}\n"
+            f"Please set them in HuggingFace Spaces secrets."
+        )
+
+# Validate on import
+try:
+    validate_environment()
+except ValueError as e:
+    print(f"⚠️  Configuration Warning: {e}")

pipeline/__init__.py

@@ -0,0 +1,17 @@
+"""
+Pipeline modules for automated consensus analysis
+"""
+
+from .critique_extraction import extract_critiques
+from .disagreement_detection import detect_disagreements
+from .search_retrieval import search_and_retrieve
+from .disagreement_resolution import resolve_disagreements
+from .meta_review import generate_meta_review
+
+__all__ = [
+    'extract_critiques',
+    'detect_disagreements',
+    'search_and_retrieve',
+    'resolve_disagreements',
+    'generate_meta_review',
+]

pipeline/critique_extraction.py

@@ -0,0 +1,145 @@
+import json
+import os
+from typing import List, Dict
+import google.generativeai as genai
+from pydantic import BaseModel
+import asyncio
+import time
+
+from dotenv import load_dotenv
+load_dotenv()
+
+# Configure Gemini
+genai.configure(api_key=os.getenv("GEMINI_API_KEY"))
+
+class CritiquePoint(BaseModel):
+    Methodology: List[str] = []
+    Experiments: List[str] = []
+    Clarity: List[str] = []
+    Significance: List[str] = []
+    Novelty: List[str] = []
+
+async def extract_single_critique(review_text: str, retries: int = 5) -> Dict:
+    """
+    Extract critique points from a single review using Gemini
+    
+    Args:
+        review_text: The review text to analyze
+        retries: Maximum number of retries
+    
+    Returns:
+        Dictionary with categorized critique points
+    """
+    prompt = f"""
+    Extract key critique points from the following research paper review.
+    Categorize them into aspects: Methodology, Experiments, Clarity, Significance, Novelty.
+    Return a structured JSON with these categories as keys and lists of critique points as values.
+
+    Review:
+    {review_text}
+    
+    Respond with ONLY valid JSON in this format:
+    {{
+        "Methodology": ["point1", "point2"],
+        "Experiments": ["point1"],
+        "Clarity": ["point1", "point2"],
+        "Significance": ["point1"],
+        "Novelty": ["point1"]
+    }}
+    """
+    
+    model = genai.GenerativeModel(
+        model_name="gemini-2.0-flash",
+        generation_config={
+            "response_mime_type": "application/json",
+        }
+    )
+    
+    for attempt in range(retries):
+        try:
+            response = await asyncio.to_thread(
+                model.generate_content,
+                prompt
+            )
+            
+            if not response.text.strip():
+                raise ValueError("Empty response from Gemini")
+            
+            result = json.loads(response.text)
+            
+            # Validate structure
+            critique = CritiquePoint(**result)
+            return critique.model_dump()
+            
+        except genai.types.generation_types.BlockedPromptException as e:
+            print(f"Content blocked by safety filters: {e}")
+            return {
+                "Methodology": [],
+                "Experiments": [],
+                "Clarity": [],
+                "Significance": [],
+                "Novelty": [],
+                "error": "Content blocked by safety filters"
+            }
+            
+        except Exception as e:
+            wait_time = 2 ** attempt
+            print(f"Attempt {attempt + 1} failed: {e}. Retrying in {wait_time}s...")
+            
+            if attempt < retries - 1:
+                await asyncio.sleep(wait_time)
+            else:
+                return {
+                    "Methodology": [],
+                    "Experiments": [],
+                    "Clarity": [],
+                    "Significance": [],
+                    "Novelty": [],
+                    "error": str(e)
+                }
+
+async def extract_critiques(reviews: List[str]) -> List[Dict]:
+    """
+    Extract critique points from multiple reviews
+    
+    Args:
+        reviews: List of review texts
+    
+    Returns:
+        List of dictionaries with categorized critique points
+    """
+    if not reviews:
+        return []
+    
+    # Filter valid reviews (must be strings with substantial content)
+    valid_reviews = [r for r in reviews if isinstance(r, str) and len(r.strip()) > 100]
+    
+    if not valid_reviews:
+        return []
+    
+    # Process reviews concurrently with rate limiting
+    tasks = []
+    for review in valid_reviews:
+        tasks.append(extract_single_critique(review))
+        # Small delay to avoid overwhelming the API
+        await asyncio.sleep(0.5)
+    
+    results = await asyncio.gather(*tasks, return_exceptions=True)
+    
+    # Filter out exceptions and return valid results
+    critiques = []
+    for i, result in enumerate(results):
+        if isinstance(result, Exception):
+            print(f"Review {i} failed: {result}")
+            critiques.append({
+                "Methodology": [],
+                "Experiments": [],
+                "Clarity": [],
+                "Significance": [],
+                "Novelty": [],
+                "error": str(result)
+            })
+        else:
+            critiques.append(result)
+    
+    return critiques

pipeline/disagreement_detection.py

@@ -0,0 +1,174 @@
+import json
+import os
+from typing import List, Dict
+from itertools import combinations
+import google.generativeai as genai
+from pydantic import BaseModel, Field
+import asyncio
+
+from dotenv import load_dotenv
+load_dotenv()
+
+# Configure Gemini
+genai.configure(api_key=os.getenv("GEMINI_API_KEY"))
+
+class DisagreementDetails(BaseModel):
+    Methodology: List[str] = Field(default_factory=list)
+    Experiments: List[str] = Field(default_factory=list)
+    Clarity: List[str] = Field(default_factory=list)
+    Significance: List[str] = Field(default_factory=list)
+    Novelty: List[str] = Field(default_factory=list)
+
+class DisagreementResult(BaseModel):
+    review_pair: List[int]
+    disagreement_score: float = Field(..., ge=0.0, le=1.0)
+    disagreement_details: DisagreementDetails
+
+def list_to_string(lst: List[str]) -> str:
+    """Convert list to formatted string"""
+    return "\n".join(f"- {item}" for item in lst) if lst else "None"
+
+async def compare_review_pair(
+    review1: Dict,
+    review2: Dict,
+    idx1: int,
+    idx2: int,
+    retries: int = 5
+) -> Dict:
+    """
+    Compare two reviews and detect disagreements
+    
+    Args:
+        review1: First review's critique points
+        review2: Second review's critique points
+        idx1: Index of first review
+        idx2: Index of second review
+        retries: Maximum retry attempts
+    
+    Returns:
+        Disagreement analysis results
+    """
+    prompt = f"""
+    Compare the following two reviews and identify disagreements across different aspects.
+    Assess disagreement level (0.0 = perfect agreement, 1.0 = complete disagreement) and 
+    list specific points of disagreement for each category.
+    
+    Review 1:
+    Methodology: {list_to_string(review1.get('Methodology', []))}
+    Experiments: {list_to_string(review1.get('Experiments', []))}
+    Clarity: {list_to_string(review1.get('Clarity', []))}
+    Significance: {list_to_string(review1.get('Significance', []))}
+    Novelty: {list_to_string(review1.get('Novelty', []))}
+
+    Review 2:
+    Methodology: {list_to_string(review2.get('Methodology', []))}
+    Experiments: {list_to_string(review2.get('Experiments', []))}
+    Clarity: {list_to_string(review2.get('Clarity', []))}
+    Significance: {list_to_string(review2.get('Significance', []))}
+    Novelty: {list_to_string(review2.get('Novelty', []))}
+    
+    Respond with ONLY valid JSON in this exact format:
+    {{
+        "disagreement_score": 0.5,
+        "disagreement_details": {{
+            "Methodology": ["specific disagreement point 1"],
+            "Experiments": ["specific disagreement point 1"],
+            "Clarity": [],
+            "Significance": ["specific disagreement point 1"],
+            "Novelty": []
+        }}
+    }}
+    """
+    
+    model = genai.GenerativeModel(
+        model_name="gemini-2.0-flash",
+        generation_config={
+            "response_mime_type": "application/json",
+        }
+    )
+    
+    for attempt in range(retries):
+        try:
+            response = await asyncio.to_thread(
+                model.generate_content,
+                prompt
+            )
+            
+            if not response.text.strip():
+                raise ValueError("Empty response from Gemini")
+            
+            result = json.loads(response.text)
+            
+            # Validate structure
+            disagreement = DisagreementResult(
+                review_pair=[idx1, idx2],
+                disagreement_score=result["disagreement_score"],
+                disagreement_details=result["disagreement_details"]
+            )
+            
+            return disagreement.model_dump()
+            
+        except Exception as e:
+            wait_time = 2 ** attempt
+            print(f"Disagreement detection attempt {attempt + 1} failed: {e}")
+            
+            if attempt < retries - 1:
+                await asyncio.sleep(wait_time)
+            else:
+                return {
+                    "review_pair": [idx1, idx2],
+                    "disagreement_score": 0.0,
+                    "disagreement_details": {
+                        "Methodology": [],
+                        "Experiments": [],
+                        "Clarity": [],
+                        "Significance": [],
+                        "Novelty": []
+                    },
+                    "error": str(e)
+                }
+
+async def detect_disagreements(critique_points: List[Dict]) -> List[Dict]:
+    """
+    Detect disagreements across all review pairs
+    
+    Args:
+        critique_points: List of critique point dictionaries
+    
+    Returns:
+        List of disagreement analyses
+    """
+    if len(critique_points) < 2:
+        return []
+    
+    # Generate all review pairs
+    review_pairs = list(combinations(range(len(critique_points)), 2))
+    
+    if not review_pairs:
+        return []
+    
+    # Process pairs concurrently with rate limiting
+    tasks = []
+    for idx1, idx2 in review_pairs:
+        tasks.append(
+            compare_review_pair(
+                critique_points[idx1],
+                critique_points[idx2],
+                idx1,
+                idx2
+            )
+        )
+        # Small delay between API calls
+        await asyncio.sleep(0.3)
+    
+    results = await asyncio.gather(*tasks, return_exceptions=True)
+    
+    # Filter results
+    disagreements = []
+    for i, result in enumerate(results):
+        if isinstance(result, Exception):
+            print(f"Review pair {review_pairs[i]} failed: {result}")
+        else:
+            disagreements.append(result)
+    
+    return disagreements

pipeline/disagreement_resolution.py

@@ -0,0 +1,242 @@
+import json
+import os
+from typing import List, Dict
+from openai import OpenAI
+from pydantic import BaseModel
+import asyncio
+
+from dotenv import load_dotenv
+load_dotenv()
+
+# Initialize OpenRouter client
+client = OpenAI(
+    base_url="https://openrouter.ai/api/v1",
+    api_key=os.getenv("OPENROUTER_API_KEY"),
+)
+
+class ResolutionDetails(BaseModel):
+    accepted_critique_points: Dict[str, List[str]]
+    rejected_critique_points: Dict[str, List[str]]
+    final_resolution_summary: str
+
+class DisagreementResolutionResult(BaseModel):
+    review_pair: List[int]
+    resolution_details: ResolutionDetails
+
+def construct_resolution_prompt(
+    paper_title: str,
+    paper_abstract: str,
+    disagreement: Dict,
+    combined_critiques: Dict,
+    sota_results: str,
+    retrieved_evidence: Dict
+) -> tuple:
+    """
+    Construct prompt for disagreement resolution
+    
+    Args:
+        paper_title: Title of the paper
+        paper_abstract: Abstract of the paper
+        disagreement: Disagreement analysis results
+        combined_critiques: Combined critique points
+        sota_results: State-of-the-art findings
+        retrieved_evidence: Retrieved evidence per category
+    
+    Returns:
+        Tuple of (system_prompt, user_prompt)
+    """
+    system_prompt = """
+    You are an AI specialized in resolving academic peer review disagreements. 
+    Your task is to analyze critiques, verify evidence, and provide a structured resolution.
+    
+    Respond in the following JSON format:
+    {
+      "accepted_critique_points": {"category": ["critique_1", "critique_2"]},
+      "rejected_critique_points": {"category": ["critique_3"]},
+      "final_resolution_summary": "After analyzing critiques and evidence, we conclude that..."
+    }
+    """
+    
+    disagreement_details = disagreement.get('disagreement_details', {})
+    disagreement_score = disagreement.get('disagreement_score', 0.0)
+    
+    user_prompt = f"""
+    ### **Paper Details**
+    **Title:** {paper_title}  
+    **Abstract:** {paper_abstract}  
+
+    ### **Reviewer Disagreement (Score: {disagreement_score})**
+    - **Methodology:** {', '.join(disagreement_details.get('Methodology', ['N/A']))}
+    - **Experiments:** {', '.join(disagreement_details.get('Experiments', ['N/A']))}
+    - **Clarity:** {', '.join(disagreement_details.get('Clarity', ['N/A']))}
+    - **Significance:** {', '.join(disagreement_details.get('Significance', ['N/A']))}
+    - **Novelty:** {', '.join(disagreement_details.get('Novelty', ['N/A']))}
+
+    ### **Supporting Information**
+    **Combined Critique Points from Reviews:**  
+    {json.dumps(combined_critiques, indent=2)}
+
+    **State-of-the-Art (SoTA) Findings:**  
+    {sota_results[:2000]}
+
+    **Retrieved Evidence:**  
+    {json.dumps(retrieved_evidence, indent=2)[:2000]}
+
+    ### **Resolution Task**
+    1. Validate critique points and categorize them into accepted or rejected.
+    2. Compare with SoTA research and retrieved evidence.
+    3. Provide a final resolution summary explaining whether the disagreement is justified.
+    
+    Respond with ONLY valid JSON.
+    """
+    
+    return system_prompt, user_prompt
+
+async def resolve_single_disagreement(
+    paper_title: str,
+    paper_abstract: str,
+    disagreement: Dict,
+    combined_critiques: Dict,
+    sota_results: str,
+    retrieved_evidence: Dict,
+    retries: int = 5
+) -> Dict:
+    """
+    Resolve a single disagreement using DeepSeek-R1
+    
+    Args:
+        paper_title: Paper title
+        paper_abstract: Paper abstract
+        disagreement: Disagreement analysis
+        combined_critiques: Combined critique points
+        sota_results: SoTA findings
+        retrieved_evidence: Evidence per category
+        retries: Maximum retry attempts
+    
+    Returns:
+        Resolution results
+    """
+    system_prompt, user_prompt = construct_resolution_prompt(
+        paper_title,
+        paper_abstract,
+        disagreement,
+        combined_critiques,
+        sota_results,
+        retrieved_evidence
+    )
+    
+    messages = [
+        {"role": "system", "content": system_prompt},
+        {"role": "user", "content": user_prompt},
+    ]
+    
+    for attempt in range(retries):
+        try:
+            response = await asyncio.to_thread(
+                client.chat.completions.create,
+                model="deepseek/deepseek-r1",
+                messages=messages,
+                response_format={"type": "json_object"},
+            )
+            
+            if not response.choices or not response.choices[0].message.content.strip():
+                raise ValueError("Empty response from DeepSeek-R1")
+            
+            # Parse response (remove potential prefix)
+            content = response.choices[0].message.content.strip()
+            if content.startswith("```json"):
+                content = content[7:-3].strip()
+            elif content.startswith("```"):
+                content = content[3:-3].strip()
+            
+            llm_output = json.loads(content)
+            
+            # Validate required keys
+            required_keys = {
+                "accepted_critique_points",
+                "rejected_critique_points",
+                "final_resolution_summary"
+            }
+            
+            if not required_keys.issubset(llm_output.keys()):
+                raise ValueError(f"Missing keys. Present: {llm_output.keys()}")
+            
+            # Validate structure
+            resolution = DisagreementResolutionResult(
+                review_pair=disagreement.get('review_pair', [0, 1]),
+                resolution_details=ResolutionDetails(**llm_output)
+            )
+            
+            return resolution.model_dump()
+            
+        except Exception as e:
+            wait_time = 2 ** attempt
+            print(f"Resolution attempt {attempt + 1} failed: {e}")
+            
+            if attempt < retries - 1:
+                await asyncio.sleep(wait_time)
+            else:
+                return {
+                    "review_pair": disagreement.get('review_pair', [0, 1]),
+                    "resolution_details": {
+                        "accepted_critique_points": {},
+                        "rejected_critique_points": {},
+                        "final_resolution_summary": f"Error: {str(e)}"
+                    },
+                    "error": str(e)
+                }
+
+async def resolve_disagreements(
+    paper_title: str,
+    paper_abstract: str,
+    disagreements: List[Dict],
+    critique_points: List[Dict],
+    search_results: Dict
+) -> List[Dict]:
+    """
+    Resolve all disagreements
+    
+    Args:
+        paper_title: Paper title
+        paper_abstract: Paper abstract
+        disagreements: List of disagreement analyses
+        critique_points: List of critique points
+        search_results: Search and retrieval results
+    
+    Returns:
+        List of resolution results
+    """
+    if not disagreements:
+        return []
+    
+    combined_critiques = search_results.get('Combined_Critiques', {})
+    sota_results = search_results.get('SoTA_Results', '')
+    retrieved_evidence = search_results.get('Retrieved_Evidence', {})
+    
+    # Process disagreements with rate limiting
+    tasks = []
+    for disagreement in disagreements:
+        tasks.append(
+            resolve_single_disagreement(
+                paper_title,
+                paper_abstract,
+                disagreement,
+                combined_critiques,
+                sota_results,
+                retrieved_evidence
+            )
+        )
+        # Delay between API calls
+        await asyncio.sleep(1)
+    
+    results = await asyncio.gather(*tasks, return_exceptions=True)
+    
+    # Filter results
+    resolutions = []
+    for i, result in enumerate(results):
+        if isinstance(result, Exception):
+            print(f"Resolution {i} failed: {result}")
+        else:
+            resolutions.append(result)
+    
+    return resolutions

pipeline/meta_review.py

@@ -0,0 +1,170 @@
+import json
+import os
+from typing import List, Dict
+from openai import OpenAI
+from pydantic import BaseModel
+import asyncio
+
+from dotenv import load_dotenv
+load_dotenv()
+
+# Initialize OpenRouter client
+client = OpenAI(
+    base_url="https://openrouter.ai/api/v1",
+    api_key=os.getenv("OPENROUTER_API_KEY"),
+)
+
+class MetaReviewResult(BaseModel):
+    meta_review: str
+
+def construct_meta_review_prompt(
+    paper_title: str,
+    paper_abstract: str,
+    resolutions: List[Dict],
+    search_results: Dict
+) -> tuple:
+    """
+    Construct prompt for meta-review generation
+    
+    Args:
+        paper_title: Paper title
+        paper_abstract: Paper abstract
+        resolutions: List of disagreement resolutions
+        search_results: Search and retrieval results
+    
+    Returns:
+        Tuple of (system_prompt, user_prompt)
+    """
+    # Aggregate all resolutions
+    all_accepted = {}
+    all_rejected = {}
+    resolution_summaries = []
+    
+    for resolution in resolutions:
+        details = resolution.get('resolution_details', {})
+        
+        # Merge accepted points
+        accepted = details.get('accepted_critique_points', {})
+        for category, points in accepted.items():
+            if category not in all_accepted:
+                all_accepted[category] = []
+            all_accepted[category].extend(points)
+        
+        # Merge rejected points
+        rejected = details.get('rejected_critique_points', {})
+        for category, points in rejected.items():
+            if category not in all_rejected:
+                all_rejected[category] = []
+            all_rejected[category].extend(points)
+        
+        # Collect summaries
+        summary = details.get('final_resolution_summary', '')
+        if summary:
+            resolution_summaries.append(summary)
+    
+    system_prompt = """
+    You are an expert meta-reviewer. Your task is to generate a structured, comprehensive 
+    meta-review based on reviewer critiques, disagreements, and resolutions.
+    Your review should be clear, concise, well-structured, and provide actionable feedback.
+    
+    Respond with ONLY the meta-review text (no JSON, no preamble).
+    """
+    
+    user_prompt = f"""
+    ### **Paper Details**
+    **Title:** {paper_title}  
+    **Abstract:** {paper_abstract}  
+
+    ### **Disagreement Resolution Summaries**
+    {chr(10).join(f"- {summary}" for summary in resolution_summaries)}
+
+    ### **Accepted Critique Points (Valid Feedback)**
+    {json.dumps(all_accepted, indent=2)}
+
+    ### **Rejected Critique Points (Unjustified Criticism)**
+    {json.dumps(all_rejected, indent=2)}
+
+    ### **State-of-the-Art (SoTA) Findings**
+    {search_results.get('SoTA_Results', '')[:2000]}
+
+    ### **Retrieved Evidence for Validation**
+    {json.dumps(search_results.get('Retrieved_Evidence', {}), indent=2)[:2000]}
+
+    ### **Meta-Review Task**
+    Generate a comprehensive meta-review that:
+    1. Summarizes the paper's main contribution and approach
+    2. Discusses the strengths of the paper (based on accepted critiques and evidence)
+    3. Discusses the weaknesses and concerns (based on valid accepted critiques)
+    4. Addresses key disagreements among reviewers and how they were resolved
+    5. Compares the paper's claims with state-of-the-art research
+    6. Provides a final verdict on the paper's quality, novelty, significance, and clarity
+    7. Offers constructive recommendations for improvement
+    
+    Format the meta-review professionally with clear sections.
+    """
+    
+    return system_prompt, user_prompt
+
+async def generate_meta_review(
+    paper_title: str,
+    paper_abstract: str,
+    resolutions: List[Dict],
+    search_results: Dict,
+    retries: int = 5
+) -> str:
+    """
+    Generate a meta-review using DeepSeek-R1
+    
+    Args:
+        paper_title: Paper title
+        paper_abstract: Paper abstract
+        resolutions: List of disagreement resolutions
+        search_results: Search and retrieval results
+        retries: Maximum retry attempts
+    
+    Returns:
+        Generated meta-review text
+    """
+    if not resolutions:
+        return "Unable to generate meta-review: No disagreement resolutions available."
+    
+    system_prompt, user_prompt = construct_meta_review_prompt(
+        paper_title,
+        paper_abstract,
+        resolutions,
+        search_results
+    )
+    
+    messages = [
+        {"role": "system", "content": system_prompt},
+        {"role": "user", "content": user_prompt},
+    ]
+    
+    for attempt in range(retries):
+        try:
+            response = await asyncio.to_thread(
+                client.chat.completions.create,
+                model="deepseek/deepseek-r1",
+                messages=messages,
+            )
+            
+            if not response.choices or not response.choices[0].message.content.strip():
+                raise ValueError("Empty response from DeepSeek-R1")
+            
+            meta_review_text = response.choices[0].message.content.strip()
+            
+            # Remove any JSON formatting if present
+            if meta_review_text.startswith("```"):
+                lines = meta_review_text.split("\n")
+                meta_review_text = "\n".join(lines[1:-1])
+            
+            return meta_review_text
+            
+        except Exception as e:
+            wait_time = 2 ** attempt
+            print(f"Meta-review generation attempt {attempt + 1} failed: {e}")
+            
+            if attempt < retries - 1:
+                await asyncio.sleep(wait_time)
+            else:
+                return f"Error generating meta-review: {str(e)}"

pipeline/search_retrieval.py

@@ -0,0 +1,224 @@
+import os
+from typing import Dict, List
+import asyncio
+from langchain_google_genai import ChatGoogleGenerativeAI
+from langchain_community.utilities import ArxivAPIWrapper, SerpAPIWrapper
+from langchain_community.tools.semanticscholar.tool import SemanticScholarQueryRun
+from langchain_community.tools.tavily_search import TavilySearchResults
+from langchain.agents import AgentType, initialize_agent, AgentExecutor
+from langchain.tools import Tool
+
+from dotenv import load_dotenv
+load_dotenv()
+
+# Initialize LLM
+llm = ChatGoogleGenerativeAI(
+    model=os.getenv("GEMINI_MODEL"),
+    google_api_key=os.getenv("GEMINI_API_KEY"),
+    max_retries=2,
+)
+
+# Initialize search tools
+semantic_scholar = SemanticScholarQueryRun()
+google_scholar = SerpAPIWrapper(params={"engine": "google_scholar"})
+arxiv_search = ArxivAPIWrapper()
+tavily_search = TavilySearchResults(max_results=5)
+
+# Define tools
+tools = [
+    Tool(
+        name="TavilySearch",
+        func=tavily_search.run,
+        description="Retrieves the latest State-of-the-Art (SoTA) research and current academic information"
+    ),
+    Tool(
+        name="SemanticScholar",
+        func=semantic_scholar.run,
+        description="Find academic papers from Semantic Scholar database"
+    ),
+    Tool(
+        name="GoogleScholar",
+        func=google_scholar.run,
+        description="Search for scholarly articles and citations"
+    ),
+    Tool(
+        name="ArxivSearch",
+        func=arxiv_search.run,
+        description="Find research papers from ArXiv preprint repository"
+    ),
+]
+
+# Initialize agent
+agent = initialize_agent(
+    tools=tools,
+    llm=llm,
+    agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION,
+    verbose=False,
+    handle_parsing_errors=True,
+    max_iterations=10
+)
+
+def combine_critiques(critique_points: List[Dict]) -> Dict[str, str]:
+    """
+    Combine critique points from multiple reviews into categories
+    
+    Args:
+        critique_points: List of critique dictionaries
+    
+    Returns:
+        Dictionary with combined critiques per category
+    """
+    categories = ["Methodology", "Clarity", "Experiments", "Significance", "Novelty"]
+    combined = {cat: [] for cat in categories}
+    
+    for review in critique_points:
+        for category in categories:
+            if category in review and review[category]:
+                combined[category].extend(review[category])
+    
+    # Join into strings
+    for category in categories:
+        combined[category] = " | ".join(combined[category]) if combined[category] else "No critiques"
+    
+    return combined
+
+async def search_sota(paper_title: str, paper_abstract: str, retries: int = 3) -> str:
+    """
+    Search for state-of-the-art research related to the paper
+    
+    Args:
+        paper_title: Paper title
+        paper_abstract: Paper abstract
+        retries: Maximum retry attempts
+    
+    Returns:
+        Summary of SoTA findings
+    """
+    query = (
+        f"Find the latest state-of-the-art research related to: '{paper_title}'. "
+        f"Abstract: {paper_abstract[:500]}. "
+        f"Focus on recent advances, similar methodologies, and competing approaches."
+    )
+    
+    for attempt in range(retries):
+        try:
+            result = await asyncio.to_thread(agent.run, query)
+            
+            if not result or len(result.strip()) < 50:
+                raise ValueError("Empty or insufficient response")
+            
+            return result
+            
+        except Exception as e:
+            wait_time = 2 ** attempt
+            print(f"SoTA search attempt {attempt + 1} failed: {e}")
+            
+            if attempt < retries - 1:
+                await asyncio.sleep(wait_time)
+            else:
+                return f"Error retrieving SoTA research: {str(e)}"
+
+async def retrieve_evidence_for_category(
+    category: str,
+    critiques: str,
+    retries: int = 3
+) -> str:
+    """
+    Retrieve evidence for critiques in a specific category
+    
+    Args:
+        category: Category name (e.g., "Methodology")
+        critiques: Combined critique text
+        retries: Maximum retry attempts
+    
+    Returns:
+        Evidence findings
+    """
+    if critiques == "No critiques" or not critiques.strip():
+        return f"No critiques to validate for {category}"
+    
+    query = (
+        f"Find research papers that support or contradict these critiques "
+        f"related to {category}: {critiques[:500]}"
+    )
+    
+    for attempt in range(retries):
+        try:
+            result = await asyncio.to_thread(agent.run, query)
+            
+            if not result:
+                raise ValueError("Empty response")
+            
+            return result
+            
+        except Exception as e:
+            wait_time = 2 ** attempt
+            print(f"Evidence retrieval for {category} attempt {attempt + 1} failed: {e}")
+            
+            if attempt < retries - 1:
+                await asyncio.sleep(wait_time)
+            else:
+                return f"Error retrieving evidence for {category}: {str(e)}"
+
+async def retrieve_evidence(combined_critiques: Dict[str, str]) -> Dict[str, str]:
+    """
+    Retrieve evidence for all critique categories
+    
+    Args:
+        combined_critiques: Dictionary of combined critiques per category
+    
+    Returns:
+        Dictionary of evidence per category
+    """
+    evidence_results = {}
+    
+    # Process categories with rate limiting
+    for category, critiques in combined_critiques.items():
+        evidence_results[category] = await retrieve_evidence_for_category(
+            category,
+            critiques
+        )
+        # Delay between requests
+        await asyncio.sleep(1)
+    
+    return evidence_results
+
+async def search_and_retrieve(
+    paper_title: str,
+    paper_abstract: str,
+    critique_points: List[Dict]
+) -> Dict:
+    """
+    Complete search and retrieval pipeline
+    
+    Args:
+        paper_title: Paper title
+        paper_abstract: Paper abstract
+        critique_points: List of critique point dictionaries
+    
+    Returns:
+        Dictionary with SoTA results, combined critiques, and evidence
+    """
+    try:
+        # Step 1: Search for SoTA research
+        sota_results = await search_sota(paper_title, paper_abstract)
+        
+        # Step 2: Combine critique points
+        combined_critiques = combine_critiques(critique_points)
+        
+        # Step 3: Retrieve evidence for critiques
+        evidence = await retrieve_evidence(combined_critiques)
+        
+        return {
+            "SoTA_Results": sota_results,
+            "Combined_Critiques": combined_critiques,
+            "Retrieved_Evidence": evidence
+        }
+        
+    except Exception as e:
+        return {
+            "error": str(e),
+            "SoTA_Results": "",
+            "Combined_Critiques": {},
+            "Retrieved_Evidence": {}
+        }

requirements.txt

@@ -0,0 +1,38 @@
+# Web Framework
+gradio==5.9.1
+
+# LLM Libraries
+openai==1.59.5
+google-generativeai==0.8.3
+
+# LangChain and Tools
+langchain==0.3.13
+langchain-community==0.3.13
+langchain-google-genai==2.0.8
+langgraph==0.2.59
+langgraph-checkpoint-sqlite==2.0.5
+
+# Search APIs
+tavily-python==0.5.0
+semanticscholar==0.8.4
+arxiv==2.1.3
+google-search-results==2.4.2
+
+# Data Processing
+pandas==2.2.3
+pydantic==2.10.4
+python-dotenv==1.0.1
+
+# API & Async
+fastapi==0.115.6
+uvicorn==0.34.0
+aiohttp==3.11.11
+httpx==0.28.1
+
+# Utilities
+tqdm==4.67.1
+mlflow==2.19.0
+
+# Rate Limiting & Queue
+ratelimit==2.2.1
+asyncio-throttle==1.0.1

utils/__init__.py

@@ -0,0 +1,29 @@
+"""
+Utility modules for API functionality
+"""
+
+from .rate_limiter import RateLimiter
+from .queue_manager import QueueManager
+from .validators import (
+    validate_paper_input,
+    validate_critique_input,
+    validate_disagreement_input,
+    validate_search_input,
+    PaperInput,
+    CritiqueInput,
+    DisagreementInput,
+    SearchInput,
+)
+
+__all__ = [
+    'RateLimiter',
+    'QueueManager',
+    'validate_paper_input',
+    'validate_critique_input',
+    'validate_disagreement_input',
+    'validate_search_input',
+    'PaperInput',
+    'CritiqueInput',
+    'DisagreementInput',
+    'SearchInput',
+]

utils/queue_manager.py

@@ -0,0 +1,76 @@
+import asyncio
+from typing import Coroutine, Any
+from asyncio import Semaphore, Queue
+from datetime import datetime
+
+class QueueManager:
+    """
+    Async queue manager for handling concurrent pipeline executions
+    """
+    
+    def __init__(self, max_concurrent: int = 3):
+        """
+        Initialize queue manager
+        
+        Args:
+            max_concurrent: Maximum number of concurrent tasks
+        """
+        self.max_concurrent = max_concurrent
+        self.semaphore = Semaphore(max_concurrent)
+        self.queue: Queue = Queue()
+        self.active_tasks = 0
+        self.total_processed = 0
+    
+    async def add_task(self, coro: Coroutine) -> Any:
+        """
+        Add a task to the queue and execute it
+        
+        Args:
+            coro: Coroutine to execute
+        
+        Returns:
+            Result from the coroutine
+        """
+        async with self.semaphore:
+            self.active_tasks += 1
+            try:
+                result = await coro
+                self.total_processed += 1
+                return result
+            finally:
+                self.active_tasks -= 1
+    
+    def get_queue_status(self) -> dict:
+        """
+        Get current queue status
+        
+        Returns:
+            Dictionary with queue statistics
+        """
+        return {
+            "active_tasks": self.active_tasks,
+            "max_concurrent": self.max_concurrent,
+            "total_processed": self.total_processed,
+            "available_slots": self.max_concurrent - self.active_tasks,
+            "timestamp": datetime.now().isoformat()
+        }
+    
+    async def wait_for_slot(self, timeout: float = 60.0) -> bool:
+        """
+        Wait for an available slot in the queue
+        
+        Args:
+            timeout: Maximum time to wait in seconds
+        
+        Returns:
+            True if slot became available, False if timeout
+        """
+        start_time = asyncio.get_event_loop().time()
+        
+        while self.active_tasks >= self.max_concurrent:
+            if asyncio.get_event_loop().time() - start_time > timeout:
+                return False
+            
+            await asyncio.sleep(0.5)
+        
+        return True

utils/rate_limiter.py

@@ -0,0 +1,84 @@
+import time
+from collections import defaultdict, deque
+from threading import Lock
+from typing import Dict
+
+class RateLimiter:
+    """
+    Thread-safe rate limiter for API requests
+    """
+    
+    def __init__(self, max_requests_per_minute: int = 10):
+        """
+        Initialize rate limiter
+        
+        Args:
+            max_requests_per_minute: Maximum requests allowed per minute
+        """
+        self.max_requests = max_requests_per_minute
+        self.window_seconds = 60
+        self.requests: Dict[str, deque] = defaultdict(deque)
+        self.lock = Lock()
+    
+    def _clean_old_requests(self, client_id: str):
+        """Remove requests older than the time window"""
+        current_time = time.time()
+        cutoff_time = current_time - self.window_seconds
+        
+        while self.requests[client_id] and self.requests[client_id][0] < cutoff_time:
+            self.requests[client_id].popleft()
+    
+    def allow_request(self, client_id: str = "default") -> bool:
+        """
+        Check if a request is allowed
+        
+        Args:
+            client_id: Identifier for the client (e.g., IP address)
+        
+        Returns:
+            True if request is allowed, False otherwise
+        """
+        with self.lock:
+            self._clean_old_requests(client_id)
+            
+            if len(self.requests[client_id]) >= self.max_requests:
+                return False
+            
+            self.requests[client_id].append(time.time())
+            return True
+    
+    def get_remaining_requests(self, client_id: str = "default") -> int:
+        """
+        Get number of remaining requests in current window
+        
+        Args:
+            client_id: Identifier for the client
+        
+        Returns:
+            Number of remaining requests
+        """
+        with self.lock:
+            self._clean_old_requests(client_id)
+            return max(0, self.max_requests - len(self.requests[client_id]))
+    
+    def get_reset_time(self, client_id: str = "default") -> float:
+        """
+        Get time until rate limit resets
+        
+        Args:
+            client_id: Identifier for the client
+        
+        Returns:
+            Seconds until oldest request expires
+        """
+        with self.lock:
+            self._clean_old_requests(client_id)
+            
+            if not self.requests[client_id]:
+                return 0
+            
+            oldest_request = self.requests[client_id][0]
+            current_time = time.time()
+            reset_time = oldest_request + self.window_seconds
+            
+            return max(0, reset_time - current_time)

utils/validators.py

@@ -0,0 +1,196 @@
+from typing import List, Dict, Tuple
+from pydantic import BaseModel, Field, field_validator
+
+class PaperInput(BaseModel):
+    """Validated paper input schema"""
+    paper_title: str = Field(..., min_length=5, max_length=500)
+    paper_abstract: str = Field(..., min_length=50, max_length=5000)
+    reviews: List[str] = Field(..., min_length=1, max_length=10)
+    
+    @field_validator('paper_title')
+    @classmethod
+    def validate_title(cls, v: str) -> str:
+        """Validate paper title"""
+        if not v or not v.strip():
+            raise ValueError("Paper title cannot be empty")
+        return v.strip()
+    
+    @field_validator('paper_abstract')
+    @classmethod
+    def validate_abstract(cls, v: str) -> str:
+        """Validate paper abstract"""
+        if not v or not v.strip():
+            raise ValueError("Paper abstract cannot be empty")
+        if len(v.strip()) < 50:
+            raise ValueError("Paper abstract must be at least 50 characters")
+        return v.strip()
+    
+    @field_validator('reviews')
+    @classmethod
+    def validate_reviews(cls, v: List[str]) -> List[str]:
+        """Validate reviews"""
+        if not v:
+            raise ValueError("At least one review is required")
+        
+        valid_reviews = []
+        for i, review in enumerate(v):
+            if not isinstance(review, str):
+                raise ValueError(f"Review {i} must be a string")
+            
+            cleaned = review.strip()
+            if len(cleaned) < 50:
+                raise ValueError(f"Review {i} must be at least 50 characters")
+            
+            valid_reviews.append(cleaned)
+        
+        return valid_reviews
+
+class CritiqueInput(BaseModel):
+    """Validated critique input schema"""
+    reviews: List[str] = Field(..., min_length=1, max_length=10)
+    
+    @field_validator('reviews')
+    @classmethod
+    def validate_reviews(cls, v: List[str]) -> List[str]:
+        """Validate reviews"""
+        if not v:
+            raise ValueError("At least one review is required")
+        
+        valid_reviews = []
+        for review in v:
+            if isinstance(review, str) and len(review.strip()) >= 50:
+                valid_reviews.append(review.strip())
+        
+        if not valid_reviews:
+            raise ValueError("No valid reviews found (must be at least 50 characters)")
+        
+        return valid_reviews
+
+class DisagreementInput(BaseModel):
+    """Validated disagreement detection input schema"""
+    critiques: List[Dict] = Field(..., min_length=2)
+    
+    @field_validator('critiques')
+    @classmethod
+    def validate_critiques(cls, v: List[Dict]) -> List[Dict]:
+        """Validate critique structure"""
+        if len(v) < 2:
+            raise ValueError("At least 2 critiques required for disagreement detection")
+        
+        required_keys = {'Methodology', 'Experiments', 'Clarity', 'Significance', 'Novelty'}
+        
+        for i, critique in enumerate(v):
+            if not isinstance(critique, dict):
+                raise ValueError(f"Critique {i} must be a dictionary")
+            
+            # Check if critique has the expected structure
+            if not any(key in critique for key in required_keys):
+                raise ValueError(f"Critique {i} missing required categories")
+        
+        return v
+
+class SearchInput(BaseModel):
+    """Validated search input schema"""
+    paper_title: str = Field(..., min_length=5, max_length=500)
+    paper_abstract: str = Field(..., min_length=50, max_length=5000)
+    critiques: List[Dict] = Field(..., min_length=1)
+    
+    @field_validator('paper_title')
+    @classmethod
+    def validate_title(cls, v: str) -> str:
+        """Validate paper title"""
+        if not v or not v.strip():
+            raise ValueError("Paper title cannot be empty")
+        return v.strip()
+    
+    @field_validator('paper_abstract')
+    @classmethod
+    def validate_abstract(cls, v: str) -> str:
+        """Validate paper abstract"""
+        if not v or not v.strip():
+            raise ValueError("Paper abstract cannot be empty")
+        return v.strip()
+
+def validate_paper_input(
+    paper_title: str,
+    paper_abstract: str,
+    reviews: List[str]
+) -> Tuple[bool, str]:
+    """
+    Validate paper input data
+    
+    Args:
+        paper_title: Paper title
+        paper_abstract: Paper abstract
+        reviews: List of review texts
+    
+    Returns:
+        Tuple of (is_valid, error_message)
+    """
+    try:
+        PaperInput(
+            paper_title=paper_title,
+            paper_abstract=paper_abstract,
+            reviews=reviews
+        )
+        return True, ""
+    except Exception as e:
+        return False, str(e)
+
+def validate_critique_input(reviews: List[str]) -> Tuple[bool, str]:
+    """
+    Validate critique extraction input
+    
+    Args:
+        reviews: List of review texts
+    
+    Returns:
+        Tuple of (is_valid, error_message)
+    """
+    try:
+        CritiqueInput(reviews=reviews)
+        return True, ""
+    except Exception as e:
+        return False, str(e)
+
+def validate_disagreement_input(critiques: List[Dict]) -> Tuple[bool, str]:
+    """
+    Validate disagreement detection input
+    
+    Args:
+        critiques: List of critique dictionaries
+    
+    Returns:
+        Tuple of (is_valid, error_message)
+    """
+    try:
+        DisagreementInput(critiques=critiques)
+        return True, ""
+    except Exception as e:
+        return False, str(e)
+
+def validate_search_input(
+    paper_title: str,
+    paper_abstract: str,
+    critiques: List[Dict]
+) -> Tuple[bool, str]:
+    """
+    Validate search input
+    
+    Args:
+        paper_title: Paper title
+        paper_abstract: Paper abstract
+        critiques: List of critique dictionaries
+    
+    Returns:
+        Tuple of (is_valid, error_message)
+    """
+    try:
+        SearchInput(
+            paper_title=paper_title,
+            paper_abstract=paper_abstract,
+            critiques=critiques
+        )
+        return True, ""
+    except Exception as e:
+        return False, str(e)