Upload 7 files

BLOG.md

@@ -365,3 +365,4 @@ As enterprises generate ever more complex documents, the need for intelligent, a
 ### Q: What about languages other than English?
 
 **A:** Currently optimized for English, with beta support for Spanish, French, and German. Multi-language support is on our roadmap based on customer demand.
+

DEMO_README.md

@@ -246,3 +246,4 @@ python -c "from transformers import AutoTokenizer; AutoTokenizer.from_pretrained
 ```
 
 🚀 **Your Active Reading demo will be live in minutes!**
+

FEATURE_SUMMARY.md

@@ -0,0 +1,178 @@
+# 🎯 New Features Added to Active Reading Demo
+
+## 📂 **Category Selection Feature**
+
+### What It Does
+Users can now manually select or override the document category detection:
+
+**Available Categories:**
+- **Auto-Detect** (default) - AI detects domain automatically
+- **Finance** - Financial reports, earnings, budgets
+- **Legal** - Contracts, agreements, policies  
+- **Technical** - API docs, manuals, specifications
+- **Medical** - Clinical trials, research, treatments
+- **General** - Any other document type
+
+### Category-Specific Extraction Patterns
+
+#### 📊 Finance Category
+- **Revenue**: `$150 million revenue`, `sales of $2.5B`
+- **Profit**: `profit margin 25%`, `net profit $50M`
+- **Growth**: `15% growth`, `increased by 20%`
+- **Dates**: `Q3 2024`, `fiscal year 2023`
+- **Employees**: `hire 200 engineers`, `workforce of 5000`
+- **Market Cap**: `market cap $10B`
+
+#### ⚖️ Legal Category  
+- **Parties**: `between Company A and Company B`
+- **Terms**: `term of 36 months`, `duration 3 years`
+- **Liability**: `liability not to exceed $1M`
+- **Termination**: `90 days written notice`
+- **Governing Law**: `governed by laws of Delaware`
+- **Effective Date**: `effective January 1, 2024`
+
+#### 🔧 Technical Category
+- **API Endpoints**: `GET /api/users`, `POST /auth/login`
+- **Versions**: `version 2.1.0`, `v3.5`
+- **Response Time**: `response time 150ms`
+- **Rate Limits**: `1000 requests per minute`
+- **Authentication**: `OAuth 2.0`, `JWT tokens`
+- **Status Codes**: `HTTP 200`, `status code 404`
+
+#### 🏥 Medical Category
+- **Dosage**: `50mg daily`, `100ml twice daily`
+- **Duration**: `treatment for 12 weeks`
+- **Efficacy**: `85% efficacy rate`
+- **Side Effects**: `side effects in 12% of patients`
+- **Patient Count**: `500 patients enrolled`
+- **P-Values**: `p<0.001`, `p=0.025`
+
+## 🔑 **Custom Keys Feature**
+
+### What It Does
+Users can specify their own extraction terms as comma-separated values:
+
+**Example Inputs:**
+```
+CEO, budget, deadline, timeline
+risk assessment, compliance, audit
+performance, scalability, security
+treatment, dosage, clinical trial
+```
+
+### How It Works
+- **Smart Extraction**: Finds sentences containing the custom terms
+- **Context Preservation**: Returns full sentences, not just keywords
+- **Confidence Scoring**: Shows extraction confidence levels
+- **JSON Output**: Structured data for easy integration
+
+## 🎯 **New Strategy: Category-Specific Extraction**
+
+### What's New
+Added a specialized strategy that combines:
+1. **Category-specific patterns** for targeted extraction
+2. **Custom key extraction** for user-defined terms
+3. **Structured output** with confidence scores
+4. **Domain expertise** for each business category
+
+### Example Output
+```json
+{
+  "category": "Finance",
+  "extracted_data": {
+    "revenue": ["$150 million", "$2.5 billion sales"],
+    "growth": ["15% increase", "20% growth rate"],
+    "date": ["Q3 2024", "fiscal year 2023"]
+  },
+  "custom_extractions": {
+    "CEO": ["CEO announced plans to expand", "CEO John Smith reported"],
+    "investment": ["$50M investment in AI", "investment in new markets"]
+  },
+  "confidence_scores": {
+    "revenue": 8.5,
+    "custom_CEO": 6.2
+  }
+}
+```
+
+## 🎨 **Enhanced UI Elements**
+
+### New Input Controls
+- **📂 Category Dropdown**: Manual category selection
+- **🔑 Custom Keys Input**: Text field for custom extraction terms
+- **📊 Enhanced Strategy Selection**: Added "Category-Specific Extraction"
+
+### New Output Tabs
+- **🎯 Category Analysis**: Dedicated tab for category-specific results
+- **Enhanced JSON**: Structured category extraction data
+- **Confidence Scores**: Shows extraction reliability
+
+### Improved User Experience
+- **Dynamic Help Text**: Context-aware guidance
+- **Example Suggestions**: Sample custom keys for each category
+- **Better Visual Organization**: Clearer result presentation
+
+## 🚀 **Usage Examples**
+
+### Finance Document Analysis
+```
+Document Category: Finance
+Custom Keys: CEO, quarterly results, investment
+Strategy: Category-Specific Extraction
+```
+
+**Result**: Extracts revenue figures, profit margins, growth rates PLUS CEO mentions, quarterly data, and investment information.
+
+### Legal Contract Review
+```
+Document Category: Legal  
+Custom Keys: liability, termination, governing law
+Strategy: Category-Specific Extraction
+```
+
+**Result**: Finds contract parties, terms, dates PLUS specific liability clauses, termination conditions, and jurisdiction details.
+
+### Technical Documentation
+```
+Document Category: Technical
+Custom Keys: security, performance, scalability  
+Strategy: Category-Specific Extraction
+```
+
+**Result**: Extracts API endpoints, versions, rate limits PLUS security features, performance metrics, and scalability considerations.
+
+## 🎯 **Why This Makes Active Reading Better**
+
+### 1. **Adaptive Intelligence**
+- AI now adapts not just to document type, but to user-specific needs
+- Combines automated domain detection with custom requirements
+
+### 2. **Enterprise Flexibility**  
+- Users can extract exactly what they need for their business case
+- Supports diverse enterprise document analysis workflows
+
+### 3. **Structured Output**
+- Category-specific patterns ensure consistent extraction
+- Custom keys add user-defined flexibility
+- JSON format enables easy integration
+
+### 4. **Demonstrable Value**
+- Shows how Active Reading adapts to different business domains
+- Proves the framework can handle real enterprise requirements
+- Highlights the superiority over one-size-fits-all approaches
+
+## 🎨 **Implementation Impact**
+
+### What Changed in Code
+- **Added**: `extract_category_specific_info()` method
+- **Enhanced**: `process_document()` function with category/custom key parameters  
+- **New**: Category-specific regex patterns for each domain
+- **Improved**: UI with additional input controls and output tabs
+
+### Backward Compatibility
+- ✅ All existing strategies still work
+- ✅ Auto-detection remains the default
+- ✅ Original demo functionality preserved
+- ✅ Enhanced with new capabilities
+
+This makes your Active Reading demo much more interactive and showcases the adaptive intelligence that makes it superior to traditional document processing approaches! 🚀

README.md

@@ -4,7 +4,7 @@ emoji: 🧠
 colorFrom: blue
 colorTo: purple
 sdk: gradio
-sdk_version: 5.44.0
+sdk_version: 4.0.0
 app_file: app.py
 pinned: false
 license: mit
@@ -63,7 +63,7 @@ This is a simplified demo. The complete Enterprise Active Reading Framework incl
 - **Scalable Deployment**: Docker, Kubernetes, monitoring
 - **Advanced Evaluation**: Custom benchmarks and performance metrics
 
-
+For the full implementation, visit: [GitHub Repository](https://github.com/your-repo/active-reader)
 
 ## Citation
 
@@ -71,4 +71,5 @@ Based on the research paper:
 ```
 Lin, J., Berges, V.P., Chen, X., Yih, W.T., Ghosh, G., & Oğuz, B. (2024). 
 Learning Facts at Scale with Active Reading. arXiv:2508.09494.
-```
+```
+

SPACE_BLOG.md

@@ -157,3 +157,4 @@ Contribute new reading strategies and domain adaptations!
 *Built on cutting-edge research, optimized for real-world enterprise use.*
 
 **Tags:** `#ActiveReading` `#AI` `#NLP` `#DocumentAnalysis` `#MachineLearning` `#Enterprise`
+

app.py

@@ -122,6 +122,92 @@ class SimpleActiveReader:
             return "Medical"
         else:
             return "General"
+    
+    def extract_category_specific_info(self, text: str, category: str, custom_keys: List[str]) -> Dict[str, Any]:
+        """Extract information based on selected category and custom keys"""
+        results = {
+            "category": category,
+            "extracted_data": {},
+            "custom_extractions": {},
+            "confidence_scores": {}
+        }
+        
+        # Category-specific extraction patterns
+        category_patterns = {
+            "Finance": {
+                "revenue": r'\$?[\d,]+\.?\d*\s*(?:million|billion|thousand|M|B|K)?\s*(?:revenue|sales|income)',
+                "profit": r'profit.*?\$?[\d,]+\.?\d*|margin.*?[\d,]+\.?\d*%',
+                "growth": r'(?:growth|increase|decrease).*?[\d,]+\.?\d*%',
+                "date": r'\b(?:Q[1-4]|quarter|fiscal|FY)\s*\d{4}|\b\d{1,2}[/-]\d{1,2}[/-]\d{2,4}',
+                "employees": r'(?:employees|staff|workforce).*?[\d,]+',
+                "market_cap": r'market\s*cap.*?\$?[\d,]+\.?\d*\s*(?:million|billion|M|B)'
+            },
+            "Legal": {
+                "parties": r'between\s+([^,]+)\s+and\s+([^,]+)|party.*?([A-Z][a-z]+\s+[A-Z][a-z]+)',
+                "term": r'term.*?(\d+)\s*(?:years?|months?|days?)',
+                "liability": r'liability.*?\$?[\d,]+\.?\d*',
+                "termination": r'terminat.*?(\d+)\s*days?\s*notice',
+                "governing_law": r'governed?\s*by.*?laws?\s*of\s*([^,.]+)',
+                "effective_date": r'effective.*?(\d{1,2}[/-]\d{1,2}[/-]\d{2,4})'
+            },
+            "Technical": {
+                "api_endpoint": r'(?:GET|POST|PUT|DELETE)\s+[/\w-]+|endpoint.*?[/\w-]+',
+                "version": r'version\s*[\d.]+|v[\d.]+',
+                "response_time": r'response.*?(\d+).*?(?:ms|milliseconds|seconds)',
+                "rate_limit": r'rate.*?limit.*?(\d+).*?(?:per|/)\s*(?:minute|hour|second)',
+                "authentication": r'auth.*?(OAuth|JWT|API\s*key|token)',
+                "status_code": r'status.*?(\d{3})|HTTP.*?(\d{3})'
+            },
+            "Medical": {
+                "dosage": r'(\d+)\s*(?:mg|ml|units?)\s*(?:daily|twice|once)',
+                "duration": r'(?:for|duration).*?(\d+)\s*(?:days?|weeks?|months?)',
+                "efficacy": r'efficacy.*?(\d+)%|success.*?(\d+)%',
+                "side_effects": r'side\s*effects?.*?(\d+)%',
+                "patient_count": r'(?:patients?|subjects?).*?(\d+)',
+                "p_value": r'p[<>=]\s*([\d.]+)'
+            },
+            "General": {
+                "numbers": r'\b\d+(?:,\d{3})*(?:\.\d+)?\b',
+                "dates": r'\b\d{1,2}[/-]\d{1,2}[/-]\d{2,4}\b|\b\d{4}\b',
+                "percentages": r'\d+(?:\.\d+)?%',
+                "names": r'\b[A-Z][a-z]+\s+[A-Z][a-z]+\b',
+                "organizations": r'\b[A-Z][a-zA-Z\s&]+(?:Inc|LLC|Corp|Company|Ltd)\b'
+            }
+        }
+        
+        # Extract category-specific information
+        patterns = category_patterns.get(category, category_patterns["General"])
+        
+        for key, pattern in patterns.items():
+            matches = re.findall(pattern, text, re.IGNORECASE)
+            if matches:
+                # Clean up matches
+                cleaned_matches = []
+                for match in matches:
+                    if isinstance(match, tuple):
+                        # Handle tuple results from groups
+                        match = ' '.join([m for m in match if m])
+                    cleaned_matches.append(str(match).strip())
+                
+                results["extracted_data"][key] = cleaned_matches
+                results["confidence_scores"][key] = len(cleaned_matches) / len(text.split()) * 100
+        
+        # Extract custom keys if provided
+        if custom_keys:
+            for custom_key in custom_keys:
+                custom_key = custom_key.strip()
+                if not custom_key:
+                    continue
+                    
+                # Create a pattern to find sentences containing the custom key
+                pattern = f'[^.]*{re.escape(custom_key)}[^.]*'
+                matches = re.findall(pattern, text, re.IGNORECASE)
+                
+                if matches:
+                    results["custom_extractions"][custom_key] = [match.strip() for match in matches]
+                    results["confidence_scores"][f"custom_{custom_key}"] = len(matches) / len(text.split()) * 100
+        
+        return results
 
 # Initialize the model
 try:
@@ -130,22 +216,31 @@ except Exception as e:
     logger.error(f"Failed to initialize model: {e}")
     active_reader = None
 
-def process_document(text: str, strategy: str) -> tuple:
+def process_document(text: str, strategy: str, category: str = None, custom_keys: str = "") -> tuple:
     """
-    Process document with selected strategy
+    Process document with selected strategy, category, and custom keys
     
-    Returns: (result_text, facts_json, questions_json, summary_text, domain)
+    Returns: (result_text, facts_json, questions_json, summary_text, domain, category_data)
     """
     if not active_reader:
-        return "Error: Model not loaded", "", "", "", ""
+        return "Error: Model not loaded", "", "", "", "", ""
     
     if not text.strip():
-        return "Please enter some text to analyze.", "", "", "", ""
+        return "Please enter some text to analyze.", "", "", "", "", ""
     
     try:
         # Detect domain
         domain = active_reader.detect_domain(text)
         
+        # Use manual category if provided, otherwise use detected domain
+        selected_category = category if category and category != "Auto-Detect" else domain
+        
+        # Parse custom keys
+        custom_keys_list = [key.strip() for key in custom_keys.split(",") if key.strip()] if custom_keys else []
+        
+        # Extract category-specific information
+        category_data = active_reader.extract_category_specific_info(text, selected_category, custom_keys_list)
+        
         # Apply selected strategy
         if strategy == "Fact Extraction":
             facts = active_reader.extract_facts(text)
@@ -173,7 +268,7 @@ def process_document(text: str, strategy: str) -> tuple:
             questions = active_reader.generate_questions(text)
             summary = active_reader.generate_summary(text)
             
-            result = f"""**Domain:** {domain}
+            result = f"""**Domain:** {domain} | **Category:** {selected_category}
 
 **Summary:**
 {summary}
@@ -187,12 +282,45 @@ def process_document(text: str, strategy: str) -> tuple:
             facts_json = json.dumps(facts, indent=2)
             questions_json = json.dumps(questions, indent=2)
             summary_text = summary
+            
+        elif strategy == "Category-Specific Extraction":
+            # New strategy for category-specific extraction
+            extracted_data = category_data["extracted_data"]
+            custom_extractions = category_data["custom_extractions"]
+            
+            result = f"""**Category:** {selected_category}
+
+**Category-Specific Extractions:**
+"""
+            
+            for key, values in extracted_data.items():
+                if values:
+                    result += f"\n**{key.replace('_', ' ').title()}:**\n"
+                    for value in values[:3]:  # Show first 3 matches
+                        result += f"• {value}\n"
+                    if len(values) > 3:
+                        result += f"• ... and {len(values) - 3} more\n"
+            
+            if custom_extractions:
+                result += f"\n**Custom Key Extractions:**\n"
+                for key, values in custom_extractions.items():
+                    result += f"\n**{key}:**\n"
+                    for value in values[:2]:  # Show first 2 matches
+                        result += f"• {value}\n"
+                    if len(values) > 2:
+                        result += f"• ... and {len(values) - 2} more\n"
+            
+            facts_json = json.dumps(extracted_data, indent=2)
+            questions_json = json.dumps(custom_extractions, indent=2)
+            summary_text = f"Extracted {len(extracted_data)} category-specific fields and {len(custom_extractions)} custom fields"
+        
+        category_json = json.dumps(category_data, indent=2)
         
-        return result, facts_json, questions_json, summary_text, domain
+        return result, facts_json, questions_json, summary_text, domain, category_json
         
     except Exception as e:
         logger.error(f"Processing error: {e}")
-        return f"Error processing document: {str(e)}", "", "", "", ""
+        return f"Error processing document: {str(e)}", "", "", "", "", ""
 
 def create_demo():
     """Create the Gradio demo interface"""
@@ -255,11 +383,25 @@ def create_demo():
                 
                 # Strategy selection
                 strategy_selector = gr.Radio(
-                    choices=["Fact Extraction", "Question Generation", "Summarization", "Complete Analysis"],
+                    choices=["Fact Extraction", "Question Generation", "Summarization", "Complete Analysis", "Category-Specific Extraction"],
                     value="Complete Analysis",
                     label="Active Reading Strategy"
                 )
                 
+                # Category selection
+                category_selector = gr.Dropdown(
+                    choices=["Auto-Detect", "Finance", "Legal", "Technical", "Medical", "General"],
+                    value="Auto-Detect",
+                    label="📂 Document Category (overrides auto-detection)"
+                )
+                
+                # Custom keys input
+                custom_keys_input = gr.Textbox(
+                    placeholder="e.g., budget, deadline, CEO, risk assessment (comma-separated)",
+                    label="🔑 Custom Extraction Keys",
+                    info="Enter specific terms you want to extract information about"
+                )
+                
                 # Process button
                 process_btn = gr.Button("🚀 Apply Active Reading", variant="primary", size="lg")
             
@@ -282,6 +424,9 @@ def create_demo():
                     
                     with gr.Tab("📝 Summary"):
                         summary_output = gr.Textbox(lines=5, label="Document Summary")
+                    
+                    with gr.Tab("🎯 Category Analysis"):
+                        category_output = gr.Code(language="json", label="Category-Specific Extractions")
         
         # Event handlers
         def load_sample_text(sample_choice):
@@ -297,8 +442,8 @@ def create_demo():
         
         process_btn.click(
             fn=process_document,
-            inputs=[text_input, strategy_selector],
-            outputs=[results_output, facts_output, questions_output, summary_output, domain_output]
+            inputs=[text_input, strategy_selector, category_selector, custom_keys_input],
+            outputs=[results_output, facts_output, questions_output, summary_output, domain_output, category_output]
         )
         
         # How it works and blog section
@@ -331,6 +476,35 @@ def create_demo():
                 - 🔧 **Technology**: API docs, technical specifications, system manuals
                 - 🏥 **Healthcare**: Clinical trials, research papers, treatment protocols
                 - 🏢 **General Business**: Proposals, memos, strategic documents
+                
+                ### 🎯 Category-Specific Extraction
+                
+                **Finance Category extracts:**
+                - Revenue, profit margins, growth rates
+                - Financial dates (Q1 2024, fiscal year)
+                - Employee counts, market cap
+                
+                **Legal Category extracts:**
+                - Contract parties, terms, liability amounts
+                - Termination clauses, governing law
+                - Effective dates and obligations
+                
+                **Technical Category extracts:**
+                - API endpoints, version numbers
+                - Response times, rate limits
+                - Authentication methods, status codes
+                
+                **Medical Category extracts:**
+                - Dosages, treatment duration
+                - Efficacy rates, side effects
+                - Patient counts, statistical significance
+                
+                ### 🔑 Custom Keys Feature
+                
+                Add your own extraction terms like:
+                - `budget, timeline, deliverables` for project docs
+                - `CEO, board, shareholders` for corporate docs  
+                - `security, compliance, audit` for IT policies
                 """)
             
             with gr.Tab("📖 About the Research"):
@@ -373,10 +547,11 @@ def create_demo():
                 
                 **🎮 5-Minute Demo:**
                 1. Select **"Financial Report"** from sample documents
-                2. Choose **"Complete Analysis"** strategy
-                3. Click **"🚀 Apply Active Reading"**
-                4. Explore the extracted facts, questions, and domain detection
-                5. Try different strategies to see how AI adapts!
+                2. Choose **"Category-Specific Extraction"** strategy
+                3. Set category to **"Finance"** (or leave as Auto-Detect)
+                4. Add custom keys: **"CEO, growth, investment"**
+                5. Click **"🚀 Apply Active Reading"**
+                6. Check the **"🎯 Category Analysis"** tab to see targeted extraction!
                 
                 **🔍 Advanced Exploration:**
                 1. **Upload your own document** (paste text up to 2000 words)
@@ -386,12 +561,12 @@ def create_demo():
                 
                 ### Sample Documents Available
                 
-                | Document Type | What You'll Learn |
-                |---------------|-------------------|
-                | 📊 **Financial Report** | How AI extracts metrics, growth data, and financial insights |
-                | ⚖️ **Legal Contract** | How AI identifies key terms, obligations, and risk factors |
-                | 🔧 **Technical Manual** | How AI processes specifications, procedures, and system details |
-                | 🏥 **Medical Research** | How AI handles clinical data, statistics, and medical terminology |
+                | Document Type | Category | Example Custom Keys | What You'll Learn |
+                |---------------|----------|-------------------|-------------------|
+                | 📊 **Financial Report** | Finance | `CEO, growth, investment, Q3` | Revenue extraction, profit analysis, growth metrics |
+                | ⚖️ **Legal Contract** | Legal | `termination, liability, governing law` | Contract terms, obligations, risk factors |
+                | 🔧 **Technical Manual** | Technical | `endpoint, authentication, rate limit` | API specs, system requirements, procedures |
+                | 🏥 **Medical Research** | Medical | `efficacy, patients, side effects` | Clinical data, statistical analysis, treatment outcomes |
                 
                 ### Next Steps
                 

requirements.txt

@@ -1,5 +1,6 @@
 # Minimal requirements for Hugging Face Spaces demo
 torch>=2.0.0
 transformers>=4.30.0
-gradio
+gradio>=4.0.0
 numpy>=1.24.0
+