Upload folder using huggingface_hub

README.md

@@ -1,559 +1,158 @@
 ---
-license: apache-2.0
 language:
 - en
+license: apache-2.0
 tags:
 - medical
 - biomedical
 - drug-safety
-- adverse-drug-reactions
+- adverse-drug-events
 - pharmacovigilance
 - relation-extraction
 - dual-encoder
 - clinical-nlp
-- biolinkbert
-- entity-markers
-- hard-negative-mining
-- focal-loss
-- causal-reasoning
-- mimicause
-- mimic
-- clinical-notes
 datasets:
-- ade-benchmark-corpus/ade_corpus_v2
-- chrisvoncsefalvay/mimicause-private
+- ade_corpus_v2
+- mimicause
 metrics:
 - f1
+- precision
+- recall
 - roc_auc
 pipeline_tag: text-classification
-model-index:
-- name: CRAG-dual-encoder-mimicause
-  results:
-  - task:
-      type: text-classification
-      name: Drug-ADR Relation Extraction
-    dataset:
-      name: ADE Corpus V2 + MIMICause
-      type: ade-benchmark-corpus/ade_corpus_v2
-      config: Ade_corpus_v2_drug_ade_relation
-    metrics:
-    - type: f1
-      value: 0.9889
-      name: F1 Score
-    - type: roc_auc
-      value: 0.9981
-      name: ROC-AUC
+library_name: pytorch
+base_model:
+- michiyasunaga/BioLinkBERT-base
 ---
 
-# CRAG-dual-encoder-mimicause
-
-**CRAG: Causal Reasoning for Adversomics Graphs**
+# CRAG: Causal Reasoning for Adversomics Graphs
 
-This is the **best-performing model** in the CRAG dual-encoder family, achieving **98.89% F1** and **99.81% AUC** on drug-ADR relation extraction. It combines training on the ADE Corpus V2 with the MIMICause dataset, leveraging diverse causal reasoning patterns from real clinical notes.
+**CRAG** (Causal Reasoning for Adversomics Graphs) is a dual-encoder model for extracting adverse drug event (ADE) relationships from clinical narratives. It achieves state-of-the-art performance on ADE extraction, significantly outperforming both specialized biomedical language models and large language models.
 
 ## Model Description
 
-CRAG-dual-encoder-mimicause extends the ADE model architecture with additional training on MIMICause, a dataset of causal relations annotated in MIMIC-III clinical notes. This provides the model with:
-
-- **Diverse causal language patterns** from real clinical documentation
-- **Multiple causation types:** Cause, Enable, Prevent, Hinder
-- **Clinical writing style exposure** beyond curated biomedical literature
-
-### Why MIMICause?
-
-The ADE Corpus V2 provides high-quality drug-ADR pairs from MEDLINE, but clinical notes contain different linguistic patterns:
-
-| Source | Style | Patterns |
-|--------|-------|----------|
-| ADE Corpus (MEDLINE) | Formal, structured | "Drug X induced condition Y" |
-| MIMICause (MIMIC-III) | Clinical, abbreviated | "pt on X c/b Y", "Y 2/2 X" |
-
-Training on both improves generalization to real-world clinical text.
-
-### Architecture
-
-Same enhanced dual-encoder architecture as CRAG-dual-encoder-ade:
-
-```
-┌─────────────────────────────────────────────────────────────────┐
-│                 CRAG Dual-Encoder MIMICause                     │
-├─────────────────────────────────────────────────────────────────┤
-│                                                                 │
-│   Drug Context                    ADR Context                   │
-│   "[DRUG] steroid [/DRUG]         "[ADR] myopathy [/ADR]        │
-│    induced myopathy"               from steroids"               │
-│           │                             │                       │
-│           ▼                             ▼                       │
-│   ┌─────────────┐               ┌─────────────┐                 │
-│   │ BioLinkBERT │               │ BioLinkBERT │                 │
-│   │   Drug      │               │    ADR      │                 │
-│   │  Encoder    │               │   Encoder   │                 │
-│   └──────┬──────┘               └──────┬──────┘                 │
-│          │                             │                        │
-│          ▼                             ▼                        │
-│   ┌─────────────┐               ┌─────────────┐                 │
-│   │  Attention  │               │  Attention  │                 │
-│   │   Pooling   │               │   Pooling   │                 │
-│   └──────┬──────┘               └──────┬──────┘                 │
-│          │                             │                        │
-│          ▼                             ▼                        │
-│   ┌─────────────┐               ┌─────────────┐                 │
-│   │ Projection  │               │ Projection  │                 │
-│   │  768 → 256  │               │  768 → 256  │                 │
-│   └──────┬──────┘               └──────┬──────┘                 │
-│          │                             │                        │
-│          └──────────┬──────────────────┘                        │
-│                     ▼                                           │
-│             ┌──────────────┐                                    │
-│             │   Bilinear   │                                    │
-│             │   + Concat   │                                    │
-│             └──────┬───────┘                                    │
-│                    ▼                                            │
-│             ┌──────────────┐                                    │
-│             │  Classifier  │                                    │
-│             │  512→256→1   │                                    │
-│             └──────┬───────┘                                    │
-│                    ▼                                            │
-│                P(causal)                                        │
-└─────────────────────────────────────────────────────────────────┘
-```
-
-### Model Specifications
-
-- **Base Model:** `michiyasunaga/BioLinkBERT-base`
-- **Hidden Dimension:** 768
-- **Fusion Dimension:** 256
-- **Attention Heads:** 4
-- **Total Parameters:** 238,667,009
-- **Special Tokens:** `[DRUG]`, `[/DRUG]`, `[ADR]`, `[/ADR]`
-
-## Training Data
-
-### Combined Dataset
-
-| Dataset | Split | Examples | Source |
-|---------|-------|----------|--------|
-| ADE Corpus V2 | Train | 13,642 | MEDLINE case reports |
-| ADE Corpus V2 | Val | 2,047 | MEDLINE case reports |
-| MIMICause | Train | 2,281 | MIMIC-III clinical notes |
-| MIMICause | Val | 403 | MIMIC-III clinical notes |
-| **Combined** | **Train** | **15,923** | - |
-
-### MIMICause Label Mapping
-
-MIMICause provides nuanced causal annotations that were mapped to binary labels:
-
-| MIMICause Label | Mapped To | Rationale |
-|-----------------|-----------|-----------|
-| Cause(E1,E2) | Positive | Direct causation |
-| Cause(E2,E1) | Positive | Direct causation (reversed) |
-| Enable(E1,E2) | Positive | Enabling = contributing cause |
-| Enable(E2,E1) | Positive | Enabling (reversed) |
-| Prevent(E1,E2) | Negative | Opposite of causation |
-| Prevent(E2,E1) | Negative | Opposite (reversed) |
-| Hinder(E1,E2) | Negative | Partial prevention |
-| Hinder(E2,E1) | Negative | Partial prevention (reversed) |
-| Other | Negative | No causal relationship |
-
-### Entity Classification in MIMICause
-
-MIMICause annotates generic entities (E1, E2). Drug entities were identified using:
-
-1. **Lexicon matching:** Common drug names (aspirin, morphine, etc.)
-2. **Suffix patterns:** -ine, -ol, -mab, -pril, -statin, etc.
-3. **Default assignment:** When ambiguous, E1 = drug, E2 = ADR
+CRAG uses a dual-encoder architecture with:
+- **Two separate BioLinkBERT encoders**: One for drug mentions, one for adverse event mentions
+- **Attention pooling**: Multi-head attention mechanism for sequence representation
+- **Bilinear fusion**: Captures complex drug-ADR interactions
+- **Multi-view concatenation**: Combines bilinear output, individual embeddings, and element-wise products
 
-## Training Procedure
-
-### Phase 1: Contrastive Pre-training (5 epochs)
-
-```python
-CONFIG = {
-    "temperature": 0.07,
-    "hard_negative_ratio": 0.5,
-    "batch_size": 16,
-    "gradient_accumulation_steps": 4,
-    "max_length": 256,  # Longer for clinical notes
-}
-```
-
-- InfoNCE loss with hard negative mining
-- Combined ADE + MIMICause training data
-- Learns unified embedding space for both data sources
-
-### Phase 2: Classification Fine-tuning (8 epochs)
-
-```python
-CONFIG = {
-    "learning_rate": 2e-5,
-    "warmup_ratio": 0.1,
-    "layerwise_lr_decay": 0.9,
-    "focal_gamma": 2.0,
-    "focal_alpha": 0.75,
-    "weight_decay": 0.01,
-}
-```
+### Training Approach
 
-- Focal loss for hard example mining
-- Cosine learning rate schedule
-- Layer-wise learning rate decay (0.9× per layer)
+The model is trained in two phases:
+1. **Phase 1 - Contrastive Pre-training**: InfoNCE loss with hard negative mining to learn discriminative drug-ADR embeddings
+2. **Phase 2 - Classification Fine-tuning**: Focal loss to handle class imbalance and refine the classifier
 
 ## Performance
 
-### Final Metrics
+### Test Set Results
 
-| Metric | Value |
+| Metric | Score |
 |--------|-------|
-| **F1 Score** | 98.89% |
-| **ROC-AUC** | 99.81% |
-| **Optimal Threshold** | 0.70 |
-
-### Training Progression
-
-| Epoch | Train Loss | Train F1 | Val F1 | Val AUC |
-|-------|------------|----------|--------|---------|
-| 1 | 0.1842 | 93.21% | 94.56% | 98.12% |
-| 2 | 0.0523 | 96.34% | 97.23% | 99.34% |
-| 4 | 0.0187 | 97.89% | 98.12% | 99.78% |
-| 8 | 0.0059 | 98.23% | **98.89%** | **99.81%** |
-
-### Comparison with CRAG Family
+| **F1 Score** | 0.9332 |
+| **Precision** | 0.9075 |
+| **Recall** | 0.9603 |
+| **AUC-ROC** | 0.9765 |
 
-| Model | F1 | AUC | Δ F1 vs Base |
-|-------|-----|-----|--------------|
-| CRAG-dual-encoder-base | 88.3% | - | - |
-| CRAG-dual-encoder-ade | 97.5% | 99.1% | +9.2% |
-| **CRAG-dual-encoder-mimicause** | **98.89%** | **99.81%** | **+10.59%** |
+### Comparison with Baselines
 
-### MIMICause Contribution
-
-Training on the additional 2,281 MIMICause examples improved:
-- **F1:** 97.5% → 98.89% (+1.39 percentage points)
-- **AUC:** 99.1% → 99.81% (+0.71 percentage points)
-
-The MIMICause data provides complementary signal through:
-- Clinical language patterns different from MEDLINE
-- Exposure to causal reasoning beyond simple drug-ADR pairs
-- Real-world clinical documentation style
+| Model | F1 Score | AUC-ROC | F1 Improvement |
+|-------|----------|---------|----------------|
+| BioLinkBERT (zero-shot) | 0.215 | 0.523 | - |
+| GPT-4 Turbo | 0.734 | 0.713 | - |
+| Qwen2.5-1.5B-Instruct | 0.714 | 0.728 | - |
+| **CRAG (this model)** | **0.933** | **0.977** | **+27% vs GPT-4** |
 
 ## Usage
 
-### Loading the Model
-
 ```python
 import torch
-import torch.nn as nn
-from transformers import AutoTokenizer, AutoModel
-
-class AttentionPooling(nn.Module):
-    def __init__(self, hidden_dim=768, num_heads=4):
-        super().__init__()
-        self.attention = nn.MultiheadAttention(hidden_dim, num_heads, batch_first=True)
-        self.query = nn.Parameter(torch.randn(1, 1, hidden_dim))
-
-    def forward(self, hidden_states, attention_mask):
-        batch_size = hidden_states.size(0)
-        query = self.query.expand(batch_size, -1, -1)
-        key_padding_mask = ~attention_mask.bool()
-        pooled, _ = self.attention(query, hidden_states, hidden_states,
-                                   key_padding_mask=key_padding_mask)
-        return pooled.squeeze(1)
-
-class CRAGDualEncoder(nn.Module):
-    def __init__(self, model_name="michiyasunaga/BioLinkBERT-base",
-                 hidden_dim=768, fusion_dim=256, dropout=0.1):
-        super().__init__()
-
-        # Dual encoders
-        self.drug_encoder = AutoModel.from_pretrained(model_name)
-        self.adr_encoder = AutoModel.from_pretrained(model_name)
-
-        # Attention pooling
-        self.drug_pooler = AttentionPooling(hidden_dim)
-        self.adr_pooler = AttentionPooling(hidden_dim)
-
-        # Projection heads
-        self.drug_projection = nn.Sequential(
-            nn.Linear(hidden_dim, fusion_dim),
-            nn.LayerNorm(fusion_dim),
-            nn.GELU(),
-            nn.Dropout(dropout),
-            nn.Linear(fusion_dim, fusion_dim),
-        )
-        self.adr_projection = nn.Sequential(
-            nn.Linear(hidden_dim, fusion_dim),
-            nn.LayerNorm(fusion_dim),
-            nn.GELU(),
-            nn.Dropout(dropout),
-            nn.Linear(fusion_dim, fusion_dim),
-        )
-
-        # Fusion and classification
-        self.bilinear = nn.Bilinear(fusion_dim, fusion_dim, fusion_dim)
-        self.fusion_norm = nn.LayerNorm(fusion_dim)
-        self.classifier = nn.Sequential(
-            nn.Linear(fusion_dim * 2, fusion_dim),
-            nn.LayerNorm(fusion_dim),
-            nn.GELU(),
-            nn.Dropout(dropout),
-            nn.Linear(fusion_dim, fusion_dim // 2),
-            nn.GELU(),
-            nn.Dropout(dropout),
-            nn.Linear(fusion_dim // 2, 1),
-        )
-
-    def encode_drug(self, input_ids, attention_mask):
-        outputs = self.drug_encoder(input_ids=input_ids, attention_mask=attention_mask)
-        pooled = self.drug_pooler(outputs.last_hidden_state, attention_mask)
-        return self.drug_projection(pooled)
-
-    def encode_adr(self, input_ids, attention_mask):
-        outputs = self.adr_encoder(input_ids=input_ids, attention_mask=attention_mask)
-        pooled = self.adr_pooler(outputs.last_hidden_state, attention_mask)
-        return self.adr_projection(pooled)
-
-    def forward(self, drug_input_ids, drug_attention_mask, adr_input_ids, adr_attention_mask):
-        drug_repr = self.encode_drug(drug_input_ids, drug_attention_mask)
-        adr_repr = self.encode_adr(adr_input_ids, adr_attention_mask)
-
-        bilinear_out = self.bilinear(drug_repr, adr_repr)
-        bilinear_out = self.fusion_norm(bilinear_out)
-        combined = torch.cat([bilinear_out, drug_repr + adr_repr], dim=-1)
-
-        return self.classifier(combined)
+from transformers import AutoTokenizer
 
-# Load model
-tokenizer = AutoTokenizer.from_pretrained("chrisvoncsefalvay/CRAG-dual-encoder-mimicause")
-model = CRAGDualEncoder()
+# Load tokenizer
+tokenizer = AutoTokenizer.from_pretrained("michiyasunaga/BioLinkBERT-base")
+tokenizer.add_special_tokens({'additional_special_tokens': ['[DRUG]', '[/DRUG]', '[ADR]', '[/ADR]']})
 
-# Load weights
+# Load model
 from huggingface_hub import hf_hub_download
-weights_path = hf_hub_download(
-    repo_id="chrisvoncsefalvay/CRAG-dual-encoder-mimicause",
-    filename="pytorch_model.bin"
-)
-model.load_state_dict(torch.load(weights_path, map_location="cpu"))
-model.eval()
-```
+import torch
 
-### Inference
+# Download and load the model
+model_path = hf_hub_download(repo_id="chrisvoncsefalvay/CRAG-dual-encoder-mimicause", filename="pytorch_model.pt")
+checkpoint = torch.load(model_path, map_location='cpu')
 
-```python
-def predict_causal_relationship(model, tokenizer, text, drug, adr, threshold=0.70):
-    """
-    Predict if a drug-ADR pair has a causal relationship.
-
-    Args:
-        model: Loaded CRAGDualEncoder
-        tokenizer: Loaded tokenizer
-        text: Clinical text containing both entities
-        drug: Drug entity string
-        adr: ADR entity string
-        threshold: Classification threshold (default 0.70)
-
-    Returns:
-        dict with probability and prediction
-    """
-    # Create marked contexts
-    drug_context = text.replace(drug, f"[DRUG] {drug} [/DRUG]", 1)
-    adr_context = text.replace(adr, f"[ADR] {adr} [/ADR]", 1)
-
-    # Tokenize
-    drug_enc = tokenizer(
-        drug_context,
-        max_length=256,
-        padding="max_length",
-        truncation=True,
-        return_tensors="pt"
-    )
-    adr_enc = tokenizer(
-        adr_context,
-        max_length=256,
-        padding="max_length",
-        truncation=True,
-        return_tensors="pt"
-    )
-
-    # Predict
-    with torch.no_grad():
-        logit = model(
-            drug_enc["input_ids"],
-            drug_enc["attention_mask"],
-            adr_enc["input_ids"],
-            adr_enc["attention_mask"]
-        )
-        prob = torch.sigmoid(logit).item()
-
-    return {
-        "probability": prob,
-        "is_causal": prob >= threshold,
-        "confidence": abs(prob - 0.5) * 2  # 0 = uncertain, 1 = confident
-    }
-
-# Example: Clinical note from MIMIC-style text
-result = predict_causal_relationship(
-    model, tokenizer,
-    text="Steroid myopathy: Patient had a history of steroid induced myopathy and had presented with an ongoing steroid taper.",
-    drug="steroid",
-    adr="myopathy"
-)
-print(f"Probability: {result['probability']:.3f}")
-print(f"Is Causal: {result['is_causal']}")
-print(f"Confidence: {result['confidence']:.3f}")
-# Output:
-# Probability: 0.967
-# Is Causal: True
-# Confidence: 0.934
+# For inference, see the example notebook in the repository
+
+# Example input format:
+drug_context = "The patient developed [DRUG] aspirin [/DRUG] induced gastric bleeding."
+adr_context = "The patient developed aspirin induced [ADR] gastric bleeding [/ADR]."
 ```
 
-### Batch Processing
+## Model Architecture
 
-```python
-def batch_predict(model, tokenizer, pairs, batch_size=32):
-    """
-    Predict for multiple drug-ADR pairs efficiently.
-
-    Args:
-        pairs: List of (text, drug, adr) tuples
-
-    Returns:
-        List of probability scores
-    """
-    results = []
-
-    for i in range(0, len(pairs), batch_size):
-        batch = pairs[i:i+batch_size]
-
-        drug_texts = [p[0].replace(p[1], f"[DRUG] {p[1]} [/DRUG]", 1) for p in batch]
-        adr_texts = [p[0].replace(p[2], f"[ADR] {p[2]} [/ADR]", 1) for p in batch]
-
-        drug_enc = tokenizer(drug_texts, max_length=256, padding=True,
-                            truncation=True, return_tensors="pt")
-        adr_enc = tokenizer(adr_texts, max_length=256, padding=True,
-                           truncation=True, return_tensors="pt")
-
-        with torch.no_grad():
-            logits = model(
-                drug_enc["input_ids"],
-                drug_enc["attention_mask"],
-                adr_enc["input_ids"],
-                adr_enc["attention_mask"]
-            )
-            probs = torch.sigmoid(logits).squeeze(-1).tolist()
-            results.extend(probs)
-
-    return results
 ```
+CRAGDualEncoder(
+  (drug_encoder): BioLinkBERT-base (110M params)
+  (adr_encoder): BioLinkBERT-base (110M params)
+  (drug_pooler): AttentionPooling (4 heads)
+  (adr_pooler): AttentionPooling (4 heads)
+  (drug_projection): Linear(768 -> 256) + LayerNorm + GELU + Linear
+  (adr_projection): Linear(768 -> 256) + LayerNorm + GELU + Linear
+  (bilinear): Bilinear(256, 256 -> 256)
+  (classifier): Linear(1024 -> 256) + LayerNorm + GELU + Linear(256 -> 128) + GELU + Linear(128 -> 1)
+)
 
-## Intended Uses
+Total Parameters: 238,798,081
+```
 
-### Primary Applications
+## Training Data
 
-1. **Clinical Pharmacovigilance**
-   - Real-time ADR detection in EHR systems
-   - Screening clinical notes for safety signals
-   - Supporting adverse event reporting (FAERS, EudraVigilance)
+The model was trained on a combination of:
+- **ADE Corpus v2**: Biomedical literature annotations for drug-adverse event pairs
+- **MIMICause**: Clinical notes from MIMIC-III with causal ADE annotations
 
-2. **Drug Safety Knowledge Graphs**
-   - Automated edge extraction for drug-ADR graphs
-   - Causal graph construction for pharmacovigilance
-   - Integration with existing drug databases (DrugBank, SIDER)
+| Split | Samples | Positive | Negative |
+|-------|---------|----------|----------|
+| Train | 12,978 | 6,264 | 6,714 |
+| Validation | 1,667 | 812 | 855 |
+| Test | 1,681 | 807 | 874 |
 
-3. **Literature Mining**
-   - Systematic review automation
-   - Case report screening
-   - Meta-analysis support
+## Training Configuration
 
-4. **Regulatory Applications**
-   - Pre-submission safety signal detection
-   - Post-market surveillance
-   - Label update recommendations
+| Parameter | Phase 1 (Contrastive) | Phase 2 (Classification) |
+|-----------|----------------------|--------------------------|
+| Epochs | 5 | 8 |
+| Batch Size | 16 | 16 |
+| Learning Rate | 2e-5 | 2e-5 |
+| Loss Function | InfoNCE (τ=0.07) | Focal (γ=2.0, α=0.75) |
+| Hard Negatives | 50% | - |
 
-### Recommended Threshold Selection
+## Experiment Tracking
 
-| Use Case | Threshold | Precision | Recall | F1 |
-|----------|-----------|-----------|--------|-----|
-| High-sensitivity screening | 0.50 | 95.1% | 99.2% | 97.1% |
-| Balanced (default) | 0.70 | 98.5% | 98.9% | 98.7% |
-| High-precision extraction | 0.85 | 99.5% | 96.8% | 98.1% |
+- **WandB Run**: [68d4wq4u](https://wandb.ai/chrisvoncsefalvay/crag-experiments/runs/68d4wq4u)
+- **Dataset Artifacts**: [HuggingFace Dataset](https://huggingface.co/datasets/chrisvoncsefalvay/crag-experiment-artifacts)
 
 ## Limitations
 
-1. **English Only:** Not validated on non-English clinical text
-2. **Binary Output:** Does not distinguish causation types or severity
-3. **Entity Detection Required:** Assumes drug and ADR entities are pre-identified
-4. **Training Data Scope:**
-   - ADE Corpus: MEDLINE case reports
-   - MIMICause: US ICU clinical notes (MIMIC-III)
-   - May not generalize to other clinical settings or countries
-5. **Temporal Reasoning:** Does not explicitly model temporal relationships
-
-## Ethical Considerations
-
-### Clinical Deployment
-
-- **Human-in-the-Loop:** Always require expert review for clinical decisions
-- **Confidence Communication:** Surface probability scores, not just binary predictions
-- **Error Analysis:** Monitor for systematic biases in drug or ADR categories
-
-### Bias Considerations
-
-- Training data reflects US clinical practice and MEDLINE publication patterns
-- Common drugs/ADRs better represented than rare ones
-- May underperform on pediatric, geriatric, or specialty populations
-
-### Privacy
-
-- Model does not retain patient information
-- MIMICause derived from de-identified MIMIC-III data
-- No personal health information in model weights
-
-## Technical Specifications
-
-| Specification | Value |
-|---------------|-------|
-| Framework | PyTorch 2.0+ |
-| Base Model | BioLinkBERT-base |
-| Model Size | 955 MB |
-| Vocabulary Size | 30,522 + 4 special tokens |
-| Max Sequence Length | 256 tokens |
-| Inference Speed (GPU) | ~85 pairs/second |
-| Inference Speed (CPU) | ~5 pairs/second |
-| GPU Memory (inference) | ~2 GB |
-| Training Hardware | NVIDIA A10G |
-| Training Time | ~90 minutes |
+- Trained primarily on English clinical and biomedical text
+- Requires drug and ADR spans to be pre-identified (not an end-to-end NER+RE model)
+- Performance may vary on drug/ADR pairs not seen during training
+- Best suited for binary relation classification, not relation type classification
 
 ## Citation
 
+If you use this model, please cite:
+
 ```bibtex
-@misc{crag-dual-encoder-mimicause-2024,
-  title={CRAG: Causal Reasoning for Adversomics Graphs - Dual-Encoder Enhanced with Clinical Causal Reasoning},
-  author={von Csefalvay, Chris},
-  year={2024},
-  publisher={Hugging Face},
-  url={https://huggingface.co/chrisvoncsefalvay/CRAG-dual-encoder-mimicause}
+@misc{crag2024,
+  author = {von Csefalvay, Chris},
+  title = {CRAG: Causal Reasoning for Adversomics Graphs},
+  year = {2024},
+  publisher = {Hugging Face},
+  howpublished = {\url{https://huggingface.co/chrisvoncsefalvay/CRAG-dual-encoder-mimicause}}
 }
 ```
 
-## References
-
-- Gurulingappa, H., et al. (2012). Development of a benchmark corpus to support the automatic extraction of drug-related adverse effects from medical case reports. *Journal of Biomedical Informatics*.
-- Yasunaga, M., et al. (2022). LinkBERT: Pretraining Language Models with Document Links. *ACL*.
-- Khetan, V., et al. (2022). MIMICause: Representation and automatic extraction of causal relation types from clinical notes. *ACL Findings*.
-- Johnson, A.E.W., et al. (2016). MIMIC-III, a freely accessible critical care database. *Scientific Data*.
-
-## Related Models
-
-- [CRAG-dual-encoder-base](https://huggingface.co/chrisvoncsefalvay/CRAG-dual-encoder-base) - Baseline PubMedBERT model
-- [CRAG-dual-encoder-ade](https://huggingface.co/chrisvoncsefalvay/CRAG-dual-encoder-ade) - Enhanced ADE-only model
-
-## Model Card Authors
-
-Chris von Csefalvay ([@chrisvoncsefalvay](https://huggingface.co/chrisvoncsefalvay))
-
-## Model Card Contact
+## License
 
-For questions, issues, or collaboration inquiries:
-- Open a discussion on this model's repository
-- Email: [email protected]
-- GitHub: [@chrisvoncsefalvay](https://github.com/chrisvoncsefalvay)
+Apache 2.0

config.json

@@ -0,0 +1,10 @@
+{
+  "model": {
+    "base_model": "michiyasunaga/BioLinkBERT-base",
+    "hidden_dim": 768,
+    "fusion_dim": 256,
+    "dropout": 0.1,
+    "pooling": "attention",
+    "attention_heads": 4
+  }
+}

pytorch_model.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:38572981c02b8494a10a530410b836f3228484726ed3d9fb0cbeae13b6b492ad
+size 955369379

test_results.json

@@ -0,0 +1,18 @@
+{
+  "f1": 0.9331727874774233,
+  "precision": 0.9074941451990632,
+  "recall": 0.9603469640644362,
+  "auc": 0.9765410779251344,
+  "confusion_matrix": [
+    [
+      795,
+      79
+    ],
+    [
+      32,
+      775
+    ]
+  ],
+  "num_samples": 1681,
+  "num_positive": 807
+}

tokenizer.json

@@ -1,21 +1,7 @@
 {
   "version": "1.0",
-  "truncation": {
-    "direction": "Right",
-    "max_length": 256,
-    "strategy": "LongestFirst",
-    "stride": 0
-  },
-  "padding": {
-    "strategy": {
-      "Fixed": 256
-    },
-    "direction": "Right",
-    "pad_to_multiple_of": null,
-    "pad_id": 0,
-    "pad_type_id": 0,
-    "pad_token": "[PAD]"
-  },
+  "truncation": null,
+  "padding": null,
   "added_tokens": [
     {
       "id": 0,