| | --- |
| | language: |
| | - english |
| | thumbnail: |
| | tags: |
| | - token classification |
| | license: |
| | datasets: |
| | - EMBO/sd-panels |
| | metrics: |
| | - |
| | --- |
| | |
| | # sd-ner |
| |
|
| | ## Model description |
| |
|
| | This model is a [RoBERTa base model](https://huggingface.co/roberta-base) that was further trained using a masked language modeling task on a compendium of english scientific textual examples from the life sciences using the [BioLang dataset](https://huggingface.co/datasets/EMBO/biolang) and fine-tuned for token classification on the SourceData [sd-panels](https://huggingface.co/datasets/EMBO/sd-panels) dataset to perform Named Entity Recognition of bioentities. |
| |
|
| |
|
| | ## Intended uses & limitations |
| |
|
| | #### How to use |
| |
|
| | The intended use of this model is for Named Entity Recognition of biological entitie used in SourceData annotations (https://sourcedata.embo.org), including small molecules, gene products (genes and proteins), subcellular components, cell line and cell types, organ and tissues, species as well as experimental methods. |
| |
|
| | To have a quick check of the model: |
| |
|
| | ```python |
| | from transformers import pipeline, RobertaTokenizerFast, RobertaForTokenClassification |
| | example = """<s> F. Western blot of input and eluates of Upf1 domains purification in a Nmd4-HA strain. The band with the # might corresponds to a dimer of Upf1-CH, bands marked with a star correspond to residual signal with the anti-HA antibodies (Nmd4). Fragments in the eluate have a smaller size because the protein A part of the tag was removed by digestion with the TEV protease. G6PDH served as a loading control in the input samples </s>""" |
| | tokenizer = RobertaTokenizerFast.from_pretrained('roberta-base', max_len=512) |
| | model = RobertaForTokenClassification.from_pretrained('EMBO/sd-ner') |
| | ner = pipeline('ner', model, tokenizer=tokenizer) |
| | res = ner(example) |
| | for r in res: |
| | print(r['word'], r['entity']) |
| | ``` |
| |
|
| | #### Limitations and bias |
| |
|
| | The model must be used with the `roberta-base` tokenizer. |
| |
|
| | ## Training data |
| |
|
| | The model was trained for token classification using the [EMBO/sd-panels dataset](https://huggingface.co/datasets/EMBO/biolang) wich includes manually annotated examples. |
| |
|
| | ## Training procedure |
| |
|
| | The training was run on a NVIDIA DGX Station with 4XTesla V100 GPUs. |
| |
|
| | Training code is available at https://github.com/source-data/soda-roberta |
| |
|
| | - Command: `python -m tokcl.train /data/json/sd_panels NER --num_train_epochs=3.5` |
| | - Tokenizer vocab size: 50265 |
| | - Training data: EMBO/biolang MLM |
| | - Training with 31410 examples. |
| | - Evaluating on 8861 examples. |
| | - Training on 15 features: O, I-SMALL_MOLECULE, B-SMALL_MOLECULE, I-GENEPROD, B-GENEPROD, I-SUBCELLULAR, B-SUBCELLULAR, I-CELL, B-CELL, I-TISSUE, B-TISSUE, I-ORGANISM, B-ORGANISM, I-EXP_ASSAY, B-EXP_ASSAY |
| | - Epochs: 3.5 |
| | - `per_device_train_batch_size`: 32 |
| | - `per_device_eval_batch_size`: 32 |
| | - `learning_rate`: 0.0001 |
| | - `weight_decay`: 0.0 |
| | - `adam_beta1`: 0.9 |
| | - `adam_beta2`: 0.999 |
| | - `adam_epsilon`: 1e-08 |
| | - `max_grad_norm`: 1.0 |
| |
|
| | ## Eval results |
| |
|
| | On test set with `sklearn.metrics`: |
| |
|
| | ``` |
| | precision recall f1-score support |
| | |
| | CELL 0.77 0.81 0.79 3477 |
| | EXP_ASSAY 0.71 0.70 0.71 7049 |
| | GENEPROD 0.86 0.90 0.88 16140 |
| | ORGANISM 0.80 0.82 0.81 2759 |
| | SMALL_MOLECULE 0.78 0.82 0.80 4446 |
| | SUBCELLULAR 0.71 0.75 0.73 2125 |
| | TISSUE 0.70 0.75 0.73 1971 |
| | |
| | micro avg 0.79 0.82 0.81 37967 |
| | macro avg 0.76 0.79 0.78 37967 |
| | weighted avg 0.79 0.82 0.81 37967 |
| | ``` |
| |
|
