| | --- |
| | license: mit |
| | --- |
| | FireGenEmbedder |
| |
|
| | FireGenEmbedder is a fine-tuned version of the MiniLM model, specifically adapted for sequence classification tasks. The model has been fine-tuned on the Stanford Natural Language Inference (SNLI) dataset to predict the relationship between two sentences, classifying them into three categories: Entailment, Neutral, and Contradiction. It is designed for applications in legal and other domains requiring inference tasks. |
| |
|
| | Model Details |
| |
|
| | Base Model: sentence-transformers/all-MiniLM-L6-v2 |
| |
|
| | Fine-tuned Dataset: Stanford Natural Language Inference (SNLI) |
| |
|
| | Labels: |
| |
|
| | 0: Contradiction |
| |
|
| | 1: Neutral |
| |
|
| | 2: Entailment |
| |
|
| | Training Epochs: 3 |
| |
|
| | Batch Size: 16 (both train and eval) |
| |
|
| | Precision: Mixed precision for training on GPU |
| |
|
| | Model Usage |
| |
|
| | You can use this model to make inferences on sentence pairs by classifying their relationship. |
| |
|
| | Install Dependencies |
| |
|
| | To use this model, install the following libraries: |
| |
|
| | pip install transformers datasets sentence-transformers torch |
| |
|
| |
|
| | Example Code |
| |
|
| | Here’s an example of how to load and use the FireGenEmbedder model for inference: |
| |
|
| | from transformers import AutoTokenizer, AutoModelForSequenceClassification |
| | import torch |
| |
|
| | # Load the tokenizer and model |
| | model_name = "path_to_firegenembedder_model" |
| | tokenizer = AutoTokenizer.from_pretrained(model_name) |
| | model = AutoModelForSequenceClassification.from_pretrained(model_name) |
| |
|
| | # Move model to device (GPU or CPU) |
| | device = "cuda" if torch.cuda.is_available() else "cpu" |
| | model.to(device) |
| | |
| | # Prepare input |
| | premise = "The sky is blue." |
| | hypothesis = "The sky is not blue." |
| | |
| | inputs = tokenizer(premise, hypothesis, return_tensors="pt", padding=True, truncation=True, max_length=128).to(device) |
| | |
| | # Inference |
| | with torch.no_grad(): |
| | outputs = model(**inputs) |
| | predictions = torch.argmax(outputs.logits, dim=-1) |
| | |
| | # Print the prediction |
| | labels = ["Contradiction", "Neutral", "Entailment"] |
| | print(f"Prediction: {labels[predictions.item()]}") |
| |
|
| |
|
| | Model Fine-Tuning Process |
| |
|
| | Data: The model was fine-tuned using the Stanford Natural Language Inference (SNLI) dataset. The SNLI dataset contains labeled pairs of sentences with three classes: Entailment, Neutral, and Contradiction. |
| |
|
| | Training: |
| |
|
| | The model was fine-tuned for 3 epochs with a batch size of 16 on a GPU. |
| |
|
| | The training used mixed precision for faster computation if a GPU was available. |
| |
|
| | The model is based on the MiniLM architecture, known for being lightweight and efficient, making it suitable for real-time inference tasks. |
| |
|
| | Post-Training: |
| |
|
| | The model was saved and zipped for easy distribution. |
| |
|
| | The tokenizer and model were saved to the directory: miniLM-legal-finetuned-SNLI. |
| |
|
| | Model Evaluation |
| |
|
| | The model was evaluated using the validation set from the SNLI dataset, and results can be accessed as follows: |
| |
|
| | # Load the model and evaluate |
| | results = trainer.evaluate() |
| | print(results) |
| |
|
| | Zipped Model |
| |
|
| | You can download the model as a zip file containing both the model weights and the tokenizer: |
| |
|
| | Download Model |
| |
|
| | Citation |
| |
|
| | If you use this model in your research or application, please cite the following: |
| |
|
| | @misc{firegenembedder, |
| | author = {Your Name}, |
| | title = {FireGenEmbedder: Fine-tuned MiniLM for Legal Inference Tasks}, |
| | year = {2026}, |
| | url = {Link to your Hugging Face model page}, |
| | } |
