Hugging Face's logo

ComparisonPO
/
Mistral-Base-7B-DPO_clean

Safetensors
mistral
Model card Files
xet
Community

DPO model excluding the noisy preference pairs for Mistral-Base under trl/ultradeedback_binarized finetuning.

Downloads last month
6
Safetensors
Model size
7B params
Tensor type
BF16
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for ComparisonPO/Mistral-Base-7B-DPO_clean

Base model

mistralai/Mistral-7B-v0.1
Finetuned
alignment-handbook/zephyr-7b-sft-full
Finetuned
(382)
this model
Finetunes
1 model

Dataset used to train ComparisonPO/Mistral-Base-7B-DPO_clean