Hugging Face's logo

Trelis
/
multi-qa-MiniLM-L6-dot-v1-ft-triplets-2-cst-epoch-overlap

Sentence Similarity
sentence-transformers
Safetensors
bert
feature-extraction
Generated from Trainer
dataset_size:9729
loss:TripletLoss
text-embeddings-inference
Model card Files
xet
 Community

SentenceTransformer based on sentence-transformers/multi-qa-MiniLM-L6-dot-v1

This is a sentence-transformers model finetuned from sentence-transformers/multi-qa-MiniLM-L6-dot-v1. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Sources

Full Model Architecture 

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("Trelis/multi-qa-MiniLM-L6-dot-v1-ft-triplets-2-cst-epoch-overlap")
# Run inference
sentences = [
    'What is the minimum number of males and females required on the field of play in mixed gender competitions?',
    '5. 3. 1 this does not apply for players sent to the sin bin area. 5. 4 in mixed gender competitions, the maximum number of males allowed on the field of play is three ( 3 ), the minimum male requirement is one ( 1 ) and the minimum female requirement is one ( 1 ). 6 team coach and team officials 6. 1 the team coach ( s ) and team officials may be permitted inside the perimeter but shall be required to be positioned either in the interchange area or at the end of the field of play for the duration of the match. 6. 2 the team coach ( s ) and team officials may move from one position to the other but shall do so without delay. while in a position at the end of the field of play, the team coach ( s ) or team official must remain no closer than five ( 5 ) metres from the dead ball line and must not coach or communicate ( verbal or non - verbal ) with either team or the referees.',
    'tap and tap penalty the method of commencing the match, recommencing the match after half time and after a try has been scored. the tap is also the method of recommencing play when a penalty is awarded. the tap is taken by placing the ball on the ground at or behind the mark, releasing both hands from the ball, tapping the ball gently with either foot or touching the foot on the ball. the ball must not roll or move more than one ( 1 ) metre in any direction and must be retrieved cleanly, without touching the ground again. the player may face any direction and use either foot. provided it is at the mark, the ball does not have to be lifted from the ground prior to a tap being taken. team a group of players constituting one ( 1 ) side in a competition match. tfa touch football australia limited touch any contact between the player in possession and a defending player. a touch includes contact on the ball, hair or clothing and may be made by a defending player or by the player in possession. touch count the progressive number of touches that each team has before a change of possession, from zero ( 0 ) to six ( 6 ).',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Training Details

Training Hyperparameters

Non-Default Hyperparameters

  • eval_strategy: steps
  • per_device_train_batch_size: 32
  • per_device_eval_batch_size: 32
  • learning_rate: 2e-05
  • num_train_epochs: 2
  • lr_scheduler_type: constant
  • warmup_ratio: 0.3

All Hyperparameters

Click to expand
  • overwrite_output_dir: False
  • do_predict: False
  • eval_strategy: steps
  • prediction_loss_only: True
  • per_device_train_batch_size: 32
  • per_device_eval_batch_size: 32
  • per_gpu_train_batch_size: None
  • per_gpu_eval_batch_size: None
  • gradient_accumulation_steps: 1
  • eval_accumulation_steps: None
  • learning_rate: 2e-05
  • weight_decay: 0.0
  • adam_beta1: 0.9
  • adam_beta2: 0.999
  • adam_epsilon: 1e-08
  • max_grad_norm: 1.0
  • num_train_epochs: 2
  • max_steps: -1
  • lr_scheduler_type: constant
  • lr_scheduler_kwargs: {}
  • warmup_ratio: 0.3
  • warmup_steps: 0
  • log_level: passive
  • log_level_replica: warning
  • log_on_each_node: True
  • logging_nan_inf_filter: True
  • save_safetensors: True
  • save_on_each_node: False
  • save_only_model: False
  • restore_callback_states_from_checkpoint: False
  • no_cuda: False
  • use_cpu: False
  • use_mps_device: False
  • seed: 42
  • data_seed: None
  • jit_mode_eval: False
  • use_ipex: False
  • bf16: False
  • fp16: False
  • fp16_opt_level: O1
  • half_precision_backend: auto
  • bf16_full_eval: False
  • fp16_full_eval: False
  • tf32: None
  • local_rank: 0
  • ddp_backend: None
  • tpu_num_cores: None
  • tpu_metrics_debug: False
  • debug: []
  • dataloader_drop_last: False
  • dataloader_num_workers: 0
  • dataloader_prefetch_factor: None
  • past_index: -1
  • disable_tqdm: False
  • remove_unused_columns: True
  • label_names: None
  • load_best_model_at_end: False
  • ignore_data_skip: False
  • fsdp: []
  • fsdp_min_num_params: 0
  • fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
  • fsdp_transformer_layer_cls_to_wrap: None
  • accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
  • deepspeed: None
  • label_smoothing_factor: 0.0
  • optim: adamw_torch
  • optim_args: None
  • adafactor: False
  • group_by_length: False
  • length_column_name: length
  • ddp_find_unused_parameters: None
  • ddp_bucket_cap_mb: None
  • ddp_broadcast_buffers: False
  • dataloader_pin_memory: True
  • dataloader_persistent_workers: False
  • skip_memory_metrics: True
  • use_legacy_prediction_loop: False
  • push_to_hub: False
  • resume_from_checkpoint: None
  • hub_model_id: None
  • hub_strategy: every_save
  • hub_private_repo: False
  • hub_always_push: False
  • gradient_checkpointing: False
  • gradient_checkpointing_kwargs: None
  • include_inputs_for_metrics: False
  • eval_do_concat_batches: True
  • fp16_backend: auto
  • push_to_hub_model_id: None
  • push_to_hub_organization: None
  • mp_parameters:
  • auto_find_batch_size: False
  • full_determinism: False
  • torchdynamo: None
  • ray_scope: last
  • ddp_timeout: 1800
  • torch_compile: False
  • torch_compile_backend: None
  • torch_compile_mode: None
  • dispatch_batches: None
  • split_batches: None
  • include_tokens_per_second: False
  • include_num_input_tokens_seen: False
  • neftune_noise_alpha: None
  • optim_target_modules: None
  • batch_eval_metrics: False
  • eval_on_start: False
  • batch_sampler: batch_sampler
  • multi_dataset_batch_sampler: proportional

Training Logs

Click to expand
Epoch Step Training Loss loss
0.0066 2 4.2577 -
0.0131 4 4.0287 -
0.0197 6 4.1022 -
0.0262 8 3.8676 -
0.0328 10 3.836 -
0.0393 12 3.5016 -
0.0459 14 3.5338 -
0.0525 16 3.2488 -
0.0590 18 3.1999 -
0.0656 20 3.1266 -
0.0721 22 2.7272 -
0.0787 24 2.9005 -
0.0852 26 2.4328 -
0.0918 28 2.0929 -
0.0984 30 1.9004 -
0.1049 32 2.0227 -
0.1115 34 1.8577 -
0.1180 36 1.6767 -
0.1246 38 1.6381 -
0.1311 40 1.391 -
0.1377 42 1.6477 -
0.1443 44 1.7922 -
0.1508 46 1.3904 -
0.1574 48 1.7869 -
0.1639 50 1.6701 -
0.1705 52 1.1823 -
0.1770 54 0.9928 -
0.1836 56 1.4254 -
0.1902 58 1.4535 -
0.1967 60 1.3876 -
0.2033 62 1.411 -
0.2098 64 0.9778 -
0.2164 66 0.9914 -
0.2230 68 0.9946 -
0.2295 70 0.9828 -
0.2361 72 0.7432 -
0.2426 74 1.315 -
0.2492 76 0.6955 -
0.2525 77 - 0.5755
0.2557 78 0.7919 -
0.2623 80 1.001 -
0.2689 82 1.086 -
0.2754 84 0.8213 -
0.2820 86 0.9834 -
0.2885 88 1.0083 -
0.2951 90 0.6879 -
0.3016 92 0.672 -
0.3082 94 0.663 -
0.3148 96 1.0285 -
0.3213 98 1.0634 -
0.3279 100 0.9316 -
0.3344 102 0.88 -
0.3410 104 0.9057 -
0.3475 106 0.7303 -
0.3541 108 0.8927 -
0.3607 110 0.6541 -
0.3672 112 0.6616 -
0.3738 114 0.9198 -
0.3803 116 0.4953 -
0.3869 118 0.7159 -
0.3934 120 0.6596 -
0.4 122 0.8359 -
0.4066 124 0.7748 -
0.4131 126 0.5949 -
0.4197 128 0.4204 -
0.4262 130 1.0151 -
0.4328 132 0.4979 -
0.4393 134 0.6496 -
0.4459 136 0.6039 -
0.4525 138 0.6333 -
0.4590 140 0.5494 -
0.4656 142 0.5599 -
0.4721 144 0.353 -
0.4787 146 0.6529 -
0.4852 148 0.5215 -
0.4918 150 0.6408 -
0.4984 152 0.6084 -
0.5049 154 0.8947 0.2713
0.5115 156 0.515 -
0.5180 158 0.4075 -
0.5246 160 0.7453 -
0.5311 162 0.4385 -
0.5377 164 0.5747 -
0.5443 166 0.725 -
0.5508 168 0.6338 -
0.5574 170 0.2453 -
0.5639 172 0.4578 -
0.5705 174 0.3541 -
0.5770 176 0.5326 -
0.5836 178 0.3699 -
0.5902 180 0.1714 -
0.5967 182 0.3149 -
0.6033 184 0.561 -
0.6098 186 0.2852 -
0.6164 188 0.2715 -
0.6230 190 0.5521 -
0.6295 192 0.2852 -
0.6361 194 0.5103 -
0.6426 196 0.3866 -
0.6492 198 0.4369 -
0.6557 200 0.3936 -
0.6623 202 0.5084 -
0.6689 204 0.4912 -
0.6754 206 0.2659 -
0.6820 208 0.6209 -
0.6885 210 0.3022 -
0.6951 212 0.2738 -
0.7016 214 0.5555 -
0.7082 216 0.3672 -
0.7148 218 0.3489 -
0.7213 220 0.6139 -
0.7279 222 0.4402 -
0.7344 224 0.2829 -
0.7410 226 0.3669 -
0.7475 228 0.729 -
0.7541 230 0.2565 -
0.7574 231 - 0.1849
0.7607 232 0.2596 -
0.7672 234 0.2359 -
0.7738 236 0.4406 -
0.7803 238 0.2629 -
0.7869 240 0.3583 -
0.7934 242 0.5298 -
0.8 244 0.6225 -
0.8066 246 0.3853 -
0.8131 248 0.4741 -
0.8197 250 0.3991 -
0.8262 252 0.5629 -
0.8328 254 0.2935 -
0.8393 256 0.3563 -
0.8459 258 0.3628 -
0.8525 260 0.2416 -
0.8590 262 0.1493 -
0.8656 264 0.2488 -
0.8721 266 0.4055 -
0.8787 268 0.1286 -
0.8852 270 0.4217 -
0.8918 272 0.3529 -
0.8984 274 0.1921 -
0.9049 276 0.1736 -
0.9115 278 0.4308 -
0.9180 280 0.0992 -
0.9246 282 0.3927 -
0.9311 284 0.3451 -
0.9377 286 0.4504 -
0.9443 288 0.3065 -
0.9508 290 0.2844 -
0.9574 292 0.4308 -
0.9639 294 0.1754 -
0.9705 296 0.2608 -
0.9770 298 0.4232 -
0.9836 300 0.3234 -
0.9902 302 0.24 -
0.9967 304 0.2112 -
1.0033 306 0.6322 -
1.0098 308 0.2987 0.1357
1.0164 310 0.4052 -
1.0230 312 0.1458 -
1.0295 314 0.2593 -
1.0361 316 0.193 -
1.0426 318 0.29 -
1.0492 320 0.299 -
1.0557 322 0.0841 -
1.0623 324 0.0534 -
1.0689 326 0.2166 -
1.0754 328 0.2431 -
1.0820 330 0.2621 -
1.0885 332 0.0986 -
1.0951 334 0.4274 -
1.1016 336 0.2388 -
1.1082 338 0.0899 -
1.1148 340 0.158 -
1.1213 342 0.1748 -
1.1279 344 0.1226 -
1.1344 346 0.1815 -
1.1410 348 0.2312 -
1.1475 350 0.4114 -
1.1541 352 0.2258 -
1.1607 354 0.1519 -
1.1672 356 0.1536 -
1.1738 358 0.103 -
1.1803 360 0.2901 -
1.1869 362 0.1629 -
1.1934 364 0.1541 -
1.2 366 0.1986 -
1.2066 368 0.2492 -
1.2131 370 0.2137 -
1.2197 372 0.1954 -
1.2262 374 0.1947 -
1.2328 376 0.2114 -
1.2393 378 0.4277 -
1.2459 380 0.3636 -
1.2525 382 0.4151 -
1.2590 384 0.2258 -
1.2623 385 - 0.1095
1.2656 386 0.2794 -
1.2721 388 0.2504 -
1.2787 390 0.3785 -
1.2852 392 0.2448 -
1.2918 394 0.3936 -
1.2984 396 0.1686 -
1.3049 398 0.2301 -
1.3115 400 0.1533 -
1.3180 402 0.2516 -
1.3246 404 0.1238 -
1.3311 406 0.1629 -
1.3377 408 0.1395 -
1.3443 410 0.1093 -
1.3508 412 0.0899 -
1.3574 414 0.1793 -
1.3639 416 0.0648 -
1.3705 418 0.2402 -
1.3770 420 0.2711 -
1.3836 422 0.1457 -
1.3902 424 0.1338 -
1.3967 426 0.3074 -
1.4033 428 0.0738 -
1.4098 430 0.1702 -
1.4164 432 0.111 -
1.4230 434 0.249 -
1.4295 436 0.1143 -
1.4361 438 0.2255 -
1.4426 440 0.3167 -
1.4492 442 0.0751 -
1.4557 444 0.1101 -
1.4623 446 0.2098 -
1.4689 448 0.2086 -
1.4754 450 0.0978 -
1.4820 452 0.3184 -
1.4885 454 0.1347 -
1.4951 456 0.2259 -
1.5016 458 0.1651 -
1.5082 460 0.2183 -
1.5148 462 0.1315 0.0771
1.5213 464 0.2672 -
1.5279 466 0.1783 -
1.5344 468 0.0408 -
1.5410 470 0.4634 -
1.5475 472 0.1762 -
1.5541 474 0.0553 -
1.5607 476 0.2445 -
1.5672 478 0.1988 -
1.5738 480 0.1985 -
1.5803 482 0.1484 -
1.5869 484 0.1403 -
1.5934 486 0.1993 -
1.6 488 0.1486 -
1.6066 490 0.2899 -
1.6131 492 0.2464 -
1.6197 494 0.1352 -
1.6262 496 0.1233 -
1.6328 498 0.0413 -
1.6393 500 0.091 -
1.6459 502 0.0828 -
1.6525 504 0.1488 -
1.6590 506 0.1246 -
1.6656 508 0.2795 -
1.6721 510 0.067 -
1.6787 512 0.168 -
1.6852 514 0.2215 -
1.6918 516 0.0854 -
1.6984 518 0.2192 -
1.7049 520 0.1479 -
1.7115 522 0.1924 -
1.7180 524 0.2075 -
1.7246 526 0.208 -
1.7311 528 0.1743 -
1.7377 530 0.0817 -
1.7443 532 0.1513 -
1.7508 534 0.3422 -
1.7574 536 0.1101 -
1.7639 538 0.2815 -
1.7672 539 - 0.0693
1.7705 540 0.1837 -
1.7770 542 0.0879 -
1.7836 544 0.0746 -
1.7902 546 0.2052 -
1.7967 548 0.1416 -
1.8033 550 0.1141 -
1.8098 552 0.0312 -
1.8164 554 0.139 -
1.8230 556 0.1078 -
1.8295 558 0.1302 -
1.8361 560 0.0124 -
1.8426 562 0.2641 -
1.8492 564 0.1625 -
1.8557 566 0.1907 -
1.8623 568 0.0 -
1.8689 570 0.1721 -
1.8754 572 0.1178 -
1.8820 574 0.0345 -
1.8885 576 0.0924 -
1.8951 578 0.0513 -
1.9016 580 0.0929 -
1.9082 582 0.1502 -
1.9148 584 0.0338 -
1.9213 586 0.1348 -
1.9279 588 0.0297 -
1.9344 590 0.0306 -
1.9410 592 0.1416 -
1.9475 594 0.0427 -
1.9541 596 0.1916 -
1.9607 598 0.1969 -
1.9672 600 0.0765 -
1.9738 602 0.1035 -
1.9803 604 0.261 -
1.9869 606 0.0845 -
1.9934 608 0.0566 -
2.0 610 0.704 -

Framework Versions

  • Python: 3.10.12
  • Sentence Transformers: 3.0.1
  • Transformers: 4.42.3
  • PyTorch: 2.1.1+cu121
  • Accelerate: 0.31.0
  • Datasets: 2.17.1
  • Tokenizers: 0.19.1

Citation

BibTeX

Sentence Transformers 

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

TripletLoss 

@misc{hermans2017defense,
    title={In Defense of the Triplet Loss for Person Re-Identification}, 
    author={Alexander Hermans and Lucas Beyer and Bastian Leibe},
    year={2017},
    eprint={1703.07737},
    archivePrefix={arXiv},
    primaryClass={cs.CV}
}
Downloads last month
116
Safetensors
Model size
22.7M params
Tensor type
F32
·
Inference Providers NEW
Sentence Similarity
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for Trelis/multi-qa-MiniLM-L6-dot-v1-ft-triplets-2-cst-epoch-overlap

Base model

sentence-transformers/multi-qa-MiniLM-L6-dot-v1
Finetuned
(5)
this model

Papers for Trelis/multi-qa-MiniLM-L6-dot-v1-ft-triplets-2-cst-epoch-overlap