hotchpotch commited on
Commit
bc16cde
·
verified ·
1 Parent(s): 2245206

Clarify pretrained checkpoint intended use

Browse files
Files changed (1) hide show
  1. README.md +8 -16
README.md CHANGED
@@ -19,9 +19,10 @@ tags:
19
 
20
  # bekko-embedding-v1-a8m-pt
21
 
22
- bekko-embedding-v1-a8m-pt is the pretrained model used as the base for [bekko-embedding-v1-a8m](https://huggingface.co/hotchpotch/bekko-embedding-v1-a8m). It is the checkpoint before the final retrieval fine-tuning stage and is intended primarily for downstream fine-tuning, pretrain-versus-fine-tuned comparisons, and ablation studies.
23
 
24
- For general-purpose retrieval, use the [fine-tuned model](https://huggingface.co/hotchpotch/bekko-embedding-v1-a8m).
 
25
 
26
  ## Model Details
27
 
@@ -42,22 +43,13 @@ For general-purpose retrieval, use the [fine-tuned model](https://huggingface.co
42
 
43
  The model was pretrained with multilingual text pairs using Matryoshka representation learning and quantization-aware training. The 4-layer backbone retains layers 0, 1, 2, and 18 from mmBERT-small.
44
 
45
- ## Usage
46
 
47
- ```python
48
- from sentence_transformers import SentenceTransformer
49
 
50
- model = SentenceTransformer("hotchpotch/bekko-embedding-v1-a8m-pt")
51
-
52
- queries = ["What is multilingual retrieval?"]
53
- documents = ["Multilingual retrieval searches documents across languages."]
54
-
55
- query_embeddings = model.encode_query(queries, normalize_embeddings=True)
56
- document_embeddings = model.encode_document(documents, normalize_embeddings=True)
57
- scores = query_embeddings @ document_embeddings.T
58
- ```
59
-
60
- This pretrained checkpoint has not received the final supervised retrieval fine-tuning used for bekko-embedding-v1-a8m, so its retrieval quality may be lower or less consistent.
61
 
62
  ## License
63
 
 
19
 
20
  # bekko-embedding-v1-a8m-pt
21
 
22
+ bekko-embedding-v1-a8m-pt is the pretrained model used as the base for [bekko-embedding-v1-a8m](https://huggingface.co/hotchpotch/bekko-embedding-v1-a8m). It is a starting point for additional training on a downstream task, not a model intended for direct use.
23
 
24
+ > [!IMPORTANT]
25
+ > Fine-tune this checkpoint for your target task before using it. It has not received the final supervised retrieval fine-tuning and should not be used directly as a general-purpose embedding or retrieval model. For general-purpose retrieval, use [bekko-embedding-v1-a8m](https://huggingface.co/hotchpotch/bekko-embedding-v1-a8m).
26
 
27
  ## Model Details
28
 
 
43
 
44
  The model was pretrained with multilingual text pairs using Matryoshka representation learning and quantization-aware training. The 4-layer backbone retains layers 0, 1, 2, and 18 from mmBERT-small.
45
 
46
+ ## Intended Use
47
 
48
+ Use this checkpoint as initialization for further training on a specific downstream task. Appropriate uses include:
 
49
 
50
+ - fine-tuning for retrieval, classification, reranking, or another target task,
51
+ - reproducing or adapting the bekko fine-tuning pipeline,
52
+ - pretrain-versus-fine-tuned comparisons and ablation studies.
 
 
 
 
 
 
 
 
53
 
54
  ## License
55