trentmkelly/discord-alt-detector

Discord Alt Detector

trentmkelly/discord-alt-detector is a SentenceTransformers embedding model for Discord stylometry. It embeds a writing sample into a vector. To compare two users, embed one sample from each user and compute cosine similarity.

This checkpoint is ultimately based on TaylorAI/gte-tiny. It is a Discord-focused fine-tune of a broader gte-tiny-based stylometric analysis model. The broader model family is still under active development.

Input Format

Build one text sample per user by concatenating many messages from that user:

• Prefer roughly 1,500-4,000 characters per sample. About 2,000+ characters is closest to the training and evaluation format.
• Use authored message content only. Do not prepend usernames, timestamps, channel names, server names, snowflake IDs, or other metadata.
• Apply light cleanup like the training importer: strip leading/trailing whitespace and replace newlines/carriage returns with spaces.
• Preserve the user's authored spelling, punctuation, casing, emojis, slang, mentions, links, and message text as much as possible.
• Separate individual Discord messages with ||.
• Chronological order is not required. The training importer shuffled each user's messages before joining them into samples.
• Do not mix two users into one sample.

Example:

first message from user || another message || short reply || longer message...

Interpreting Scores

Compute cosine similarity between the two normalized embeddings.

Recommended default mapping, based on a held-out Discord test set:

• cosine < 0.2196: treat as 0% same-author probability.
• cosine > 0.6616: treat as 100% same-author probability.
• Between those values, linearly interpolate from 0% to 100%.

This is an empirical scoring rule, not a mathematical guarantee. Scores near the middle are uncertain and should be treated as leads for further review, not as definitive identification.

On the Discord slice of the held-out test set, this checkpoint reached:

• pair accuracy: 96.70%
• average precision: 0.9870
• best-threshold cosine: about 0.399

Length Ablation

The held-out Discord test samples are mostly a little over 2,000 characters after inserting || delimiters. In this ablation, both sides of each pair were truncated to the listed raw character cutoff before embedding.

cutoff chars	pair accuracy	average precision	best threshold
full	96.70%	0.9870	0.4006
2000	96.55%	0.9870	0.3857
1900	96.55%	0.9868	0.4006
1800	96.40%	0.9867	0.4006
1700	96.25%	0.9859	0.3841
1600	96.40%	0.9863	0.3991
1500	95.65%	0.9858	0.3724
1400	95.20%	0.9856	0.3492
1300	94.75%	0.9828	0.3526
1200	94.75%	0.9807	0.3304
1100	94.45%	0.9783	0.3349
1000	93.55%	0.9754	0.3265
900	92.95%	0.9711	0.3251
800	91.30%	0.9658	0.3041
700	89.66%	0.9627	0.3352
600	89.36%	0.9568	0.3031
500	87.71%	0.9424	0.2561
400	86.06%	0.9248	0.2770
300	82.01%	0.9014	0.2874
200	77.06%	0.8387	0.3593
100	67.02%	0.7173	0.4080

Example Code

from sentence_transformers import SentenceTransformer
from sklearn.metrics.pairwise import cosine_similarity

LOWER = 0.2196
UPPER = 0.6616

def same_author_probability(cosine: float) -> float:
    if cosine <= LOWER:
        return 0.0
    if cosine >= UPPER:
        return 1.0
    return (cosine - LOWER) / (UPPER - LOWER)

model = SentenceTransformer("trentmkelly/discord-alt-detector")

sample_a = "hey || yeah that makes sense || idk maybe later || lmao"
sample_b = "nah i get it || maybe later then || that was funny lmao"

embeddings = model.encode(
    [sample_a, sample_b],
    normalize_embeddings=True,
)

cos = float(cosine_similarity([embeddings[0]], [embeddings[1]])[0][0])
prob = same_author_probability(cos)

print(f"cosine={cos:.4f}")
print(f"same-author probability={prob:.1%}")

Caveats

• This model is intended for stylometric similarity, not identity proof.
• More text generally gives more stable results.
• Same community, same channel, copied memes, bots, quoted text, or coordinated posting style can create false positives.
• Very short samples can be unstable.
• The model was tuned for Discord-like writing and may not generalize cleanly to long-form prose or other platforms.