sandiumenge/twitter-bitcoin-emotion-classification

Twitter bitcoin related emotion classification

This model aims to classify tweets related to bitcoin or crypto topics as "Greed", "Excitement", "Optimism", "Neutral", "Pessimism", "Fear", "Anger".

The reason behind having this model when there are plenty of them already posted is that bicoin related tweets get often classified instantly as spam, and because of that as greed. So having this model already trained over a bitcoin related dataset removes this prejudice making it possible to work with bitcoin related tweets.

The model is a fine-tuned version of vinai/bertweet-base (a roBERTa based model fine-tuned with 850M English Tweets) and trained for emotion classification over a bitcoin related dataset.

Example of classification

BERTweet was trained over normalized tweets such as: tweet = "DHEC confirms HTTPURL via @USER :crying_face:", so for better results it's recomended to normalize the texts before applying the spam detection. What it does is converting user mentions and web/url links into special tokens @USER and HTTPURL, respectively and other preprocessing modifications. To do so, copy or download the TweetNormalizer provided at the BERTweet webpage:

!git clone https://github.com/VinAIResearch/BERTweet.git
!pip install emoji

from transformers import pipeline
import sys

# (Optional to improve accuracy)
sys.path.append('/content/BERTweet') # Or whatever directory the folder got downloaded in
from TweetNormalizer import normalizeTweet

classifier = pipeline("text-classification",
                      model="sandiumenge/twitter-bitcoin-emotion-classification",
                      tokenizer="sandiumenge/twitter-bitcoin-emotion-classification",
                      truncation=True,
                      padding=True,
                      max_length=128
)

tweet = "I'm winning iPhone XS，BTC，ETH and other Awards. Join with us!@freecoinhunt https://t.co/VIUwLmdy4n"
normalized_tweet = normalizeTweet(tweet)
# "I 'm winning iPhone XS ， BTC ， ETH and other Awards . Join with us ! @USER HTTPURL"

print(classifier(normalized_tweet))

>> [{'label': 'Greed', 'score': 0.8803344011306763}]

It achieves the following results on the evaluation set:

• Loss: 1.7032
• Accuracy: 0.6561
• F1: 0.6572
• Macro F1: 0.6226
• Precision: 0.6630
• Recall: 0.6561

The model is trained over real data instead of a specific dataset, making the results appear worse than most other competitors, which adds uncertainty and also has Tweets representing multiple emotions, making it harder to classify. Although, the model has been trained with a cost matrix to make the predictions fall faster for similar emotions, penalizing less mistaking "Optimism" with "Excitement" than mistaking it for "Pessimism".