Twitter sentiment analysis in Python

#How Twitter sentiment analysis works

Sentiment analysis reads a tweet's text and returns one of three labels, usually with a confidence score, then counts the labels across a few hundred or a few million tweets to measure how people feel about a topic. Every project runs the same five steps:

1. Collect the tweets that mention the topic, with a search query and a date window.

2. Clean the text: replace @mentions and links, drop retweets and copies.

3. Score each tweet with a model: positive, neutral or negative.

4. Check the model on a few hundred tweets you labeled yourself.

5. Aggregate the labels into a number per day, per product or per account.

Brands run it on mentions after a launch or an outage, support teams on replies to their posts, analysts on a ticker or a token, and researchers on a debate or an election. It measures how a crowd leans, not what one person meant: a model reads the words, so sarcasm and mixed tweets ("great camera, awful battery") are where it fails most.

Two decisions shape the result more than anything else: which tweets you collect, and which scorer you trust. The next sections take them in that order.

#Get the tweets with Sorsa API

For the collection step, use Sorsa API, a Twitter (X) API and scraper: one search request returns about 20 tweets with X's full search syntax, from $0.07 per 1,000 tweets, at 20 requests per second on every plan. You need one API key and no X developer account, and Sorsa API never asks for an X login, so no account of yours is at risk.

Pick the endpoint by the question you ask:

• Opinion about a topic, product or brand: POST /search-tweets (opens in a new tab).
• What people say to or about one account, with minimum likes or replies: POST /mentions (opens in a new tab), covered in our Twitter mentions API guide.
• Reactions under one post: POST /comments (opens in a new tab), the same replies our guide on how to see comments on X reads.
• Tweet IDs you already have: POST /tweet-info-bulk (opens in a new tab), 100 tweets per request, from $0.014 per 1,000.

1. Get a key. Sign up and copy your API key from the dashboard. The first 100 requests are free, with no card.

2. Write the query. Shape the sample before you pay for it: lang:en keeps one language for an English model, -filter:retweets drops copies, -from: leaves out the brand's own posts, and since: with until: fixes the window. Our list of Twitter search operators has the rest, and the free Twitter Advanced Search tool shows what a query returns before you write code.

3. Send a test request. One request returns about 20 tweets and a cursor to the next page. A cursor is a token that points to the next page of results.

   curl -X POST "https://api.sorsa.io/v3/search-tweets" \
     -H "ApiKey: YOUR_API_KEY" \
     -H "Content-Type: application/json" \
     -d '{"order": "latest",
          "query": "\"battery life\" headphones lang:en -filter:retweets"}'
   

   {
     "tweets": [
       {
         "id": "2116943508275302411",
         "full_text": "Two full days of battery on these headphones",
         "created_at": "2026-11-01T18:42:09Z",
         "lang": "en",
         "is_reply": false,
         "likes_count": 31,
         "reply_count": 4,
         "retweet_count": 2,
         "user": {
           "id": "1834470112963551232",
           "username": "nadia_reviews_gear",
           "followers_count": 2318
         }
       }
     ],
     "next_cursor": "DAADDAABCgABHWAk1xBVYAoKAAIdYCTSHxWQ"
   }
   

   Show full response

4. Collect the pages into a table. Send next_cursor back until it comes back empty, and keep one row per tweet ID. A 429 means more than 20 requests in a second, so wait and retry; a 403 means a missing key or no requests left.

   import time
   
   import pandas as pd
   import requests
   
   URL = "https://api.sorsa.io/v3/search-tweets"
   HEADERS = {"ApiKey": "YOUR_API_KEY"}
   
   
   def collect(query, limit=2000):
       body = {"query": query, "order": "latest"}
       rows = {}
       while len(rows) < limit:
           r = requests.post(URL, headers=HEADERS, json=body, timeout=30)
           if r.status_code == 429:
               time.sleep(1)
               continue
           r.raise_for_status()
           page = r.json()
           for t in page.get("tweets", []):
               rows[t["id"]] = {
                   "id": t["id"],
                   "created_at": t["created_at"],
                   "text": t["full_text"],
                   "lang": t.get("lang"),
                   "is_reply": t.get("is_reply", False),
                   "likes": t.get("likes_count", 0),
                   "author": t["user"]["username"],
               }
           if not page.get("next_cursor"):
               break
           body["next_cursor"] = page["next_cursor"]
       return pd.DataFrame(rows.values())
   
   
   df = collect('"battery life" headphones lang:en -filter:retweets')
   print(len(df), "tweets")
   

What the tweets cost

On Sorsa API a sentiment sample costs from $0.07 per 1,000 tweets through search; the official X API bills $5.00 per 1,000 post reads.

On the official X API the bill also grows with the authors: from November 22, 2026, X bills the profile returned with each post as a user read, $0.010 more per post (X's notice (opens in a new tab)), and its pay-per-use plan stops at 3M post reads a month (X's price list (opens in a new tab)). On Sorsa API every tweet carries its author's full profile, and failed requests are never charged.

The pages of one query come one after another, because each needs the cursor from the last. For a big sample, split the window into days and run the days side by side, up to 20 requests per second on one key. The same search reaches back to 2006, which our guide to historical Twitter data covers for long time series.

#Clean tweets the way the model expects

Clean less than most tutorials do: replace @mentions with @user and links with http, drop retweets and exact copies, and keep emojis, capital letters and punctuation. The RoBERTa model was trained on text prepared exactly this way, and VADER reads capital letters, exclamation marks and emojis as stronger feeling.

import re


def prepare(text):
    text = re.sub(r"@\w+", "@user", text)
    return re.sub(r"https?://\S+", "http", text)


df = df[df["lang"] == "en"]
df = df[~df["text"].str.startswith("RT @")]
df["clean"] = df["text"].map(prepare)
df = df.drop_duplicates(subset="clean")

Exact copies matter more than they look: giveaway and bot campaigns post the same sentence hundreds of times, and one campaign can swing a day's score. For near-duplicates and a gold set of hand labels, our guide to building a Twitter dataset for machine learning has the code.

Good to know

The cleaning regex many tutorials copy, [^a-zA-Z0-9\s], deletes every emoji and every exclamation mark. Both carry sentiment: in VADER's own examples, a line with three emojis and no other opinion words scores +0.70.

#Four ways to score sentiment, tested on 12.3K tweets

On the TweetEval sentiment test set, 12.3K tweets that people labeled positive, neutral or negative, the RoBERTa model got 72.3% right, a logistic regression trained on 45.6K labeled tweets 59.6%, VADER 53.0% and TextBlob 48.3%. We ran all four on November 2, 2026, on 4 CPU threads; the table has the full scores.

• RoBERTa (cardiffnlp, latest)

  Accuracy

  72.3%

  Macro recall

  73.4%

  Called positive (people: 19.3%)

  20.0%

  Tweets per second

  7

• TF-IDF + logistic regression

  Accuracy

  59.6%

  Macro recall

  60.1%

  Called positive (people: 19.3%)

  21.1%

  Tweets per second

  34.0K

• VADER, ±0.05 band

  Accuracy

  53.0%

  Macro recall

  57.0%

  Called positive (people: 19.3%)

  38.9%

  Tweets per second

  27.6K

• TextBlob, sign of polarity

  Accuracy

  48.3%

  Macro recall

  49.4%

  Called positive (people: 19.3%)

  37.2%

  Tweets per second

  6,867

Accuracy is the share of tweets labeled the way people labeled them. For scale, calling every tweet neutral scores 48.3%, because almost half the test tweets are neutral, so VADER adds less than 5 points over doing nothing. Macro recall is TweetEval's own score: the average of how many negative, neutral and positive tweets each method found, so a method can't win by calling everything neutral. Two caveats: the RoBERTa model was fine-tuned on TweetEval's training split, so this test is its home ground, and the logistic regression learned from the same split. On your own topic, expect both to lose a few points, and check them as step 4 says. The speeds come from a shared 4-core server that was busy with other work, so treat them as a floor.

RoBERTa: the most accurate

The twitter-roberta-base-sentiment-latest (opens in a new tab) model from Cardiff NLP is a RoBERTa model trained on about 124M tweets from 2018 to 2021, then fine-tuned for three labels. It runs on an ordinary CPU and needs no API key; at the speed we measured on a busy server, 10K tweets take about 23 minutes, so run it in batches overnight or on a GPU for big samples.

from transformers import pipeline

scorer = pipeline(
    "sentiment-analysis",
    model="cardiffnlp/twitter-roberta-base-sentiment-latest",
)
out = scorer(df["clean"].tolist(), batch_size=32, truncation=True)
df["label"] = [o["label"] for o in out]
df["confidence"] = [round(o["score"], 3) for o in out]

positive 0.978  Battery life on these headphones is great, two full days
positive 0.859  Oh great, another update that breaks Bluetooth. Love ...
neutral  0.789  The firmware update rolls out to all devices on Monday.
positive 0.960  not bad at all tbh, @user was right http
negative 0.860  @user oh great, the update broke Bluetooth AGAIN http

For tweets in other languages, the same team's twitter-xlm-roberta-base-sentiment (opens in a new tab) model was trained on about 198M tweets and fine-tuned in 8 languages, among them Spanish, Portuguese, French and Arabic. For emotions rather than polarity, their emotion model returns 11 labels such as anger, fear and joy.

VADER: instant rules, a weak neutral line

VADER is a list of about 7,500 scored words plus rules for capitals, exclamation marks, "but" and negation, built for social media. It scored about 27.6K tweets a second on one core, which makes it a fine first look at a million tweets. Its weak spot is the neutral band: it called 38.9% of the test tweets positive, twice the 19.3% people did.

from vaderSentiment.vaderSentiment import SentimentIntensityAnalyzer

vader = SentimentIntensityAnalyzer()


def vader_label(text, band=0.05):
    c = vader.polarity_scores(text)["compound"]
    if c >= band:
        return "positive"
    return "negative" if c <= -band else "neutral"


df["vader"] = df["text"].map(vader_label)

• Battery life on these headphones is great, two full days

  VADER (compound)

  positive (+0.62)

  RoBERTa

  positive (0.98)

• Oh great, another update that breaks Bluetooth. Love that for me.

  VADER (compound)

  positive (+0.85)

  RoBERTa

  positive (0.86)

• The firmware update rolls out to all devices on Monday.

  VADER (compound)

  neutral (+0.00)

  RoBERTa

  neutral (0.79)

• not bad at all tbh, @kai_builds_tools was right https://t.co/x1y2z3 (opens in a new tab)

  VADER (compound)

  positive (+0.43)

  RoBERTa

  positive (0.96)

• @nadia_reviews_gear oh great, the update broke Bluetooth AGAIN https://t.co/k3v9q2 (opens in a new tab)

  VADER (compound)

  positive (+0.32)

  RoBERTa

  negative (0.86)

Sarcasm beats both scorers some of the time. Both read "Love that for me" as praise; RoBERTa caught the sarcasm in the last tweet, where VADER saw "great" and scored it positive.

Logistic regression: train it on your own labels

A TF-IDF model with logistic regression learns from tweets you label yourself, so it fits your topic's words: "sick" in a sneaker thread, "bullish" in a crypto one. Trained on 45.6K TweetEval tweets in 13 seconds, it reached 59.6%. It pays off when you have a few thousand labels and a vocabulary the general models don't know.

from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import make_pipeline

model = make_pipeline(
    TfidfVectorizer(ngram_range=(1, 2), min_df=2,
                    sublinear_tf=True),
    LogisticRegression(max_iter=2000,
                       class_weight="balanced"),
)
model.fit(train_texts, train_labels)
df["lr"] = model.predict(df["clean"])

TextBlob, the fourth method, scores a lexicon of English adjectives that its authors tested on movie reviews, and calls a tweet neutral only when its polarity is exactly zero. It scored 48.3%, the same as calling every tweet neutral, so it adds nothing on tweets. A large language model is a fifth option: prompted with the tweet and the three labels, it can catch sarcasm that word lists miss, but it costs money per tweet and can answer differently on a rerun. Check it against the same hand-labeled sample before you trust it on a whole dataset.

#The neutral band decides your headline

With VADER, the threshold you pick moves the result more than the tweets do. At the ±0.05 band from VADER's documentation it calls 38.9% of the test tweets positive; widen the band to ±0.45 and the positive share falls to 21.4% while accuracy rises to 58.1%. Move the slider to see every step.

Report the method and its threshold with every number you publish, and never compare shares from two different scorers. If you use VADER, tune the band on a few hundred tweets from your own topic that you labeled by hand.

#Turn labels into a score you can track

The number most teams track is net sentiment: the share of positive tweets minus the share of negative ones, from -100 to +100. Count it per day, and leave out days with fewer than about 100 tweets, where a handful of posts can swing the score.

df["day"] = pd.to_datetime(df["created_at"]).dt.date
shares = pd.crosstab(df["day"], df["label"], normalize="index")
daily = pd.DataFrame({
    "tweets": df.groupby("day").size(),
    "net": ((shares["positive"] - shares["negative"]) * 100).round(1),
})
print(daily[daily["tweets"] >= 100])

Keep replies and original posts apart (the is_reply field), because complaints cluster in replies to a brand's own posts. Weigh tweets by likes only on purpose, and say so next to the chart. To keep the score current, run the collection on a schedule with since: set to the last run; our guide to real-time Twitter monitoring covers polling at scale.

#Run your first sentiment analysis today

Collect a few thousand tweets on your topic with Sorsa API's POST /search-tweets, about 20 per request from $0.07 per 1,000, score them with the RoBERTa model, and check the labels on 200 tweets of your own. The 100 free requests on a new account cover the first 2,000 tweets.

#FAQ