How to get historical Twitter data back to 2006

#How far back historical Twitter data goes

Historical Twitter data starts on March 21, 2006, the day of the first tweet, and every public post still online since then can be found by searching for it. No one hands out the complete archive as a file, so you pull the slice you need: by query, by account or by tweet ID.

Three kinds of posts are out of reach for every API and every tool: tweets their authors deleted, posts from protected accounts and posts from suspended accounts. A dataset collected in 2019 and collected again today comes back smaller for exactly that reason.

For a few lookups by hand, the free Twitter Advanced Search tool runs the same date operators in the browser. Anything you will count, chart or train a model on needs an API.

What a 2014 tweet returns today

A historical tweet comes back with its text and date exactly as posted, but with today's counters and the author's profile as it is today, from any API.

• Text, date, ID, links and media

  What you get for a tweet posted in 2014

  As posted in 2014

• Like, repost, reply and quote counts

  What you get for a tweet posted in 2014

  Today's totals, not the counts in 2014

• Author: handle, bio, follower count

  What you get for a tweet posted in 2014

  The author's profile today, a renamed handle included

So a study of how a hashtag spread in 2014 can count its tweets per day exactly. It cannot read 2014 follower counts from the archive, because a tweet never stored them. If you need point-in-time numbers in the future, collect tweets as they appear and keep the counts yourself.

#Three ways to pull historical tweets

For historical tweets, use Sorsa API: one API key reaches every public tweet back to 2006 at 20 requests per second on every plan, from $0.07 per 1,000 by search. The endpoint depends on what you start with, a topic and dates, one account or a list of tweet IDs, and every tweet arrives with its author's full profile.

Search covers most research: a hashtag during an election, a brand's mentions in 2019, a cashtag across five years. The account route reads one timeline from the newest post to the oldest. The ID route rebuilds a dataset someone already published, and it costs the least per tweet because one request carries 100 tweets.

#How to search historical tweets with the API

Searching the archive with Sorsa API takes four steps and one key, and none of them asks for an X account, an app review or OAuth. The same endpoint runs live searches too, which our Twitter search API guide covers.

1. Get a key. Sign up and copy your API key from the dashboard. The 100 free requests are already on it.

2. Write the query. Put the words first, then the window: since:2014-04-01 includes that day and until:2014-05-01 stops before it, both in UTC. Narrow it with from:, lang:, min_faves: or -filter:retweets; every operator is in our list of Twitter search operators.

3. Send the first request. POST the query to POST /search-tweets (opens in a new tab) with "order": "latest", so the pages walk through the window in time order.

   curl -X POST "https://api.sorsa.io/v3/search-tweets" \
     -H "ApiKey: YOUR_API_KEY" \
     -H "Content-Type: application/json" \
     -d '{
       "query": "\"rainfall survey\" since:2014-04-01 until:2014-05-01",
       "order": "latest"
     }'
   

   {
     "tweets": [
       {
         "id": "456721912166192197",
         "full_text": "Day 3 of the rainfall survey: 41 mm in the valley.",
         "created_at": "2014-04-17T09:12:44Z",
         "lang": "en",
         "likes_count": 38,
         "retweet_count": 12,
         "reply_count": 4,
         "quote_count": 1,
         "is_reply": false,
         "user": {
           "id": "583920117",
           "username": "mira_archive_notes",
           "display_name": "Mira Field Notes",
           "followers_count": 2214,
           "created_at": "2012-06-02T18:40:11Z"
         }
       }
     ],
     "next_cursor": "DAADDAABCgABGEZx0iWAJ8kKAAIYRm0eqZvwRg"
   }
   

   Show full response

4. Page through the window. Send next_cursor back in the body with the same query, and stop when a response comes without one. Pages hold about 20 tweets and their size varies, so a short page never means the window is done.

A 403 answer means the key is missing, wrong or out of requests; a 429 means more than 20 requests in one second, so wait a second and send it again.

Good to know

Use "order": "latest" for a backfill: it walks a date window in time order, which is how a complete collection works. "popular" puts the most-engaged tweets first, and it suits finding the top posts of a period, not counting every post in it.

#How to backfill years of tweets without gaps

A backfill of several years runs best as many small date windows, a month or a week each, paged to the end and saved one at a time. A failed window then reruns alone, every window can be checked by its count, and because until: excludes its own day, back-to-back windows never save a tweet twice.

The script below walks a date range month by month and pages each window through next_cursor. It writes one JSON Lines file per month and skips months already on disk, so you can stop it and start it again at any point.

import json
import os
import time
from datetime import date

import requests

URL = "https://api.sorsa.io/v3/search-tweets"
HEADERS = {"ApiKey": "YOUR_API_KEY"}


def months(start, end):
    """Yield (since, until) pairs, one calendar month each."""
    d = start
    while d < end:
        nxt = date(d.year + d.month // 12, d.month % 12 + 1, 1)
        yield d, min(nxt, end)
        d = nxt


def backfill(words, start, end, folder="archive"):
    os.makedirs(folder, exist_ok=True)
    for since, until in months(start, end):
        path = f"{folder}/{since:%Y-%m}.jsonl"
        if os.path.exists(path):
            continue  # this month is already saved
        body = {"query": f"{words} since:{since} until:{until}",
                "order": "latest"}
        rows = []
        while True:
            r = requests.post(URL, headers=HEADERS, json=body)
            if r.status_code == 429:
                time.sleep(1)  # over 20 requests a second
                continue
            r.raise_for_status()
            page = r.json()
            rows += page.get("tweets", [])
            if not page.get("next_cursor"):
                break
            body["next_cursor"] = page["next_cursor"]
        with open(path, "w", encoding="utf-8") as f:
            for t in rows:
                f.write(json.dumps(t) + "\n")
        print(since, len(rows), "tweets")


backfill('"rainfall survey" lang:en', date(2015, 1, 1), date(2021, 1, 1))

// backfill.mjs, Node.js 18 or later
import fs from "node:fs";

const URL = "https://api.sorsa.io/v3/search-tweets";
const headers = {
  ApiKey: "YOUR_API_KEY",
  "Content-Type": "application/json",
};
const day = (d) => d.toISOString().slice(0, 10);
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));

async function backfill(words, firstYear, endYear, folder = "archive") {
  fs.mkdirSync(folder, { recursive: true });
  for (let y = firstYear; y < endYear; y++) {
    for (let m = 0; m < 12; m++) {
      const since = new Date(Date.UTC(y, m, 1));
      const until = new Date(Date.UTC(y, m + 1, 1));
      const path = `${folder}/${day(since).slice(0, 7)}.jsonl`;
      if (fs.existsSync(path)) continue; // this month is already saved
      const body = {
        query: `${words} since:${day(since)} until:${day(until)}`,
        order: "latest",
      };
      const rows = [];
      while (true) {
        const res = await fetch(URL, {
          method: "POST",
          headers,
          body: JSON.stringify(body),
        });
        if (res.status === 429) { await sleep(1000); continue; }
        if (!res.ok) throw new Error(`HTTP ${res.status}`);
        const page = await res.json();
        rows.push(...(page.tweets ?? []));
        if (!page.next_cursor) break;
        body.next_cursor = page.next_cursor;
      }
      const lines = rows.map((t) => JSON.stringify(t) + "\n");
      fs.writeFileSync(path, lines.join(""));
      console.log(day(since), rows.length, "tweets");
    }
  }
}

await backfill('"rainfall survey" lang:en', 2015, 2021);

Busy topics such as a cashtag or a world event are best cut into days or hours; a quiet hashtag fits a month. For windows finer than a day, X's operators also take Unix seconds: since_time:1397692800 until_time:1397779200 is April 17, 2014, in UTC.

#What a historical backfill costs

A historical backfill on Sorsa API costs from $0.07 per 1,000 tweets by search and from $0.014 per 1,000 by tweet ID. So 1M historical tweets cost from $70, or from $14 when you already hold the IDs. The official X API bills $5,000 for the same million posts.

At every size, a backfill by search costs about 1/70 of the official X API's price, and a backfill by ID about 1/350.

• 100K

  Sorsa API, by search

  from $7

  Sorsa API, by tweet ID

  from $1.40

  Official X API, posts only

  $500

  X billing cycles at the 3M cap

  1

• 1M

  Sorsa API, by search

  from $70

  Sorsa API, by tweet ID

  from $14

  Official X API, posts only

  $5,000

  X billing cycles at the 3M cap

  1

• 10M

  Sorsa API, by search

  from $700

  Sorsa API, by tweet ID

  from $140

  Official X API, posts only

  $50,000

  X billing cycles at the 3M cap

  4

• 100M

  Sorsa API, by search

  from $7,000

  Sorsa API, by tweet ID

  from $1,400

  Official X API, posts only

  $500,000

  X billing cycles at the 3M cap

  34, or Enterprise

The "from" prices use Sorsa API's lowest rate per request; on the Pro plan, search results cost about $0.10 per 1,000 and tweets by ID about $0.02. Failed requests are never charged, and the author's profile inside each tweet costs nothing extra.

Good to know

From November 22, 2026, the official X API bills every object an expansion returns at its standard rate, as X's developer team announced on October 23 (opens in a new tab). A post read with its author then costs $0.015 instead of $0.005. A historical dataset nearly always needs its authors, so plan for up to $15.00 per 1,000 posts there.

The official X API also stops pay-per-use at 3M post reads per billing cycle. A 10M-tweet backfill takes four billing cycles, and the only way past the cap is an Enterprise contract from about $42,000 a month. Every other official rate is in our X API pricing breakdown.

#Historical tweets on the official X API

The official X API serves historical tweets through full-archive search, GET /2/tweets/search/all, which reaches back to March 2006 and returns up to 500 posts per request. It is open to pay-per-use and Enterprise customers, according to X's search documentation (opens in a new tab). Before the first call you need an approved developer account, a project and an app in the Developer Console, and the app's keys and tokens. Each read is drawn from credits you buy in advance at $0.005 per post, per X's price list (opens in a new tab). Dates go into start_time and end_time as ISO 8601 timestamps.

Measured on what a backfill needs, Sorsa API wins on price, author data, volume and setup, and reaches the same March 2006 start.

• Sorsa API

  Price per 1,000 tweets

  from $0.07 by search, from $0.014 by ID

  Author profile

  In every tweet, no extra charge

  Volume per month

  No post cap, 20 requests per second

  Setup

  One API key

• Official X API

  Price per 1,000 tweets

  $5.00

  Author profile

  $10.00 per 1,000 authors from Nov 22, 2026

  Volume per month

  3M posts, then Enterprise

  Setup

  Developer account, app, OAuth token, prepaid credits

Writing is where the official X API stays in the picture: Sorsa API is read-only by design, so it never posts and never touches your X account. To publish, pair it with the official X API; to read, one key is enough, as the guide to a Twitter API without a developer account explains.

#Who could search the Twitter archive, 2006 to 2026

For most of Twitter's history the full archive sat behind enterprise contracts. Researchers had free access for about two years, and since February 2026 any developer can buy it from X post by post.

The Academic Research track, opened on January 26, 2021, gave approved researchers free full-archive search. When Twitter announced its paid tiers (opens in a new tab) on March 30, 2023, it retired the legacy access levels. Academics were pointed to the general plans while the company looked "at new ways to continue serving this community". Academic access stopped working that summer, and no academic tier has come back. Researchers can request Sorsa API's academic research access from an institutional email. Each grant is tailored to the study, from requests at the volume it needs to a deep discount.

#One account's history, or a published list of tweet IDs

To pull everything one account posted, call POST /user-tweets (opens in a new tab) with its username, user_id or profile link and page through next_cursor from the newest post to the oldest. For one stretch of years, search from: the handle with since: and until: instead. The guide to download all tweets from a user walks through the export to CSV.

Rehydration is fetching the full tweets behind a list of tweet IDs, the form in which most research datasets are shared. POST /tweet-info-bulk (opens in a new tab) takes up to 100 IDs or links per request and returns each tweet with its author, from $0.014 per 1,000, so a 1M-ID dataset costs from $14. IDs of tweets deleted or protected since publication come back missing, so a rehydrated set is always a little smaller than the original. Turning the rest into training data is covered in how to build a Twitter dataset for machine learning.

#Start your backfill with 100 free requests

For historical Twitter data, start with Sorsa API: every public tweet back to 2006 through one API key, at 20 requests per second on every plan. Search costs from $0.07 per 1,000 tweets, and lookups by ID from $0.014 per 1,000. The 100 free requests return about 2,000 tweets from any date window, with no card and no expiry. Pull one month of your topic, check it, then plan the full range.

#FAQ