Pricing

Rating

Developer

Actor stats

Categories

Share

X (Twitter) Profile Scraper

Extract X (Twitter) profile data with advanced anti-detection measures. Provide your session cookies, enter usernames or URLs, and get structured data instantly.

Important: X/Twitter requires login to view profiles. You must provide your session cookies for this scraper to work.

Features

• Simple Input - Accepts usernames, @handles, full URLs, or mixed formats
• Profile Data - Display name, bio, location, website, join date, followers, following
• Engagement Metrics - Followers count, following count, tweets count, likes count
• Verification Status - Detects blue verified and legacy verified badges
• Media - Profile picture (high-res) and banner image URLs
• Anti-Detection - Browser fingerprinting, stealth headers, popup dismissal
• Graceful Degradation - Returns partial data when full access is blocked

Prerequisites: Getting Your Cookies

X/Twitter now requires authentication to view any profile data. You must provide your session cookies.

Step-by-Step Guide

1. Install Cookie-Editor extension

   • Chrome
   • Firefox

2. Log into X/Twitter

   • Go to x.com and log in with your account

3. Export your cookies

   • Click the Cookie-Editor extension icon
   • Click "Export" button
   • Select "Export as JSON"
   • Copy the entire JSON array

4. Paste into the actor

   • Paste the JSON into the cookies input field

Required Cookies

The scraper validates that you have these essential cookies:

Example Cookie Format

[
{"name":"auth_token","value":"abc123def456...","domain":".x.com","path":"/", ...},
{"name":"ct0","value":"xyz789...","domain":".x.com","path":"/", ...},
 ...
]

Note: Cookie-Editor exports all cookies with the correct format. Just paste the entire output.

Usage

Input Formats

Enter X/Twitter profiles in any format (one per line or comma-separated):

elonmusk
@OpenAI
https://x.com/BillGates
https://twitter.com/satloops
x.com/apaborkar

Input Parameters

Output

Successful Scrape

{
"inputUrl":"https://x.com/elonmusk",
"scrapedUrl":"https://x.com/elonmusk",
"username":"elonmusk",
"displayName":"Elon Musk",
"bio":"The people voted for major change...",
"location":"Earth",
"website":"https://x.com/America",
"joinedDate":"June 2009",
"followers":215000000,
"following":1024,
"tweetsCount":65000,
"likesCount":52000,
"isVerified":false,
"isBlueVerified":true,
"profilePicture":"https://pbs.twimg.com/profile_images/.../photo_400x400.jpg",
"bannerImage":"https://pbs.twimg.com/profile_banners/.../1500x500",
"recentPosts":[
{
"id":"1883847583920193820",
"text":"This is the tweet content...",
"timestamp":"2026-01-26T08:30:00.000Z",
"timeAgo":"2h",
"url":"https://x.com/elonmusk/status/1883847583920193820",
"replies":15420,
"reposts":32100,
"likes":245000,
"views":12500000,
"bookmarks":8200,
"isRepost":false,
"hasImage":true,
"hasVideo":false
}
],
"scrapedAt":"2026-01-26T10:30:00.000Z",
"dataSource":"devwithbobby/x-profile-scraper",
"isPartialData":false
}

Recent Posts Structure

Each post in the recentPosts array contains:

Partial Data (Login Wall)

When X blocks full access, the scraper returns available meta data:

{
"inputUrl":"https://x.com/someuser",
"scrapedUrl":"https://x.com/someuser",
"username":"someuser",
"displayName":"Some User",
"bio":"Bio extracted from meta tags",
"isPartialData":true,
"scrapedAt":"2026-01-26T10:30:00.000Z"
}

Error Cases

{
"inputUrl":"https://x.com/deletedaccount",
"username":"deletedaccount",
"error":"Account not found",
"scrapedAt":"2026-01-26T10:30:00.000Z"
}

Output Fields

Technical Details

Anti-Detection Measures

• Browser Fingerprinting - Realistic Chrome/desktop fingerprints via useFingerprints
• Stealth Headers - Proper Sec-Ch-Ua, Sec-Fetch-* headers matching real Chrome
• WebDriver Masking - Overrides navigator.webdriver and other automation indicators
• Human-like Behavior - Random delays, scrolling, variable viewport sizes
• Popup Dismissal - Automatically closes X login modals and prompts

Block Detection

The scraper detects and handles:

• Login redirects
• Suspended accounts
• Deleted/non-existent accounts
• Rate limiting
• Login wall modals

Limitations

• Authentication required - X requires login to view profiles; cookies must be provided
• Rate limiting - X may block after many requests from the same IP
• Private accounts - Cannot scrape protected/private profiles
• Cookie expiration - Session cookies expire; you may need to re-export periodically

Best Practices

1. Use Residential Proxies for highest success rates (X heavily blocks datacenter IPs)
2. Keep concurrency at 1 to avoid triggering rate limits
3. Refresh cookies weekly - Export fresh cookies if you see authentication failures
4. Space out runs to avoid IP-based blocks
5. Use low request rates (10/min or less recommended)

Proxy Options

Cost Estimation

• Residential proxy: ~$12.50 per 1000 requests (recommended)
• Datacenter proxy: ~$0.25 per 1000 requests (may have lower success)
• Compute: ~$0.15 per 100 profiles

Summary Statistics

After each run, a summary is saved to the key-value store:

{
"totalRequested":10,
"totalScraped":8,
"totalErrors":2,
"averageFollowers":1250000,
"verifiedCount":3,
"proxyUsed":"RESIDENTIAL",
"authenticated":true,
"completedAt":"2026-01-26T10:35:00.000Z"
}

Troubleshooting

"No data could be extracted"

• Check your cookies - Make sure you exported all cookies, not just specific ones
• Refresh cookies - Log out of X, log back in, and export fresh cookies
• Verify auth_token - Ensure the auth_token cookie is present in your export

"Missing required cookie: auth_token"

• You need to be logged into X when exporting cookies
• Make sure you're on x.com (not twitter.com) when exporting

Rate limiting / blocked

• Reduce maxRequestsPerMinute to 5
• Use RESIDENTIAL proxy type
• Wait a few hours before retrying

Support

For issues or feature requests, contact the author or open an issue on the actor's page.