X / Twitter Post Scraper

Pricing

Rating

Developer

Actor stats

Categories

Share

Collect structured data from public X (Twitter) posts using direct post URLs. Paste one or more status links — from x.com or twitter.com — and receive clean JSON records in your Apify dataset.

Use it for social listening, brand monitoring, research, content archiving, competitive analysis, and analytics pipelines — without copying posts manually from the site.

What you get

Each saved row is one post, organized into clear sections:

Results stream to the dataset as posts are collected, so you can preview progress before the run finishes.

Quick start on Apify

1. Open the Actor in the Apify Console.
2. Add one or more Post URLs — public X status links such as https://x.com/username/status/1234567890.
3. Choose output options:
   • Flatten output (default) — clean, normalized JSON for most use cases.
   • Include author profile — attach the author's public profile to each post (each unique handle is fetched once per run).
4. Click Start and open the Dataset tab when the run finishes.

Example input:

{
"postUrls":[
{"url":"https://x.com/elonmusk/status/2067438979667140620"},
{"url":"https://x.com/hackernoon/status/2066573502832906644"}
],
"flattenOutput":true,
"includeAuthorProfile":true
}

Supported URL formats:

• https://x.com/{username}/status/{post_id}
• https://twitter.com/{username}/status/{post_id}

Input parameters

Post URLs

All URLs in the list are processed in parallel. Invalid URLs are saved as error rows with a message; other URLs in the same run can still succeed.

Flatten output

When enabled, each result is a compact, user-friendly record with normalized field names (author, metrics, entities, and related sections).

When disabled, each result includes additional detail — top-level engagement fields, grouped media with video variants, link card data, and a richer author block — while staying readable (no raw technical payloads).

Include author profile

When enabled, the Actor fetches public profile data for each post author and adds an authorProfile object to the result.

• Profile data includes bio, follower/following/post counts, verification, avatar, banner, and profile links when available.
• If multiple input URLs share the same author, that profile is fetched once per run and reused.

What you will see during a run

The run log is written for end users — progress messages describe collection activity only.

Typical messages:

• Starting scrape for N post URL(s) in parallel.
• Progress: 25/100 post(s) collected. (periodic updates on large runs)
• Run finished. Saved N post(s) to the dataset.

If a post cannot be processed, you will see a warning for that URL; other URLs in the same run can still succeed. Failed URLs are saved to the dataset with an error field.

On the Apify free plan, you may also see a short notice when free-tier limits apply (see Free plan limits).

Output overview

Dataset structure

• Results are saved to a single Apify dataset
• Each row is one post object (or one error object for a failed URL)
• With Include author profile enabled, profile data is nested under authorProfile on each post row

Field presence varies by post: not every post includes media, link previews, view counts, or full profile details. Empty arrays ([]) or null mean the data was not published for that post — not necessarily an error.

Output modes

Output fields (flattened mode)

Top-level post

author

metrics

entities

media

Array of media items (photos, videos, GIFs) with type, URLs, dimensions, duration, and alt text when available.

link_preview

Present on link-card posts: title, description, domain, URL, and preview image.

context

authorProfile (when enabled)

Example output

The excerpts below come from a real run with Flatten output and Include author profile enabled. Full output is available in the run dataset.

Example input

{
"postUrls":[
{"url":"https://x.com/hackernoon/status/2066573502832906644"}
],
"flattenOutput":true,
"includeAuthorProfile":true
}

Excerpt — link post with card preview and profile

{
"id":"2066573502832906644",
"url":"https://x.com/hackernoon/status/2066573502832906644",
"post_type":"link",
"text":"Some engineering teams fix 90% of software issues before customers ever notice. \n\nLearn how AI is changing debugging without replacing developers: https://t.co/rEOC2CXJz7",
"created_at":"Mon Jun 15 17:28:12 +0000 2026",
"author":{
"username":"hackernoon",
"name":"HackerNoon | Learn Any Technology",
"verified":true,
"verified_type":"Business"
},
"metrics":{
"likes":221,
"retweets":23,
"replies":4,
"bookmarks":162,
"views":"734692"
},
"entities":{
"urls":[
{
"short":"https://t.co/rEOC2CXJz7",
"expanded":"https://hackernoon.com/the-technical-infrastructure-of-automated-debugging",
"display":"hackernoon.com/the-technical-…"
}
]
},
"link_preview":{
"title":"The Technical Infrastructure of Automated Debugging | HackerNoon",
"description":"PlayerZero combines AI, telemetry, and system modeling to accelerate root cause analysis and help engineering teams debug faster.",
"domain":"hackernoon.com",
"url":"https://t.co/rEOC2CXJz7",
"image_url":"https://pbs.twimg.com/card_img/2066562404574531584/4F9VBmbS?format=jpg&name=orig"
},
"authorProfile":{
"username":"hackernoon",
"name":"HackerNoon | Learn Any Technology",
"bio":"how hackers start their afternoons. where 50k+ technologists publish blog posts for 4M+ monthly readers. write your story 👉https://t.co/PGmtSCSd5V",
"counts":{
"followers":92589,
"following":5448,
"posts":135804
},
"verified":true,
"verified_type":"Business"
}
}

Excerpt — video post

{
"id":"2067438979667140620",
"url":"https://x.com/elonmusk/status/2067438979667140620",
"post_type":"media",
"text":"https://t.co/ujL9EUqFK4",
"author":{
"username":"elonmusk",
"name":"Elon Musk",
"verified":true
},
"metrics":{
"likes":104664,
"retweets":11595,
"replies":5069,
"views":"26356947"
},
"media":[
{
"type":"video",
"url":"https://pbs.twimg.com/ext_tw_video_thumb/1848648724500672512/pu/img/NMjczVOjehsLLTWy.jpg",
"width":1280,
"height":720,
"duration_ms":60821
}
]
}

Video-only posts may show a short link as the post text; media details are under media.

Scraping many URLs

You can add hundreds or thousands of post URLs in one run.

• URLs are processed in parallel for faster collection.
• Progress is logged at intervals (for example every 25 posts) instead of one line per post.
• Duplicate authors with Include author profile enabled still trigger only one profile fetch per unique handle per run.

Tip: Start with a small batch (5–10 URLs) to validate output shape before a large export.

Free plan limits

On the Apify free plan, the Actor may automatically apply:

Paid Apify plans can use the full input without this cap. If limits apply, the run log will include a short notice.

Pricing

This Actor uses pay-per-event billing on the Apify Store:

See the Pricing tab on the Store listing for current rates and plan discounts.

Posts saved with an error field (failed URLs) are still written to the dataset; check the Pricing tab for how failed items are billed.

Data quality and limitations

Public post data only
The Actor reads data shown on public X post and profile pages. It does not access private accounts, direct messages, or content that requires login beyond what X shows to a guest visitor.

Coverage varies by post
View counts, link previews, media metadata, and profile fields are not always published. Some fields may be null or empty even when the post is live.

Post text
Media-heavy posts sometimes contain only a short link in the text field; full media details appear under media.

Profile links in bio
Expanded destination URLs for t.co links in bios may not always be available.

Live data changes
Engagement counts and profile stats change over time. Re-run on a schedule to refresh data.

Scope
This Actor scrapes individual post URLs only. Profile timelines, search, followers, and hashtag pagination are not included.

Compliance
You are responsible for using collected data in line with applicable laws and X's terms of service.

Tips for best results

1. Use direct status URLs — Copy the link from the post's share menu or browser address bar.
2. Enable profiles when you need bio and counts — Turn on Include author profile for enrichment; leave it off for faster, lower-cost post-only runs.
3. Keep flatten on for most workflows — Use the default flattened output for CSV exports and APIs; disable it when you need video variants or extra card fields.
4. Test with a few URLs first — Confirm field coverage before large batch runs.
5. Schedule recurring runs — Monitor key accounts or campaigns with Apify schedules.
6. Handle errors in the dataset — Filter rows where success is false or an error field is present.
7. Deduplicate by post ID — Use the id field if the same URL appears more than once in your input list.

Frequently asked questions

Why did my post URL fail?
The post may have been deleted, made private, or temporarily unavailable. Check the run log and the dataset row for the error message.

Why is view count a string?
X publishes view counts as formatted strings (for example "734692"). This matches what appears in the source data.

Why is post text only a t.co link?
Some posts — especially video-only posts — contain little or no caption text. Check the media section for the full content.

Why is the same profile attached to multiple posts?
When Include author profile is enabled, each unique author is resolved once per run and reused for all their posts.

Does this scrape a user's full timeline?
No. Provide specific post URLs. Timeline and search scraping are out of scope for this Actor.

Can I use twitter.com links instead of x.com?
Yes. Both domains are supported.

What happens if I submit duplicate post URLs?
Each URL is processed independently. You may receive duplicate rows unless you deduplicate by id afterward.

Support

For Actor-specific issues, use the Issues tab on the Apify Store listing or contact the publisher through Apify.

For platform questions (runs, billing, API, schedules), see Apify documentation and Apify support.