Substack Scraper

Pricing

Rating

Developer

Actor stats

Categories

Share

Scrape any Substack publication via its public RSS feed. Pulls post title, URL, author, publication date, body HTML, categories, cover image. Multi-publication batch supported. HTTP-only with curl_cffi Chrome TLS impersonation. No auth, no proxy.

What this actor does

• Accepts publication URLs in any form: full URL, custom domain, *.substack.com, or bare slug
• Auto-rewrites to <publication>/feed
• Parses RSS feed → extracts title / link / pubDate / dc:creator / content:encoded / categories / enclosure
• Filters: category, published-after, keyword in title/summary
• Optional body HTML inclusion (default on)
• Approximate wordCount and readingTimeMinutes
• Empty fields are omitted

Output per post

• title, url, guid
• author — from <dc:creator>
• publishedAt — ISO 8601 UTC (parsed from RFC 822 pubDate)
• publishedAtRaw — original RFC 822 string
• summary — plain-text version of <description> (capped at 500 chars)
• bodyHtml — full HTML body from <content:encoded> (when includeBody=true)
• wordCount, readingTimeMinutes
• categories[]
• coverImage — from <enclosure> URL
• publication, publicationUrl
• recordType: "post", scrapedAt

Input

Example: scrape Platformer + Noahpinion

{
"publications":["platformer.news","noahpinion.substack.com"],
"publishedAfter":"2024-01-01",
"maxItems":100
}

Example: filter by keyword

{
"publications":["platformer.news"],
"containsKeyword":"antitrust",
"includeBody":true
}

Example: bare slugs (auto-resolved to .substack.com)

{
"publications":["noahpinion","thedailyupside"]
}

Use cases

• Newsletter intel — track competitor publications, harvest content
• Market research — newsletters in your domain (analyst notes, sector reports)
• RSS aggregation — consolidate multiple Substacks into a single feed
• Content analysis — bulk-export newsletter posts for NLP / topic modeling
• Backup — archive your own / a friend's Substack posts

FAQ

Do I need a Substack account? No. The actor only reads public RSS feeds.

Why does it use TLS impersonation? Substack's edge sometimes 403s requests with default Python TLS fingerprint. curl_cffi with chrome131 profile sends a real Chrome handshake, which Substack accepts.

What's the post URL format? https://<publication>/p/<slug>. The actor preserves whatever the RSS feed returns.

Are paid-only posts included? Substack's public RSS includes free posts and the public previews of paid posts. Full paid post content is not accessible without a subscription.

How fresh is the data? Real-time. RSS feeds update within minutes of post publish.

Can I scrape multiple publications in one run? Yes — pass multiple entries in publications. The actor walks each feed sequentially and dedupes by URL.

What if a publication's RSS is blocked / rate-limited? The actor retries with exponential backoff on 403/429/5xx. After 3 retries it skips to the next publication and logs a warning.

Custom-domain Substacks? Yes — pass the custom domain (e.g. platformer.news, stratechery.com). The actor appends /feed regardless of subdomain shape.