Pricing

Rating

Developer

Actor stats

Categories

Share

Substack Newsletter Scraper 📰

Extract complete data from Substack newsletters including posts, authors, engagement metrics, and full article text.

What does this Actor do? 🎯

This actor scrapes Substack newsletters and extracts 13 data points per post:

Content Data

• Post headline & subheading
• Full article text (or preview for paid posts)
• Post URL

Author Information

• Author name
• Author profile URL

Publishing Data

• Publication date
• Free vs Paid status

Engagement Metrics

• Number of likes ❤️
• Number of comments 💬
• Number of restacks 🔄

Perfect for 💡

• Content Researchers - Analyze trends across newsletters
• Competitive Analysis - Track what competitors publish
• Data Scientists - Build training datasets
• Writers - Research popular topics
• AI Developers - Collect training data

Input

{
"substackUrls":[
"https://tedhope.substack.com",
"https://theankler.com"
],
"scrapingMethod":"sitemap",
"maxPostsPerSubstack":100,
"batchSize":20
}

Output

{
"substack_url":"https://example.substack.com",
"post_url":"https://example.substack.com/p/post-title",
"headline":"Amazing Post Title",
"subheading":"Subtitle here",
"author_name":"John Doe",
"author_url":"https://substack.com/@johndoe",
"date":"December 10, 2024",
"free_or_paid":"Free",
"likes":156,
"comments":23,
"restacks":12,
"article_text":"Full article content...",
"content_type":"full"
}

Features ✨

• Two scraping methods - Sitemap (fast) or Archive (fallback)
• Complete metadata - 13 fields per post
• Engagement metrics - Likes, comments, restacks
• Smart extraction - Handles paywalled content
• Structured output - Clean JSON/CSV format

Pricing 💰

Memory is automatically adjusted based on your input — no manual configuration needed.

Start with Apify's free tier - includes $5 monthly credit!

Tips 💪

1. Test first - Try 2-3 Substacks initially
2. Use sitemap - Faster and more reliable
3. Set limits - Use maxPostsPerSubstack for large newsletters
4. Schedule runs - Set up weekly/monthly automation

FAQ ❓

Q: Can I scrape paywalled content? A: You'll get previews for paid posts, not full content.

Q: How long does it take? A: ~10-30 seconds per Substack with 100 posts.

Q: What if sitemap doesn't work? A: Use "archive" method as fallback.

Q: How much memory does it use? A: Memory is allocated dynamically based on how many URLs you provide — you don't need to configure anything manually. Typical runs use 512MB–1GB.

Changelog 📋

v1.0.4 — April 2, 2026 🔧

Bug Fixes & Performance Improvements

• Fixed: Incomplete scraping results — The scraper was mishandling Substack's nested sitemap indexes, causing runs to stop prematurely after just a few posts. It now correctly parses all sitemap levels and captures 100% of available posts.
• Fixed: Memory misconfiguration — Default memory was incorrectly set to 16GB, making runs unnecessarily expensive. Memory is now dynamically allocated based on your input (scales with number of URLs provided), with a maximum cap of 2GB. Typical single-URL runs now use ~1GB.
• Improved: Cost efficiency — For a standard single-newsletter scrape, costs are now reduced by ~94% compared to previous versions.

v1.0.0 — Initial Release 🚀

• Full post scraping via Sitemap and Archive methods
• 13 data fields per post including engagement metrics
• Batch processing support
• Paid/free post detection

Support 📧

• 📖 Apify Documentation
• 💬 Contact Support

About 👨‍💻

Built by MUSTAPHA LIAICHI - Automation & Web Scraping Specialist

Happy Scraping! 🚀