Beehiiv Newsletter Discovery Scraper

Pricing

Rating

Developer

Actor stats

Categories

Share

Discover and scrape newsletters from Beehiiv's public directory — the fastest-growing newsletter platform. Browse the full newsletter catalog, get detailed profiles for individual newsletters by URL or subdomain, or extract recent posts and publication history from any Beehiiv newsletter. No login or API key required.

What It Scrapes

Newsletter profiles:

• Name, subdomain, and canonical URL
• Description / tagline
• Avatar / logo image
• Content category (Technology, Business, Finance, etc.)
• Published post count

Post records:

• Title, URL, excerpt/preview text
• Publication date
• Author name
• Parent newsletter name and URL

Supported Modes

Input Parameters

Output Fields

Newsletter Record

Post Record

Example Output

Newsletter record:

{
"name":"TLDR Newsletter",
"subdomain":"tldr",
"url":"https://tldr.beehiiv.com",
"description":"5-minute daily newsletter covering the biggest stories in tech.",
"category":"Technology",
"avatarUrl":"https://cdn.beehiiv.com/tldr-logo.png",
"postCount":1250,
"recordType":"newsletter",
"scrapedAt":"2026-05-17T10:00:00+00:00"
}

Post record:

{
"title":"OpenAI Releases GPT-5",
"url":"https://tldr.beehiiv.com/p/openai-releases-gpt5",
"publishedAt":"2026-04-01T08:00:00+00:00",
"excerpt":"OpenAI announced GPT-5, its most capable model yet, featuring...",
"author":"Dan Ni",
"newsletterName":"TLDR Newsletter",
"newsletterUrl":"https://tldr.beehiiv.com",
"recordType":"post",
"scrapedAt":"2026-05-17T10:00:00+00:00"
}

Frequently Asked Questions

Do I need a Beehiiv account or API key? No. This scraper uses Beehiiv's public discover page and publicly accessible newsletter pages. No authentication is required.

What newsletter categories are available? Technology, Business, Finance, Marketing, Health, Sports, Entertainment, Politics, Science, Food, Travel, Lifestyle, Education, and Other.

How do I scrape a specific newsletter? Use mode=getNewsletter and provide the newsletter URL in startUrls (e.g. https://milkroad.beehiiv.com), or just the subdomain in subdomains (e.g. milkroad).

Can I get all posts from a newsletter? Use mode=searchPosts with the newsletter URL or subdomain. Posts are discovered via the newsletter's sitemap.xml and post listing page. Set maxPostsPerNewsletter up to 100.

Does this work for newsletters on custom domains? Yes — if you know the full URL of a newsletter on a custom domain, provide it in startUrls. The scraper will fetch the page and extract metadata.

Why might some newsletters not have a description or post count? Not all newsletter homepages expose this information in their HTML meta tags. When a field cannot be reliably populated, it is omitted from the output rather than filled with a placeholder.

Limitations

• Beehiiv's discover page is Cloudflare-protected; the scraper uses curl_cffi with Chrome impersonation to bypass this.
• Newsletter subscriber counts are not publicly exposed — Beehiiv does not display them without login.
• Post content (full article body) is not scraped — only title, excerpt, URL, and date are collected.
• The discover page structure can change; the scraper uses multiple fallback strategies (Next.js data, inline JSON, href extraction) for maximum resilience.
• Rate limits may apply during heavy scraping; the scraper includes built-in retry logic and delays between requests.