Pricing

Rating

Developer

Actor stats

Categories

Share

Website Crawler - SEO Audit Crawler with Markdown Extraction

Fast, reliable website crawler built for SEO audits and AI/LLM content analysis. Auto-discovers sitemaps, extracts metadata, headings, and LLM-ready markdown content from every page.

BYOK (Bring Your Own Key) -- you provide your own API credentials.

Before You Start

This actor requires your own API credentials to fetch real data.

Where to get your key: Your Firecrawl API key for web crawling. When provided, Firecrawl is used as the primary crawler (faster, handles JS rendering). Falls back to Playwright if not provided or if Firecrawl fails.

You can test with Demo Mode first (free, no key needed) to see the output format before committing.

Quick Start

Test with Demo Mode (free, no API key needed)

{
"demoMode":true,
"startUrls":"https://example.com"
}

Run with real data

{
"demoMode":false,
"startUrls":"https://example.com",
"maxCrawlPages":25,
"maxCrawlDepth":2,
"crawlSitemap":true,
"firecrawlApiKey":"YOUR_API_KEY_HERE",
"useFirecrawl":true
}

Input Parameters

*Required when Demo Mode is off.

Pricing

This actor uses pay-per-event billing:

Demo mode is free -- no charges for sample data.

Troubleshooting

"API key is required"

You have Demo Mode turned off but didn't provide an API key. Either:

• Turn Demo Mode on to test with sample data
• Add your API key in the input

"API error 403" or "Unauthorized"

Your API key is invalid, expired, or doesn't have access to this specific API endpoint. Double-check your key and account permissions.

"API error 429" or "Rate limit"

Too many requests. Wait a minute and try again, or reduce the number of items per run.

No results or empty dataset

Check the run log for error messages. Common causes:

• Invalid input format (check the examples above)
• API key without proper permissions
• The target data doesn't exist or is too small to track

How do I test without an API key?

Enable Demo Mode in the input. This returns realistic sample data so you can verify the output format works for your workflow.

Built by John Rippy | Actor Arsenal