Pricing

Rating

Developer

Actor stats

Categories

Share

Website Content Crawler

Crawl entire websites — extract all text, headings, metadata & images.

✨ Features

• 🌐 Full website crawler that extracts structured content (text, headings, metadata, links, images) from any domain. Free platform compute pricing.
• Handles pagination for large datasets
• Supports proxy configuration for reliable scraping
• Exports data in JSON, CSV, and Excel formats
• Built on Apify's reliable cloud infrastructure
• Easy to integrate with webhooks and API

🔧 How It Works

1. Configure input — set your search parameters, URLs, or filters
2. Run the actor — it handles all the scraping automatically
3. Get your data — download results in JSON, CSV, or via API

📋 Input Parameters

See the .actor/input_schema.json for full configuration options. Key parameters include:

• Target URLs, search queries, or identifiers
• Pagination limits and filters
• Proxy configuration
• Output format preferences

📊 Output

The actor returns structured data in JSON format. See the .actor/OUTPUT_SCHEMA.json for detailed field descriptions.

💰 Pricing

• Model: Free to use with platform compute pricing
• You only pay for Apify platform compute time — no per-result charges
• $5 free monthly credit for new Apify users
• No subscription required

🚀 Quick Start

# Run via Apify CLI
apify call website-content-crawler
# Or use the API
curl-X POST "https://api.apify.com/v2/acts/flamoqad35tLmtiuD/runs"\
-H"Authorization: Bearer YOUR_API_TOKEN"\
-H"Content-Type: application/json"\
-d'{"input": {}}'

📚 Use Cases

• Market research and competitive analysis
• Lead generation and sales prospecting
• Social media monitoring and brand tracking
• Data-driven decision making
• Academic research and trend analysis

🔗 Integration

This actor can be integrated with:

• Webhooks for real-time data streaming
• Apify API for programmatic access
• Zapier/Make for no-code automation
• Custom pipelines via direct API calls

⚠️ Disclaimer

Use this actor in compliance with the target platform's terms of service and applicable laws. Data scraping should respect robots.txt and rate limits.