Pricing

Rating

Developer

Actor stats

Categories

Share

Instagram Post/Reels Scraper

A high-performance Instagram post and reels metadata scraper. Features concurrent scraping, single-session authentication, and comprehensive error handling.

Features

• Single Authentication: Logs in once and reuses the session across all posts
• Concurrent Scraping: Process multiple posts simultaneously for faster execution
• Comprehensive Metadata: Extracts detailed post and reels information
• Error Resilient: Failed posts don't stop the entire scraping process
• Rate Limit Aware: Configurable batch sizes to avoid Instagram rate limits
• Cookie-based Authentication: Secure login without storing passwords

Input Configuration

{
"postUrls":[
"https://www.instagram.com/p/ABC123/",
"https://www.instagram.com/reel/XYZ789/"
],
"cookies":[
{
"name":"sessionid",
"value":"your_session_id",
"domain":".instagram.com",
"path":"/"
},
{
"name":"csrftoken",
"value":"your_csrf_token",
"domain":".instagram.com",
"path":"/"
},
{
"name":"ds_user_id",
"value":"your_user_id",
"domain":".instagram.com",
"path":"/"
}
]
}

Parameters

Supported URL Formats

• Regular posts: https://www.instagram.com/p/{shortcode}/
• Reels: https://www.instagram.com/reel/{shortcode}/
• Reels (alternate): https://www.instagram.com/reels/{shortcode}/
• IGTV: https://www.instagram.com/tv/{shortcode}/

Getting Instagram Cookies

Method 1: Browser Developer Tools

1. Open Instagram in your browser and log in
2. Open Developer Tools (F12)
3. Go to the "Application" or "Storage" tab
4. Click on "Cookies" → "https://www.instagram.com"
5. Copy the following cookies:
   • sessionid
   • csrftoken
   • ds_user_id

Method 2: Browser Extensions

Use cookie export extensions like:

• EditThisCookie (Chrome/Edge)
• Cookie-Editor (Firefox)

Export cookies in JSON format and extract the required values.

Required Cookie Format

{
"name":"sessionid",
"value":"123456789...",
"domain":".instagram.com",
"path":"/",
"expires":1234567890,
"httpOnly":true,
"secure":true
}

Output Data

Output Schema

The scraper returns an array of objects with the following structure:

{
"scraped_at":"2024-12-29T10:36:34.817Z",
"post":{
"shortcode":"DSvAQq8CeH_",
"url":"https://www.instagram.com/p/DSvAQq8CeH_/",
"type":"video",
"caption":"Post caption text...",
"date_posted":"2024-12-28T15:30:00",
"likes":15420,
"comments":342,
"is_video":true,
"video_url":"https://...",
"video_view_count":50000,
"video_duration":30.5,
"thumbnail_url":"https://...",
"media_count":1,
"tagged_users":["user1","user2"],
"location":"New York, NY",
"accessibility_caption":"Image description"
},
"user":{
"username":"example_user",
"user_id":"123456789",
"full_name":"Example User",
"biography":"Bio text...",
"external_url":"https://example.com",
"followers":50000,
"followees":500,
"total_posts":1250,
"is_verified":true,
"is_private":false,
"is_business_account":true,
"profile_pic_url":"https://..."
}
}

Post Metadata Fields

User Metadata Fields

Performance and Limits

Rate Limits

Instagram enforces rate limits to prevent abuse:

• Approximate limit: 200-500 requests per hour per account
• Each post scrape: 2-3 requests (post + user profile)
• Safe limit: 50-100 posts per run
• Conservative limit: 20-30 posts per run

Best Practices

1. Start small: Test with 5-10 posts first
2. Monitor errors: Watch for 403/429 HTTP errors (rate limiting)
3. Use delays: If scraping large datasets, split into multiple runs with 10-15 minute gaps
4. Fresh cookies: Update cookies if you get authentication errors

Error Handling

Common Errors

403 Forbidden

• Cause: Invalid or expired cookies
• Solution: Get fresh cookies from your browser

Invalid URL

• Cause: Unsupported URL format
• Solution: Use supported formats (see "Supported URL Formats")

Connection Timeout

• Cause: Network issues or Instagram downtime
• Solution: Retry after a few minutes

Handling Failed Posts

The scraper continues processing even if some posts fail. Check the logs for:

Failed:2
Failed URLs:['https://instagram.com/p/ABC/','https://instagram.com/p/XYZ/']

Security Considerations

1. Rotate cookies: Update cookies periodically
2. Use secondary accounts: Don't use your main Instagram account for scraping
3. Respect rate limits: Excessive scraping can lead to account bans
4. Follow Instagram's ToS: Use responsibly and for legitimate purposes

License

This project is provided as-is for educational and research purposes. Users are responsible for complying with Instagram's Terms of Service and applicable laws.

Support

For issues and questions:

• Check the logs for detailed error messages
• Review the "Common Errors" section
• Verify your cookies are fresh and valid
• Test with a small dataset first