Pricing

Rating

Developer

Actor stats

Categories

Share

Instagram Profile Scraper

This project scrapes public Instagram profiles using a username or profile URL and extracts complete post-level information in a structured format.

The scraper is designed for data analysis, research, and automation pipelines and outputs clean, flattened data ready for storage or further processing.

✨ Features

• Scrape Instagram profiles using:

  • ✅ Username (e.g. jamshaid_ds)
  • ✅ Profile URL (e.g. https://www.instagram.com/jamshaid_ds)

• Extract profile-level data

• Extract all available post information

• Automatically normalize nested data

• Export results as:

  • CSV
  • JSON
  • Apify Dataset (Actor.push_data)

📌 Extracted Profile Information

For each Instagram profile:

• username
• full_name
• avatar_url
• bio
• website
• followers
• following
• posts_count
• is_private

📸 Extracted Post Information

For each post:

• image_url
• caption
• download_link
• posted_at
• post_type (image / video / reel)

Each row in the final dataset represents one post.

🛠️ Tech Stack

• Python 3.10+
• Pandas
• Async scraping
• Apify Actor SDK (optional)
• CSV / JSON data handling

🚀 Usage

1️⃣ Install Dependencies

$pip install-r requirements.txt

2️⃣ Provide Input

You can pass either:

• Username

jamshaid_ds

• Profile URL

https://www.instagram.com/jamshaid_ds/

The scraper automatically extracts the username from the URL if needed.

3️⃣ Run the Scraper

$python main.py

📤 Output Format

CSV Output

username,full_name,followers,following,image_url,caption,posted_at,post_type

JSON / Apify Dataset Output

{
"username":"@jamshaid_ds",
"full_name":"Jamshaid Arif",
"image_url":"https://...",
"caption":"CSLB SCRAPER...",
"posted_at":"2025-10-22 08:07:48",
"post_type":"image"
}

🔄 Data Normalization Logic

• posts_data is converted from string → dictionary
• Nested post data is flattened using pandas.json_normalize
• Final dataset is post-level, not profile-level

This ensures compatibility with:

• Databases (PostgreSQL, MySQL)
• Analytics pipelines
• Apify datasets

📦 Apify Integration (Optional)

If running as an Apify Actor:

await Actor.push_data(df_final.to_dict(orient="records"))

Each post is pushed as a separate dataset item.

⚠️ Limitations

• Only public profiles are supported
• Private accounts cannot be scraped
• Instagram rate limits may apply
• This tool does not bypass authentication

📄 Disclaimer

This project is for educational and research purposes only. Users are responsible for complying with Instagram’s Terms of Service and applicable laws.