Pricing
$11.99/month + usage
Go to Apify Store
Example: Post (
Example: Comment (
Post fields (
Comment fields (
Reddit Scraper | All-In-One
All-in-one Reddit Scraper. Scrape posts and full comment threads from any search, subreddit, user, or direct post URL. This enterprise-grade scraper is the fastest in the market and delivers clean and detailed JSON.
Pricing
$11.99/month + usage
Rating
5.0
(1)
Developer
Actor stats
5
Bookmarked
51
Total users
7
Monthly active users
1.1 days
Issues response
10 hours ago
Last modified
Categories
Share
Historical Reddit Archive Search
Slug: fatihtahta/reddit-scraper-search-fast
Overview
Historical Reddit Archive Search collects matching submissions from the stored Reddit archive. The actor uses the indexed archive path only; it does not live-crawl Reddit, visit direct Reddit URLs, or require exact dates before a keyword search can run. Each output record includes core content, authorship, engagement metrics, subreddit context, timestamps, canonical URLs, and archive provenance fields when available.
Archive Runtime
The archive storage has two layers: an OpenIndex Parquet base and a raw-backfill overlay for missing or repaired months. The runtime search path uses the CleanedWeb archive broker and B2-backed SQLite FTS indexes before reading archive rows.
Keyword archive search is index-first: prune by date, subreddit, shard metadata, token filters, and candidate IDs, then return matching archive records. Missing dateFrom, dateTo, or subredditName values are not startup blockers.
Why Use This Actor
- Market research and analytics teams: Track conversation volume, engagement, subreddit activity, and topic trends across keywords, communities, and time windows.
- Product and content teams: Discover user pain points, feature requests, language patterns, and high-performing discussion themes for roadmap and editorial planning.
- Developers and data engineering teams: Feed Reddit data into ETL pipelines, warehouses, dashboards, and APIs using structured JSON records that are easy to upsert and model.
- Lead generation and enrichment teams: Identify relevant communities, active discussions, and context around buyer interests, brand mentions, and niche topics.
- Monitoring and competitive intelligence teams: Watch competitor mentions, category shifts, launch reactions, and recurring discussion spikes without manually checking Reddit every day.
- Operations and automation teams: Run recurring jobs on a schedule and use stable record keys to deduplicate overlapping results from queries, subreddit searches, and direct URLs.
Input Parameters
Provide search phrases and optional filters to control what the archive search returns.
| Parameter | Type | Description | Default |
|---|---|---|---|
queries | string[] | Search phrases to look for in archived Reddit submissions. | – |
subredditName | string | Optional subreddit filter, without the r/ prefix. | – |
timeframe | string | Optional relative date window. Allowed values: all, year, month, week, day, hour. Ignored when exact dates are provided. | all |
dateFrom | string | Optional lower date bound for posts. Accepts YYYY-MM-DD. | – |
dateTo | string | Optional upper date bound for posts. Accepts YYYY-MM-DD. | – |
maxPosts | integer | Maximum archived submissions to save. | 100 |
Example Inputs
Scenario: historical archive search
Historical archive searches use the stored index automatically. Exact dates and subreddit filters are optional.
Scenario: filtered archive search
Output
Output destination
The actor writes results to an Apify dataset as JSON records. And the dataset is designed for direct consumption by analytics tools, ETL pipelines, and downstream APIs without post-processing.
Record envelope (all items)
Every record includes a stable category field, a Reddit identifier, and a canonical URL:
- type (string, required): Logical record type, such as
postorcomment. In the JSON output examples below, this category is represented by thekindfield. - id (string, required): Stable Reddit identifier for the entity.
- url (string, required): Canonical Reddit URL for the record.
Recommended idempotency key: type + ":" + id
Use this key for deduplication and upserts, especially when the same Reddit entity appears in overlapping queries, subreddit runs, or direct URL inputs.
Examples
Example: Post (type = "post")
Example: Comment (type = "comment")
Field reference
Post fields (type = "post")
- kind (string, required): Record category for post records.
- query (string, optional): Input query or source label that produced the record.
- id (string, required): Stable Reddit post identifier.
- title (string, required): Post title.
- body (string, optional): Post body text.
- sentiment_score (number, optional): Raw sentiment score when supplied by an internal archive enrichment path.
- sentiment_score_normalized (number, optional): Length-normalized bounded sentiment score when supplied by an internal archive enrichment path.
- sentiment_confidence (number, optional): Heuristic confidence score for the sentiment result when supplied.
- sentiment_label (string, optional): Sentiment label when supplied.
- content_category_label (string, optional): Readable content category name when supplied by an internal archive enrichment path.
- content_category_path (array, optional): Topic path from the top-level content category down to the matched post category when supplied.
- author (string, optional): Username shown on the post.
- score (number, optional): Post score at collection time.
- upvote_ratio (number, optional): Upvote ratio when available.
- num_comments (number, optional): Comment count shown on the post.
- subreddit (string, optional): Subreddit name.
- created_utc (string, optional): Post creation time in ISO format. This is the timestamp used for exact post date filtering when
dateFromordateTois provided. - url (string, required): Canonical Reddit URL for the post.
- permalink (string, optional): Relative Reddit permalink.
- canonical_url (string, optional): Canonical full URL.
- old_reddit_url (string, optional): Alternate legacy Reddit URL.
- flair (string, optional): Post flair text.
- post_hint (string, optional): Reddit post hint, useful for downstream media classification.
- over_18 (boolean, optional): Whether the post is marked NSFW.
- is_self (boolean, optional): Whether the post is a self post.
- spoiler (boolean, optional): Whether the post is marked as a spoiler.
- locked (boolean, optional): Whether the post is locked.
- is_video (boolean, optional): Whether the post is a video post.
- is_gallery (boolean, optional): Whether the post is a Reddit gallery.
- hidden (boolean, optional): Whether the post is hidden for the viewing account.
- edited (boolean | number, optional):
falsewhen untouched, otherwise Reddit's edited timestamp payload. - archived (boolean, optional): Whether the post is archived.
- pinned (boolean, optional): Whether the post is pinned in the subreddit.
- domain (string, optional): Source or linked domain.
- thumbnail (string, optional): Thumbnail URL or Reddit thumbnail marker.
- url_overridden_by_dest (string, optional): Final outbound destination URL when present.
- num_duplicates (number, optional): Duplicate count reported by Reddit.
- subreddit_id (string, optional): Internal Reddit subreddit reference.
- subreddit_name_prefixed (string, optional): Prefixed subreddit label such as
r/Baking. - subreddit_subscribers (number, optional): Subscriber count at collection time.
- media (object, optional): Media object when available.
- media_metadata (object, optional): Raw media metadata keyed by media ID.
- media_metadata.<media_id>.status (string, optional): Media validity state.
- media_metadata.<media_id>.e (string, optional): Media asset type label.
- media_metadata.<media_id>.m (string, optional): Media MIME type.
- media_metadata.<media_id>.p (array, optional): Preview image variants.
- media_metadata.<media_id>.p[].y (number, optional): Preview height.
- media_metadata.<media_id>.p[].x (number, optional): Preview width.
- media_metadata.<media_id>.p[].u (string, optional): Preview URL.
- media_metadata.<media_id>.s.y (number, optional): Original media height.
- media_metadata.<media_id>.s.x (number, optional): Original media width.
- media_metadata.<media_id>.s.u (string, optional): Original media URL.
- media_metadata.<media_id>.id (string, optional): Media asset identifier.
- gallery_data (object, optional): Reddit gallery metadata.
- gallery_data.items (array, optional): Gallery item list.
- gallery_data.items[].is_deleted (boolean, optional): Whether the gallery item is deleted.
- gallery_data.items[].media_id (string, optional): Gallery media identifier.
- gallery_data.items[].id (number, optional): Gallery item identifier.
- gallery_images (array, optional): Normalized gallery image list.
- gallery_images[].media_id (string, optional): Gallery media identifier.
- gallery_images[].caption (string, optional): Image caption text.
- gallery_images[].width (number, optional): Image width.
- gallery_images[].height (number, optional): Image height.
- gallery_images[].url (string, optional): Original image URL.
- gallery_images[].previews (array, optional): Preview image URLs.
- media_assets (array, optional): Normalized media asset list.
- media_assets[].type (string, optional): Media type label.
- media_assets[].media_id (string, optional): Media asset identifier.
- media_assets[].mime_type (string, optional): Media MIME type.
- media_assets[].original_url (string, optional): Original media URL.
- media_assets[].preview_urls (array, optional): Preview URLs for the asset.
- age_hours (number, optional): Post age in hours at collection time.
- retrieved_at (string, optional): Actor capture time in ISO format.
- media_type (string, optional): Normalized media class:
text,image,gallery,video,gif, orlink. - has_media (boolean, optional): Convenience flag for image, gallery, GIF, or video posts.
- gallery_count (number, optional): Number of normalized gallery images.
- outbound_url_host (string, optional): Parsed host from
url_overridden_by_destwhen present. - title_length (number, optional): Character length of the title.
- body_length (number, optional): Character length of the body text.
- word_count (number, optional): Lightweight whitespace-based word count across title and body.
- score_per_hour (number, optional): Score divided by post age with a minimum age floor to avoid division by zero.
- comments_per_hour (number, optional): Comment count divided by post age with a minimum age floor to avoid division by zero.
- is_deleted_or_removed (boolean, optional): Conservative deletion/removal flag derived from visible placeholders and removal metadata.
- engagement_total (number, optional): Combined engagement metric derived from score and comments.
- comment_to_score_ratio (number, optional): Comments-to-score ratio.
- is_high_engagement (boolean, optional): Convenience flag for high engagement.
- content_flags (array, optional): Content classification flags when present.
- stickied (boolean, optional): Whether the post is pinned.
- distinguished (string, optional): Distinguishing label, such as moderator status.
- total_awards_received (number, optional): Total awards on the post.
- all_awardings (array, optional): Raw awards list.
- gilded (number, optional): Gilding count.
- num_crossposts (number, optional): Number of crossposts.
- is_original_content (boolean, optional): Whether the post is marked original content.
- author_fullname (string, optional): Internal Reddit author reference when available.
- author_flair_text (string, optional): Author flair text.
- author_premium (boolean, optional): Whether the author has premium status.
- body_html (string, optional): HTML-formatted post body.
- preview (object, optional): Preview object when available.
- secure_media (object, optional): Secure media object when available.
- secure_media_embed (object, optional): Secure media embed metadata.
- crosspost_parent_list (array, optional): Crosspost parent data when available.
Comment fields (type = "comment")
- kind (string, required): Record category for comment records.
- query (string, optional): Input query or source label that produced the record.
- id (string, required): Stable Reddit comment identifier.
- postId (string, required): Parent post identifier.
- postUrl (string, required): Parent post URL.
- parentId (string, required): Parent Reddit object identifier.
- body (string, optional): Comment body text.
- sentiment_score (number, optional): Raw sentiment score when supplied by an internal archive enrichment path.
- sentiment_score_normalized (number, optional): Length-normalized bounded sentiment score when supplied.
- sentiment_confidence (number, optional): Heuristic confidence score for the sentiment result when supplied.
- sentiment_label (string, optional): Sentiment label when supplied.
- author (string, optional): Username shown on the comment.
- score (number, optional): Comment score at collection time.
- created_utc (string, optional): Comment creation time in ISO format when comment records are returned by an archive path.
- url (string, required): Canonical Reddit URL for the comment.
- permalink (string, optional): Relative Reddit permalink.
- canonical_url (string, optional): Canonical full URL.
- old_reddit_url (string, optional): Alternate legacy Reddit URL.
- root_comment_id (string, optional): Root comment ID for the thread.
- parent_kind (string, optional): Parent record type, such as
postorcomment. - comment_permalink (string, optional): Relative permalink for the comment.
- author_deleted (boolean, optional): Whether the author is deleted.
- body_deleted (boolean, optional): Whether the comment body is deleted or removed.
- stickied (boolean, optional): Whether the comment is pinned.
- distinguished (string, optional): Distinguishing label, such as moderator status.
- is_submitter (boolean, optional): Whether the author is the original post creator.
- score_hidden (boolean, optional): Whether the score is hidden.
- controversiality (number, optional): Reddit controversiality indicator.
- depth (number, optional): Nesting depth in the comment tree.
Data guarantees & handling
- Best-effort extraction: fields may vary by archive coverage, source availability, and historical record shape.
- Optional fields: always null-check in downstream code because many fields may be empty or unavailable.
- Historical archive search: keyword searches are backed by stored B2 archive indexes, not a local sample baseline or live Reddit crawling. The archive path uses index/filter pruning before returning records.
- Time filtering:
timeframeselects a relative archive window when exact dates are not provided.dateFromanddateToare optional exact bounds. - Deduplication: recommend
type + ":" + id. - Stable identifiers make inter-seed deduplication and upserts straightforward when the same entity is discovered through overlapping inputs.
How to Run on Apify
- Open the Actor in Apify Console.
- Configure your archive search phrases and optional filters.
- Set the maximum number of outputs to collect using
maxPosts. - Click Start and wait for the run to finish.
- Download results in JSON, CSV, Excel, or other supported formats.
Scheduling & Automation
Scheduling
Automated Data Collection
You can schedule recurring runs to keep your Reddit dataset current without manual work. This is useful for monitoring trends, tracking brand mentions, and maintaining fresh inputs for dashboards or data pipelines.
- Navigate to Schedules in Apify Console
- Create a new schedule (daily, weekly, or custom cron)
- Configure input parameters
- Enable notifications for run completion
- Optional: add webhooks for automated processing
Integration Options
- Webhooks: Trigger downstream actions when a run completes
- Zapier: Connect to 5,000+ apps without coding
- Make (Integromat): Build multi-step automation workflows
- Google Sheets: Export results to a spreadsheet
- Slack/Discord: Receive notifications and summaries
- Email: Send automated reports via email
Performance
Historical archive performance depends more on search selectivity than output count. Indexed archive searches can be fast even with large date ranges, while unindexed broad text scans can be expensive even when they return zero records. The archive implementation should report searched months, scanned bytes or equivalent search work, returned records, and stop reason in run summaries.
Compliance & Ethics
Responsible Data Collection
This actor returns publicly available historical Reddit submissions and metadata for legitimate business purposes, including:
- consumer research and market analysis
- brand monitoring and competitive tracking
- product feedback discovery and trend analysis
Users are responsible for making sure their use of the collected data complies with applicable laws, regulations, internal policies, and the target site's terms. This section is informational and not legal advice.
Best Practices
- Use collected data in accordance with applicable laws, regulations, and the target site's terms
- Respect individual privacy and personal information
- Use data responsibly and avoid disruptive or excessive collection
- Do not use this actor for spamming, harassment, or other harmful purposes
- Follow relevant data protection requirements where applicable, such as GDPR and CCPA
Support
For help, use the Issues tab or the actor page on Apify. Include the input you used with sensitive values redacted, the run ID, the expected behavior versus the actual behavior, and, if helpful, a small output sample.
