Pricing

Rating

Developer

Actor stats

Categories

Share

Hacker News Scraper - Stories, Comments, Scores & Search

Scrape Hacker News stories, comments, scores, and search results into clean, structured data. This Hacker News scraper collects top, new, best, Ask HN, Show HN, and jobs feeds, runs full-text keyword search, fetches specific item IDs, and optionally nests comment threads. Export to JSON, CSV, Excel, or HTML, or pull via the Apify API. No login and no API key required.

Built with Node.js 20, TypeScript, and the Apify SDK. It uses the official Hacker News Firebase API for feeds and item details and the public HN Algolia Search API for keyword search, with retries and bounded concurrency so runs are reliable and repeatable. No browser and no proxy are required.

What It Extracts

• id - numeric Hacker News item ID
• type - item type (story, comment, job, poll)
• title - story or item title
• text and textHtml - item body as plain text and HTML
• url - external link, when present
• hnUrl - the Hacker News discussion URL
• domain - hostname of the external link
• author - HN username
• score - points
• commentCount - number of comments
• parentId - parent item ID for comments
• pollId and pollParts - poll references
• createdAt and createdAtUnix - publication time (ISO and Unix)
• rank - position in the feed
• feed - source feed (top, new, best, ask, show, jobs)
• query - the search query that matched the record
• dead and deleted - moderation flags
• comments - optional nested comment threads (id, author, text, textHtml, score, createdAt, createdAtUnix, parentId, depth, dead, deleted, hnUrl)
• collectedAt - scrape timestamp

Use Cases

1. Monitor developer, startup, and technology trends across the top, new, and best feeds.
2. Track product, brand, and competitor mentions with keyword and domain filters.
3. Discover Show HN launches and emerging tools as they appear.
4. Analyze Ask HN discussions and developer sentiment with nested comments.
5. Collect Hacker News jobs feed data for recruiting and hiring research.
6. Build alerts, dashboards, and datasets for AI and research pipelines.

Pricing

This Actor uses Apify Pay Per Event pricing. You pay only for clean records delivered to the dataset. Empty, filtered, or failed records are not charged.

Input

Example input

{
"mode":"feed",
"feed":"show",
"maxResults":25,
"minScore":10,
"includeKeywords":["AI","developer"],
"includeComments":true,
"maxCommentsPerItem":20,
"commentDepth":2
}

How to Scrape Hacker News (Step by Step)

1. Click Try for free / Run.
2. Pick a mode: a feed (top, new, best, ask, show, jobs), keyword search, or specific itemIds.
3. Add filters such as minScore, includeKeywords, authors, domain, or a date range to narrow results.
4. Set maxResults (start small to test) and toggle includeComments if you want nested threads.
5. Run the Actor, then export results as JSON, CSV, Excel, or HTML, or pull them via the Apify API.

Sample Output

{
"id":48487029,
"type":"story",
"title":"Show HN: An open-source AI agent for developers",
"text":null,
"textHtml":null,
"url":"https://github.com/example/ai-agent",
"hnUrl":"https://news.ycombinator.com/item?id=48487029",
"domain":"github.com",
"author":"devbuilder",
"score":287,
"commentCount":94,
"parentId":null,
"pollId":null,
"pollParts":[],
"createdAt":"2026-06-11T05:22:06.000Z",
"createdAtUnix":1781155326,
"rank":1,
"feed":"show",
"query":null,
"dead":false,
"deleted":false,
"comments":[
{
"id":48487102,
"author":"curious_hacker",
"text":"This is great, how does it handle rate limits?",
"textHtml":"<p>This is great, how does it handle rate limits?</p>",
"score":0,
"createdAt":"2026-06-11T05:41:12.000Z",
"createdAtUnix":1781156472,
"parentId":48487029,
"depth":1,
"dead":false,
"deleted":false,
"hnUrl":"https://news.ycombinator.com/item?id=48487102"
}
],
"collectedAt":"2026-06-11T06:00:00.000Z"
}

How It Works

1. Validates the input and selects the collection mode (feed, search, or items).
2. Fetches feed and item details from the official Hacker News Firebase API, and keyword results from the public HN Algolia Search API.
3. Applies score, comment, keyword, author, domain, and date filters.
4. Optionally fetches and nests comment threads up to your depth and count limits.
5. Charges item-scraped only after a clean record is saved, then writes it to the Apify Dataset.

Known Limits

• text and textHtml are only present for items that have body content; link stories return null for these fields.
• comments are only populated when includeComments is enabled, and are bounded by maxCommentsPerItem and commentDepth.
• Keyword search uses the HN Algolia API, so results and ranking follow that service's coverage and indexing.
• Dead or deleted items are excluded unless includeDeadOrDeleted is enabled.
• maxResults is capped at 1,000 records per run.

Data Sources

This Actor uses the official Hacker News Firebase API and the public HN Algolia Search API. It does not rely on fragile page selectors.

License

Apache-2.0.