Pricing

Rating

Developer

Actor stats

Categories

Share

Hacker News Scraper – Stories, Comments, Polls & User Profiles (No API Key)

Scrape Hacker News at scale without an API key or login. This Hacker News scraper runs full-text searches, pulls stories, flattens entire comment trees, collects polls, and fetches user profiles, then lets you export to JSON, CSV, or Excel. Use it as a no-code Hacker News API alternative for data, research, and monitoring.

Built on Hacker News' two official, key-free JSON APIs (the Algolia HN Search API and the Firebase HN API). Made for developers, researchers, data teams, and trend watchers who want clean, structured HN data without parsing HTML or hitting rate limits.

Pairs well with the Reddit Scraper and Google News Scraper for end-to-end tech and news monitoring across communities.

What it does

• 🔎 Full-text search – query HN via Algolia by relevance or newest-first, filtered by tags (story, comment, poll, Ask HN, Show HN, front page) and author.
• 📰 Built-in lists – grab the official Top, New, Best, Ask HN, Show HN, or Jobs front-page lists.
• 🆔 Direct items by ID – fetch any story, poll, or comment by its Hacker News item ID.
• 💬 Comment trees – optionally pull and flatten the full nested comment tree for every story/poll, with per-comment depth and parentId.
• 👤 User profiles – karma, about text, account creation date, and submission count by username.
• 🎯 Filtering – minimum points, minimum comments, and start/end date windows.
• 🧹 Clean text – comment, story, and bio HTML is stripped and entity-decoded to plain UTF-8 text (no ' mojibake).
• 📤 Export anywhere – download results as JSON, CSV, or Excel, or pull them from the Apify API.

Why this scraper

• No API key, no login. Uses Hacker News' official, public Algolia Search and Firebase endpoints — no credentials, no account, no cookies.
• Fast and reliable. These are open JSON APIs, so there's no anti-bot or HTML markup to fight; runs are quick and stable.
• One pass, fully populated. Algolia search hits already carry the core story fields, so search results come back complete without extra per-item requests.
• Goes past Algolia's 1000-result cap. With sortBy: "date" and a startDate, the Actor automatically pages backward through creation-date windows to fetch far more than 1000 results per query.
• Resilient HTTP. Automatic retries with exponential backoff on 429/5xx/network errors.

Example input

{
"searchQueries":["large language models"],
"tags":"story",
"sortBy":"date",
"startDate":"2024-01-01",
"endDate":"2024-12-31",
"minPoints":50,
"minComments":10,
"includeComments":true,
"maxCommentDepth":3,
"storyList":"top",
"itemIds":["8863"],
"usernames":["pg"],
"maxItems":200,
"maxItemsPerQuery":100,
"proxyConfiguration":{"useApifyProxy":false}
}

Provide any combination of sources — search queries, item IDs, usernames, and/or a built-in list. Leave a field empty to skip it.

Example output

Every item carries a type field (story, comment, poll, or user) so you can split the dataset by output kind. Every item also carries a scrapedAt ISO 8601 timestamp.

Story (type: "story")

{
"type":"story",
"id":8863,
"objectID":"8863",
"title":"My YC app: Dropbox - Throw away your USB drive",
"url":"http://www.getdropbox.com/u/2/screencast.html",
"author":"dhouston",
"points":111,
"numComments":71,
"createdAt":"2007-04-04T19:16:40.000Z",
"createdAtTimestamp":1175714200,
"text":null,
"tags":["story","author_dhouston","story_8863"],
"isAskHn":false,
"isShowHn":false,
"hnUrl":"https://news.ycombinator.com/item?id=8863",
"scrapedAt":"2026-06-26T12:00:00.000Z"
}

Comment (type: "comment")

{
"type":"comment",
"id":8952,
"objectID":"8952",
"storyId":8863,
"storyTitle":"My YC app: Dropbox - Throw away your USB drive",
"storyUrl":"http://www.getdropbox.com/u/2/screencast.html",
"parentId":8863,
"author":"BrandonM",
"text":"I have a few qualms with this app...",
"createdAt":"2007-04-04T20:12:00.000Z",
"createdAtTimestamp":1175717520,
"depth":1,
"hnUrl":"https://news.ycombinator.com/item?id=8952",
"scrapedAt":"2026-06-26T12:00:00.000Z"
}

Poll (type: "poll")

{
"type":"poll",
"id":126809,
"objectID":"126809",
"title":"Poll: What's your favorite programming language?",
"url":null,
"author":"pg",
"points":99,
"numComments":116,
"createdAt":"2008-04-29T05:54:00.000Z",
"createdAtTimestamp":1209448440,
"text":null,
"tags":["poll","author_pg","story_126809"],
"parts":[126810,126811,126812],
"hnUrl":"https://news.ycombinator.com/item?id=126809",
"scrapedAt":"2026-06-26T12:00:00.000Z"
}

User (type: "user")

{
"type":"user",
"id":"pg",
"about":"Bug fixer.",
"karma":157000,
"created":1160418092,
"createdAt":"2006-10-09T18:21:32.000Z",
"submittedCount":8950,
"submitted":[1,2,3],
"hnUrl":"https://news.ycombinator.com/user?id=pg",
"scrapedAt":"2026-06-26T12:00:00.000Z"
}

Output fields

Story (type: "story")

Comment (type: "comment")

Poll (type: "poll")

Same fields as a story (type, id, objectID, title, url, author, points, numComments, createdAt, createdAtTimestamp, text, tags, hnUrl, scrapedAt), plus:

Polls do not include isAskHn / isShowHn.

User (type: "user")

Input reference

Common use cases

• Tech trend & topic research – search a keyword over a date window and export the matching stories to JSON/CSV/Excel for analysis.
• Comment & sentiment analysis – pull entire discussion threads as flat, depth-tagged comments for NLP or LLM pipelines.
• Front-page monitoring – snapshot the Top / Best / New / Ask HN / Show HN / Jobs lists on a schedule.
• Product & launch tracking – follow Show HN and Ask HN posts mentioning your product, competitor, or keyword.
• Hiring & jobs intel – scrape the Jobs list and "Who is hiring?" threads.
• User research – look up karma, account age, and submission counts in bulk.
• Dataset building – assemble a clean Hacker News dataset for machine learning, dashboards, or archival.

FAQ

Do I need a Hacker News or Algolia API key? No. This Hacker News scraper uses the public Algolia HN Search API and Firebase HN API — no API key, no login, no account required.

How do I find an item ID? It's the number in the URL, e.g. news.ycombinator.com/item?id=8863 → 8863.

Can I filter by points or comments? Yes, via minPoints / minComments. These are applied client-side — Algolia only supports server-side numeric filtering on creation date, so point/comment thresholds run after fetching.

How many search results can I get per query? Algolia caps reachable results at 1000 per query. For larger crawls, set sortBy: "date" plus a startDate; the Actor automatically pages backward through creation-date windows to fetch well beyond 1000.

Does it support pagination? Yes. Search results are paged automatically (100 hits per page), and the date-window strategy above transparently continues past the 1000-result cap.

Which countries or regions does it cover? Hacker News is a single global site with no regional editions, so results are worldwide. The APIs are not geo-restricted, so no country selection is needed.

Do I need a proxy? Usually not — HN's APIs are open and tolerant. Enable Apify Proxy only if you hit rate limits on very large runs.

Why are some stories missing comments even with includeComments on? Deleted, dead, and empty comment nodes are skipped, though their replies are still traversed. Also, maxCommentDepth may be limiting how deep the tree is flattened.

What export formats are supported? Results land in an Apify dataset you can download as JSON, CSV, or Excel, or fetch via the Apify API. Clean table views are provided for stories/polls, comments, and users.

Can I run it on a schedule? Yes. Use Apify Schedules to run front-page or keyword monitoring automatically (hourly, daily, etc.).