Pricing

Rating

Developer

Actor stats

Categories

Share

Hacker News Scraper: Stories, Comments & Users Free

The most comprehensive Hacker News scraper on Apify. Extract HN stories, comments, and user profiles at scale. Search Hacker News posts by keyword. Export to JSON, CSV, or Excel. Built on the official HN Firebase API for maximum reliability.

Why Use This Hacker News Scraper?

• Tech Trend Analysis - Track what's trending on Hacker News. Monitor emerging technologies, frameworks, and tools before they go mainstream.
• Startup Scouting - Discover new startups from Show HN and Launch HN posts. Get early intelligence on companies before they hit the press.
• Hiring Intelligence - Monitor "Who is Hiring" threads and job postings. Analyze which companies are hiring and for what roles.
• Competitive Research - Track mentions of competitors, products, or technologies. See what the developer community thinks.
• Content Curation - Build datasets of top-performing HN content. Understand what resonates with the tech community.
• Academic Research - Analyze HN data for sentiment analysis, trend detection, and community dynamics research.

Features

Input Configuration

Output - Stories

{
"id":12345678,
"title":"Show HN: I built an AI-powered code reviewer",
"url":"https://example.com/project",
"text":null,
"author":"techfounder",
"score":342,
"commentCount":156,
"createdAt":"2025-12-15T10:30:00.000Z",
"storyType":"show",
"hnUrl":"https://news.ycombinator.com/item?id=12345678"
}

Output - Users

{
"username":"techfounder",
"karma":15420,
"about":"Building cool stuff. Previously at BigCorp.",
"createdAt":"2015-03-10T08:00:00.000Z",
"submittedCount":892
}

Output - Comments (when includeComments is true)

{
"id":12345679,
"author":"commenter1",
"text":"This is a great project! I've been looking for something like this.",
"createdAt":"2025-12-15T11:00:00.000Z",
"parentId":12345678
}

Example Use Cases

Get Top 50 HN Stories

{
"category":"top",
"maxItems":50
}

Search for AI Startup Posts

{
"category":"search",
"searchQuery":"AI startup launch",
"maxItems":100
}

Get Ask HN with Comments

{
"category":"ask",
"maxItems":20,
"includeComments":true,
"maxCommentsPerStory":100
}

Get Stories + Author Profiles

{
"category":"best",
"maxItems":30,
"scrapeType":"both"
}

API Usage Examples

JavaScript

import{ ApifyClient }from'apify-client';
const client =newApifyClient({token:'YOUR_API_TOKEN'});
const run =await client.actor('cryptosignals/hackernews-scraper').call({
category:'top',
maxItems:50,
});
const{ items }=await client.dataset(run.defaultDatasetId).listItems();
console.log(items);

Python

from apify_client import ApifyClient
client = ApifyClient("YOUR_API_TOKEN")
run = client.actor("cryptosignals/hackernews-scraper").call(run_input={
"category":"top",
"maxItems":50,
})
items =list(client.dataset(run["defaultDatasetId"]).iterate_items())
print(items)

cURL

curl"https://api.apify.com/v2/acts/cryptosignals~hackernews-scraper/runs"\
-X POST \
-H"Content-Type: application/json"\
-H"Authorization: Bearer YOUR_API_TOKEN"\
-d'{"category": "top", "maxItems": 50}'

About the Data Source

This scraper uses the official Hacker News Firebase API (https://github.com/HackerNews/API) and the Algolia HN Search API for keyword search. These are public, free APIs provided by Y Combinator. No scraping of the website is involved -- all data comes from official endpoints, ensuring reliability and compliance.

FAQ

Q: Is this scraper free to use? A: The actor itself is free. You only pay for Apify compute units based on your plan.

Q: How often is the data updated? A: The HN API provides real-time data. Stories and scores are current at the time of scraping.

Q: Can I scrape historical data? A: Use the search feature to find older posts by keyword. The Firebase API only provides current story lists.

Q: Why are some comments missing? A: Deleted or dead (flagged) comments are filtered out automatically.

Q: What's the rate limit? A: The official HN API has no documented rate limit. The scraper uses concurrent requests with sensible batching.

Q: Can I schedule this to run automatically? A: Yes! Use Apify's scheduling feature to run the scraper hourly, daily, or on any custom schedule.

Tags

hacker news, hn scraper, tech news, startup intelligence, developer tools, comment scraper, user profiles, tech trends, Y Combinator, hackernews api, hn data extraction