Pricing

Rating

Developer

Actor stats

Categories

Share

👁 ParseForge Banner

🦞 Lobste.rs Tech News Scraper

🚀 Export curated tech stories from Lobste.rs in seconds. Get titles, scores, tags, authors, and discussion links. no account, no auth, no rate limits.

🕒 Last updated: 2026-05-22 · 📊 14 fields per record · Unlimited items · Global coverage

Lobste.rs is an invite-only, curated tech community where experts share and discuss links across programming, security, systems, web development, and more. Unlike mainstream aggregators, every submitter is accountable. real usernames, real reputations. This scraper taps Lobste.rs's fully public JSON API to export stories in real time, with zero authentication or proxies required.

Whether you're tracking technology trends, monitoring specific tags like security or python, or building a developer-focused news feed, this actor delivers clean, structured data in seconds.

Coverage: Thousands of curated tech stories across 50+ community-maintained tags, covering programming languages, operating systems, web, security, mathematics, hardware, and more. Every record includes the story's score, flag count, comment count, author username, tags, and optionally the full description for self-posts.

Target Audience / Use Cases

📋 What the Lobste.rs Scraper does

• Fetches stories from the hottest feed (ranked by community score) or newest feed (chronological)
• Filters by any Lobste.rs tag: programming, javascript, python, security, linux, web, ask, show, and 50+ more
• Paginates automatically to collect as many stories as you need (up to 1,000,000)
• Returns structured records with title, URL, score, flags, comment count, tags, author, and timestamps
• Captures full description text for self-posts (ask/show posts written directly on Lobste.rs)
• Uses a polite 500ms delay between pages to respect the community server

💡 Why it matters: Lobste.rs is one of the most signal-rich tech communities on the internet. Unlike Hacker News, every member is vouched by a real person. The stories that trend here represent genuine expert consensus. not viral noise.

🎬 Full Demo

🚧 Coming soon

⚙️ Input

Example 1 - Top security stories:

{
"feedType":"hottest",
"tag":"security",
"maxItems":100
}

Example 2 - Latest programming stories:

{
"feedType":"newest",
"tag":"programming",
"maxItems":50
}

⚠️ Good to Know: The tag field accepts any tag slug that Lobste.rs uses in their URLs (e.g. https://lobste.rs/t/security). Common tags include: programming, javascript, python, security, linux, web, ask, show, go, rust, c, devops, math, hardware, browsers, practices. Self-posts (ask/show) include description text; link posts have description: null.

📊 Output

Sample record (link post):

{
"title":"Chromium publishes fixed exploit 4 years later, turns out it's actually unfixed",
"url":"https://infosec.exchange/@rebane2001/116606719764376414",
"lobstersUrl":"https://lobste.rs/s/e7lsqn",
"score":97,
"flags":1,
"commentCount":13,
"tags":["browsers","security"],
"submittedBy":"hugoarnal",
"description":null,
"createdAt":"2026-05-20T15:29:05.000-05:00",
"commentsUrl":"https://lobste.rs/s/e7lsqn/chromium_publishes_fixed_exploit_4_years",
"shortId":"e7lsqn",
"scrapedAt":"2026-05-22T01:43:03.583Z",
"error":null
}

Sample record (self-post with description):

{
"title":"Internships for early university / no former employment",
"url":"https://lobste.rs/s/r87zln",
"lobstersUrl":"https://lobste.rs/s/r87zln",
"score":28,
"flags":0,
"commentCount":15,
"tags":["ask","job"],
"submittedBy":"runxiyu",
"description":"<p>As a rising freshman beginning an undergraduate computer science degree...</p>",
"createdAt":"2026-05-21T10:09:32.000-05:00",
"commentsUrl":"https://lobste.rs/s/r87zln/internships_for_early_university_no",
"shortId":"r87zln",
"scrapedAt":"2026-05-22T01:43:03.583Z",
"error":null
}

✨ Why choose this Actor

📈 How it compares to alternatives

🚀 How to use

1. Create a free Apify account (includes $5 free credit)
2. Go to the Lobste.rs Tech News Scraper page
3. Click Try for free
4. Set your feedType (hottest or newest), optional tag, and maxItems
5. Click Start and wait a few seconds
6. Download your dataset as JSON, CSV, Excel, or XML

💼 Business use cases

Tech Trend Analysis

Monitor which topics are gaining traction in the developer community. Track score velocity on security or ai tags over days or weeks. Correlate community interest spikes with product launch cycles.

Content Intelligence

Know what developers are actually reading and discussing. Use the hottest feed as an editorial signal for blog posts, videos, or podcast topics. Identify underserved niches by watching what generates high comment counts with low scores.

Competitive and Threat Intelligence

Security teams can monitor the security tag in real time. When a CVE or novel attack technique surfaces on Lobste.rs, it often trends before mainstream outlets. Get structured, timestamped records to feed into SIEM or alerting pipelines.

Developer Community Research

Academic and industry researchers can build longitudinal datasets of expert-curated tech links. Study how community reception (score, flags, comments) correlates with article quality, recency, or domain authority.

🔌 Automating Lobste.rs Scraper

Connect this actor to your workflows with no code:

• Make (Integromat): Schedule runs, then send new stories to Slack, Notion, Airtable, or Google Sheets
• Zapier: Trigger a Zap when a run finishes - push top stories to your CMS or email list
• Slack: Pipe the hottest security stories into a #security-feed channel every morning
• Google Sheets: Build a living spreadsheet of tech trends with daily scheduled runs
• Webhooks: Use Apify's webhook system to POST results to any internal API or dashboard

🌟 Beyond business use cases

Research

Build citation-quality datasets of expert-curated tech literature. Lobste.rs's invite-only model means submitters are accountable - the links represent genuine expert consensus, not algorithmic noise.

Creative

Run weekly "best of Lobste.rs" newsletters or zines. Use the description field of self-posts to surface community-written content. Build a reading list generator that matches your tag interests.

Non-profit and Open Source

Track open-source project discussions by watching tags like open-source or freebsd. Monitor community health signals across sub-communities by comparing score distributions over time.

Experimentation

Build a personal recommendation engine trained on Lobste.rs community signals. Train a classifier to predict story success from title alone, using score as the label.

🤖 Ask an AI assistant about this scraper

"I want to track security vulnerabilities discussed by expert developers. How can I use the Lobste.rs Scraper to build a daily digest?"

"Show me how to export the top 500 Python stories from Lobste.rs this month as a CSV."

"I'm building a developer newsletter. How do I connect the Lobste.rs Scraper to Mailchimp via Zapier?"

Paste any of these into ChatGPT, Claude, or Gemini alongside the actor URL for step-by-step integration guidance.

❓ Frequently Asked Questions

🔑 Do I need an account to use this? No. Lobste.rs has a completely public JSON API. This actor requires no credentials, no auth tokens, and no proxies.

📄 How many stories can I get per run? Paid users can collect up to 1,000,000 stories. Free users get a 10-item preview. Lobste.rs paginates in batches of ~25 stories per page.

🏷️ What tags are available? Lobste.rs maintains 50+ community tags. Common ones: programming, javascript, python, rust, go, c, security, linux, web, devops, math, hardware, browsers, practices, ask, show. Browse all tags at https://lobste.rs/tags.

📝 Why is description null for most stories? Only self-posts (ask/show posts written directly on Lobste.rs) have a description. Link posts that point to external URLs have description: null - this is correct behavior from the source.

⭐ What is score vs flags? score is the net community rating. flags counts how many users flagged a story as off-topic or low quality. A story with high score and low flags is strongly endorsed by the community.

🔄 How fresh is the data? Data is fetched in real time on every run. There is no caching. Results reflect Lobste.rs's current state at the moment you start the actor.

📊 Can I get comments, not just stories? This actor returns story-level data including comment counts and links to the comments page. For full comment text, you would need to fetch individual story detail pages.

🗓️ Can I schedule daily runs? Yes. Use Apify's built-in scheduler to run this actor on a cron schedule (e.g., every morning at 8am UTC) and accumulate a longitudinal dataset.

🔌 Can I send results directly to Google Sheets? Yes. Use the Apify-Google Sheets integration or connect via Make or Zapier to push each run's results into a spreadsheet automatically.

🚦 Is there a rate limit risk? The actor uses a polite 500ms delay between page requests and sends a standard User-Agent header. Lobste.rs's public API is designed for programmatic access.

💸 How is pricing calculated? This actor uses pay-per-event pricing - you only pay for the stories you actually collect. There are no run-time fees for idle pages or failed requests.

🔌 Integrate with any app

Export your Lobste.rs dataset to: JSON - CSV - Excel - XML - Google Sheets - Airtable - Notion - Slack - Zapier - Make - Power Automate - REST API - Webhooks - S3 - BigQuery - Snowflake - PostgreSQL - MongoDB - Elasticsearch

🔗 Recommended Actors

💡 Pro Tip: browse the complete ParseForge collection for 50+ specialized data extractors covering developer tools, tech communities, job boards, and more.

This actor is not affiliated with or endorsed by Lobste.rs. It uses only publicly available data from the official Lobste.rs JSON API. Use responsibly and in accordance with Lobste.rs's terms of service.