Discourse Community Scraper

Pricing

Rating

Developer

Actor stats

Categories

Share

Extract topics, discussions, and posts from any public Discourse forum. Supports latest topics, trending discussions, category browsing, tag filtering, full-text search, user activity, and complete post threads — all via the standard Discourse REST API.

Works with meta.discourse.org, community.cloudflare.com, and any self-hosted Discourse instance.

What It Does

• Latest topics — fetch the most recently active discussions
• Top/trending topics — get the most popular topics by time period (daily/weekly/monthly/yearly)
• Browse by category — explore topics within a specific forum category
• Filter by tag — find all topics with a given tag
• Full-text search — search across all forum topics by keyword
• User topics — get all topics created by a specific user
• Get posts — fetch all posts/replies within a specific topic thread

Input Schema

Modes

Output Schema

Topic Record (recordType: "discourseTopic")

Post Record (recordType: "discoursePost")

Example Output

Topic

{
"topicId":12345,
"title":"How to set up email notifications?",
"slug":"how-to-set-up-email-notifications",
"url":"https://meta.discourse.org/t/how-to-set-up-email-notifications/12345",
"postsCount":8,
"views":1250,
"likeCount":23,
"replyCount":7,
"createdAt":"2024-03-15T09:30:00.000Z",
"categoryName":"Support",
"tags":["email","notifications","configuration"],
"authorUsername":"john_doe",
"solved":true,
"forumUrl":"https://meta.discourse.org",
"recordType":"discourseTopic",
"scrapedAt":"2026-05-15T10:00:00+00:00"
}

Post

{
"postId":56789,
"topicId":12345,
"postNumber":3,
"url":"https://meta.discourse.org/t/how-to-set-up-email-notifications/12345/3",
"authorUsername":"admin_user",
"cooked":"<p>Go to Admin → Settings → Email and configure your SMTP server.</p>",
"likeCount":15,
"isAcceptedAnswer":true,
"recordType":"discoursePost",
"scrapedAt":"2026-05-15T10:00:00+00:00"
}

FAQ

Does it work with any Discourse forum? Yes. The scraper uses the standard Discourse REST API (all .json endpoints) that is available on every Discourse instance — both hosted and self-hosted. Simply provide the forum's base URL.

Do I need an API key? No API key is needed for public forums. The apiKey and apiUsername fields are only required if the forum restricts access to logged-in users.

How do I find the category slug? Look at the forum URL when browsing a category. For example, https://meta.discourse.org/c/support/6 uses the slug support or the path support/6.

How many posts can I get from a topic? Using mode=getPosts, the scraper fetches all posts in a topic, paginating through the full thread. Set maxItems to control the upper limit.

Can I scrape a private Discourse forum? Yes, if you have valid API credentials. Provide your API key in the apiKey field and your username in apiUsername. The forum administrator generates API keys in the admin panel.

What's the difference between postsCount and replyCount? postsCount is the total number of posts in the thread (including the original post). replyCount counts only replies made to other posts within the thread.

How does the solved/accepted answer work? Discourse has an optional "Solved" plugin that lets users mark a reply as the accepted answer. If a forum uses this plugin, solved=true appears on topics that have an accepted answer, and isAcceptedAnswer=true appears on the specific accepted post.

Can I search across multiple forums in one run? Each run targets one forum URL. To scrape multiple forums, run the actor multiple times with different forumUrl values.