AI llm.txt File Generator API

Pricing

$30.00 / 1,000 llm.txt file generations

AI llm.txt File Generator API

Generate llm.txt files automatically for any website. Map website directories and convert documentation into clean markdown structured llm.txt files for LLM agents.

Pricing

$30.00 / 1,000 llm.txt file generations

Rating

0.0

(0)

Developer

👁 dev00

dev00

Maintained by Community

Actor stats

Bookmarked

Total users

Monthly active users

3 days ago

Last modified

llm.txt File Generator Actor

Map directories and compile website documentation pages into the standard llm.txt markdown format optimized for LLMs and AI agents (such as Cursor, ChatGPT, and Copilot).

Features

URL Mapping: Automatically crawls and lists all sub-URLs for a target base URL to compile documentation context.
AI-Agent Ready Markdown: Converts website HTML/content into formatted markdown, stripping header/footer noise.
Pay-Per-Event pricing: Monetized per event at a cost of $0.005 (half a cent) per successful generation.
Flexible Options: Specify custom lists of sub-pages manually or let the directory crawler auto-discover them.

Input Configuration

The actor accepts the following JSON input parameters:

Field	Type	Required	Default	Description
`url`	string	Yes	-	Website base URL to map and convert (e.g. `https://apify.com/`).
`urls`	array	No	-	Explicit list of sub-URLs to parse. Auto-discovered if omitted.

Example Input

{
"url":"https://apify.com/",
"urls":[
"https://apify.com",
"https://docs.apify.com"
]
}

Dataset Output

Each execution pushes matched song records directly to the Apify dataset:

Field	Type	Description
`success`	boolean	Indicates if the generator succeeded.
`baseUrl`	string	Base URL of the crawled website.
`mappedUrlsCount`	integer	Number of child pages included.
`mappedUrls`	array	List of all child URLs included.
`llmTxtContent`	string	Compiled raw markdown output representing the llm.txt file.

👁 /llms.txt Generator avatar

/llms.txt Generator

jakub.kopecky/llmstxt-generator

The /llms.txt Generator 🕸️📄 extracts website content to create an llms.txt file for AI apps 🤖✨ like LLM fine-tuning and indexing. Output is available 📥 in the Key-Value Store for easy download and integration into workflows. 🚀

👁 User avatar

Jakub Kopecký

791

3.1

Site to Markdown — any site to clean, LLM-ready markdown

topsail/site-to-markdown

Scrape any website to clean, LLM-ready markdown — a compliant Firecrawl alternative for RAG ingestion, robots.txt always on.

👁 User avatar

Connor Teskey

👁 Website Content to Markdown for LLM Training avatar

Website Content to Markdown for LLM Training

easyapi/website-content-to-markdown-for-llm-training

🚀 Transform web content into clean, LLM-ready Markdown! 📘 Scrape multiple pages, extract main content, and convert to Markdown format. Perfect for AI researchers, data scientists, and LLM developers. Fast, efficient, and customizable. Supercharge your AI training data today! 🌐📝🧠

👁 User avatar

EasyApi

323

5.0

Public AI Crawler Policy Signal Agent

jacksu/public-ai-crawler-policy-signal-agent

Analyze public robots.txt and llms.txt files for AI crawler allow/block policy evidence, LLM guidance files, stable hashes, and useful-result pricing.

👁 User avatar

jack su

Website to Markdown – Clean LLM & RAG Content Extractor

dataquarry/website-to-markdown

Convert any public web page to clean, LLM-ready Markdown with metadata — by URL, a list of URLs, or a whole-site crawl. Strips nav/ads/boilerplate, keeps headings/lists/tables/code. Respects robots.txt. No API key.

👁 User avatar

Daniel Brenner

Llms Txt Generator

francis.businessfm/llms-txt-generator

👁 User avatar

Francis Marzyński

👁 Website to Markdown Crawler for LLM & RAG avatar

Website to Markdown Crawler for LLM & RAG

logiover/website-text-markdown-crawler

Crawl any website to clean Markdown and plain text for LLM training and RAG. HTML to Markdown, no API or login. Export website text to CSV or JSON.

👁 User avatar

Logiover

👁 HTML to Markdown/Text avatar

HTML to Markdown/Text

wowo51/html-to-md

Convert html to md or txt. Perfect for AI agents that need to cut expensive LLM costs.

👁 User avatar

Warren Harding

👁 Web-to-Markdown Generator for AI & RAG Pipelines avatar

Web-to-Markdown Generator for AI & RAG Pipelines

profitstack/web-to-markdown-generator-for-ai-rag-pipelines

Convert any website into clean, heading-based chunking, LLM-ready Markdown for RAG and AI agents.

👁 User avatar

Manas Mantri

👁 LLMs.txt File Generator avatar

LLMs.txt File Generator

justa/llms-txt-file-generator

Generate an llms.txt file from a website sitemap. Crawls all URLs, extracts titles and meta descriptions, and creates a Markdown-formatted file following the llms.txt specification. Upload then the output of your file directly on your website (Webflow, Wordpress etc.)

👁 User avatar

Benoit Eveillard

URL: https://apify.com/dev00/ai-llm-txt-file-generator-api