VOOZH about

URL: https://pypi.org/project/firecrawl-py/

⇱ firecrawl-py · PyPI

Skip to main content

firecrawl-py 4.30.2

pip install firecrawl-py

Latest release

Released:

Python SDK for Firecrawl API

Navigation

Verified details

These details have been verified by PyPI
Maintainers
👁 Avatar for sideguide from gravatar.com
 sideguide

Project description

Firecrawl Python SDK

The Firecrawl Python SDK is a library that lets you easily search, scrape, and interact with the web for AI agents — returning clean Markdown or structured data your agents can ship with. It provides a simple and intuitive interface for the Firecrawl API.

Installation

To install the Firecrawl Python SDK, you can use pip:

pipinstallfirecrawl-py

Usage

  1. Get an API key from firecrawl.dev
  2. Set the API key as an environment variable named FIRECRAWL_API_KEY or pass it as a parameter to the Firecrawl class.

Here's an example of how to use the SDK:

fromfirecrawlimport Firecrawl
fromfirecrawl.typesimport ScrapeOptions

firecrawl = Firecrawl(api_key="fc-YOUR_API_KEY")

# Scrape a website (v2):
data = firecrawl.scrape(
 'https://firecrawl.dev', 
 formats=['markdown', 'html']
)
print(data)

# Crawl a website (v2 waiter):
crawl_status = firecrawl.crawl(
 'https://firecrawl.dev', 
 limit=100, 
 scrape_options=ScrapeOptions(formats=['markdown', 'html'])
)
print(crawl_status)

Scraping a URL

To scrape a single URL, use the scrape method. It takes the URL as a parameter and returns a document with the requested formats.

# Scrape a website (v2):
scrape_result = firecrawl.scrape('https://firecrawl.dev', formats=['markdown', 'html'])
print(scrape_result)

Video extraction

Use the video format on supported video URLs, including YouTube and TikTok. The returned video field is a signed URL to the extracted video file.

doc = firecrawl.scrape('https://www.youtube.com/watch?v=dQw4w9WgXcQ', formats=['video'])
print(doc.video)

Product extraction

Use the product format on product pages to deterministically pull structured product data (title, price, availability, variants). It is the deterministic counterpart to the LLM-based json format.

doc = firecrawl.scrape('https://firecrawl.dev', formats=['product'])
print(doc.product)

Menu extraction

Use the menu format on menu pages to deterministically pull structured menu data (merchant, sections, items, prices, availability). It is the deterministic counterpart to the LLM-based json format.

doc = firecrawl.scrape('https://example.com/restaurant/menu', formats=['menu'])
print(doc.menu)

Parsing uploaded files

Use parse to upload local bytes/files (html, pdf, docx, etc.) as multipart form data and return the parsed document. parse does not support change tracking or browser-only options (actions, wait_for, location, mobile, screenshot, branding, audio, video).

fromfirecrawlimport Firecrawl
fromfirecrawl.v2.typesimport ParseOptions

firecrawl = Firecrawl(api_key="fc-YOUR_API_KEY")

doc = firecrawl.parse(
 b"<!DOCTYPE html><html><body><h1>Python Parse</h1></body></html>",
 filename="upload.html",
 content_type="text/html",
 options=ParseOptions(formats=["markdown"]),
)

print(doc.markdown)

Crawling a Website

To crawl a website, use the crawl method. It takes the starting URL and optional parameters as arguments. You can control depth, limits, formats, and more.

crawl_status = firecrawl.crawl(
 'https://firecrawl.dev', 
 limit=100, 
 scrape_options=ScrapeOptions(formats=['markdown', 'html']),
 poll_interval=30
)
print(crawl_status)

Asynchronous Crawling

Looking for async operations? Check out the Async Class section below.

To enqueue a crawl asynchronously, use start_crawl. It returns the crawl ID which you can use to check the status of the crawl job.

crawl_job = firecrawl.start_crawl(
 'https://firecrawl.dev', 
 limit=100, 
 scrape_options=ScrapeOptions(formats=['markdown', 'html']),
)
print(crawl_job)

Checking Crawl Status

To check the status of a crawl job, use the get_crawl_status method. It takes the job ID as a parameter and returns the current status of the crawl job.

crawl_status = firecrawl.get_crawl_status("<crawl_id>")
print(crawl_status)

Manual Pagination (v2)

Crawl and batch scrape status responses may include a next URL when more data is available. The SDK auto-paginates by default; to page manually, disable auto-pagination and pass the opaque next URL back to the SDK.

fromfirecrawl.v2.typesimport PaginationConfig

# Crawl: fetch one page at a time
crawl_job = firecrawl.start_crawl("https://firecrawl.dev", limit=100)
status = firecrawl.get_crawl_status(
 crawl_job.id,
 pagination_config=PaginationConfig(auto_paginate=False),
)
if status.next:
 page2 = firecrawl.get_crawl_status_page(status.next)

# Batch scrape: fetch one page at a time
batch_job = firecrawl.start_batch_scrape(["https://firecrawl.dev"])
status = firecrawl.get_batch_scrape_status(
 batch_job.id,
 pagination_config=PaginationConfig(auto_paginate=False),
)
if status.next:
 page2 = firecrawl.get_batch_scrape_status_page(status.next)

Cancelling a Crawl

To cancel an asynchronous crawl job, use the cancel_crawl method. It takes the job ID of the asynchronous crawl as a parameter and returns the cancellation status.

cancel_crawl = firecrawl.cancel_crawl(id)
print(cancel_crawl)

Map a Website

Use map to generate a list of URLs from a website. Options let you customize the mapping process, including whether to use the sitemap or include subdomains.

# Map a website (v2):
map_result = firecrawl.map('https://firecrawl.dev')
print(map_result)

Scrape-bound interactive browsing (v2)

Use a scrape job ID to keep interacting with the replayed browser context:

doc = firecrawl.scrape(
 "https://example.com",
 actions=[{"type": "click", "selector": "a[href='/pricing']"}],
)

scrape_job_id = doc.metadata_typed.scrape_id
if not scrape_job_id:
 raise RuntimeError("Missing scrape job id")

run = firecrawl.interact(
 scrape_job_id,
 code="print(await page.url())",
 language="python",
 timeout=60,
)
print(run.stdout)

firecrawl.stop_interaction(scrape_job_id)

{/* ### Extracting Structured Data from Websites

To extract structured data from websites, use the extract method. It takes the URLs to extract data from, a prompt, and a schema as arguments. The schema is a Pydantic model that defines the structure of the extracted data.

*/}

Crawling a Website with WebSockets

To crawl a website with WebSockets, use the crawl_url_and_watch method. It takes the starting URL and optional parameters as arguments. The params argument allows you to specify additional options for the crawl job, such as the maximum number of pages to crawl, allowed domains, and the output format.

# inside an async function...
nest_asyncio.apply()

# Define event handlers
defon_document(detail):
 print("DOC", detail)

defon_error(detail):
 print("ERR", detail['error'])

defon_done(detail):
 print("DONE", detail['status'])

 # Function to start the crawl and watch process
async defstart_crawl_and_watch():
 # Initiate the crawl job and get the watcher
 watcher = app.crawl_url_and_watch('firecrawl.dev', exclude_paths=['blog/*'], limit=5)

 # Add event listeners
 watcher.add_event_listener("document", on_document)
 watcher.add_event_listener("error", on_error)
 watcher.add_event_listener("done", on_done)

 # Start the watcher
 await watcher.connect()

# Run the event loop
await start_crawl_and_watch()

Error Handling

The SDK handles errors returned by the Firecrawl API and raises appropriate exceptions. If an error occurs during a request, an exception will be raised with a descriptive error message.

Async Class

For async operations, you can use the AsyncFirecrawl class. Its methods mirror the Firecrawl class, but you await them.

fromfirecrawlimport AsyncFirecrawl

firecrawl = AsyncFirecrawl(api_key="YOUR_API_KEY")

# Async Scrape (v2)
async defexample_scrape():
 scrape_result = await firecrawl.scrape(url="https://example.com")
 print(scrape_result)

# Async Parse (v2)
async defexample_parse():
 parse_result = await firecrawl.parse(
 b"<!DOCTYPE html><html><body><h1>Async Parse</h1></body></html>",
 filename="upload.html",
 content_type="text/html",
 )
 print(parse_result)

# Async Crawl (v2)
async defexample_crawl():
 crawl_result = await firecrawl.crawl(url="https://example.com")
 print(crawl_result)

v1 compatibility

For legacy code paths, v1 remains available under firecrawl.v1 with the original method names.

fromfirecrawlimport Firecrawl

firecrawl = Firecrawl(api_key="YOUR_API_KEY")

# v1 methods (feature‑frozen)
doc_v1 = firecrawl.v1.scrape_url('https://firecrawl.dev', formats=['markdown', 'html'])
crawl_v1 = firecrawl.v1.crawl_url('https://firecrawl.dev', limit=100)
map_v1 = firecrawl.v1.map_url('https://firecrawl.dev')

Project details

Verified details

These details have been verified by PyPI
Maintainers
👁 Avatar for sideguide from gravatar.com
 sideguide

Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

firecrawl_py-4.30.2.tar.gz (198.0 kB view details)

Uploaded Source

Built Distribution

Filter files by name, interpreter, ABI, and platform.

If you're not sure about the file name format, learn more about wheel file names.

Copy a direct link to the current filters

firecrawl_py-4.30.2-py3-none-any.whl (246.7 kB view details)

Uploaded Python 3

File details

Details for the file firecrawl_py-4.30.2.tar.gz.

File metadata

  • Download URL: firecrawl_py-4.30.2.tar.gz
  • Upload date:
  • Size: 198.0 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.14.2

File hashes

Hashes for firecrawl_py-4.30.2.tar.gz
Algorithm Hash digest
SHA256 e64bb4593e1659fdc92c04e7dc3bc870ba38579c85e02941d996c807e59d9f43
MD5 3929d7fc2f8c31aa09a9a93eacb93f0a
BLAKE2b-256 44cb5fce6a2fc05098822bc2b012b34c3b219c8252f9e87856b17eadfdde9795

See more details on using hashes here.

File details

Details for the file firecrawl_py-4.30.2-py3-none-any.whl.

File metadata

  • Download URL: firecrawl_py-4.30.2-py3-none-any.whl
  • Upload date:
  • Size: 246.7 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.14.2

File hashes

Hashes for firecrawl_py-4.30.2-py3-none-any.whl
Algorithm Hash digest
SHA256 c5985faba7d3772f6a97ebeb143f8782405dcc1b100419c4d0c904ca4256f3fb
MD5 25740dbfd1f3473c85d193146c02c31b
BLAKE2b-256 3fd8c7de8b0601e65011e64e54d3dc14f285e21aee1d5ec84093616931399d14

See more details on using hashes here.

Supported by

👁 Image
AWS Cloud computing and Security Sponsor 👁 Image
Datadog Monitoring 👁 Image
Depot Continuous Integration 👁 Image
Fastly CDN 👁 Image
Google Download Analytics 👁 Image
Pingdom Monitoring 👁 Image
Sentry Error logging 👁 Image
StatusPage Status page