Firecrawl

Category: Infra (web data extraction pipeline for LLMs)

Official site: firecrawl.dev

Docs: docs.firecrawl.dev

GitHub: github.com/firecrawl/firecrawl (142k+ stars)

What it is

Firecrawl is a managed web scraping and crawling API that converts any website into clean, LLM-ready markdown or structured JSON. It handles JavaScript rendering, dynamic content, and link-following out of the box. You point it at a URL; it returns content an AI agent can consume directly.

Core operations:

• Scrape - extract clean markdown, JSON, or a screenshot from a single page (1 credit per page)
• Crawl - follow all links across an entire site (1 credit per page crawled)
• Map - return a full sitemap of URLs without fetching content (1 credit per page)
• Search - query the web and retrieve full page content from results (2 credits per 10 results)
• Interact - click, type, and navigate via AI prompts using a live browser (2 credits per browser minute)

Pricing

Tier	Monthly credits	Cost
Free	1,000 credits	$0
Hobby	Higher allotment	Paid
Standard	Higher allotment	Paid
Growth	Higher allotment	Paid

Credits do not roll over. 1 credit = 1 page for scrape, crawl, or map operations.

No API key is required to get started against the free tier. Register at firecrawl.dev/app/api-keys for a key when you need higher rate limits or production access.

SDKs and interfaces

• Python SDK (firecrawl-py)
• Node.js SDK (firecrawl)
• Go, Rust, Java, Elixir SDKs
• REST API (v1)
• CLI (firecrawl-cli)
• MCP server (connects Firecrawl directly to Claude and other AI tools)

Template / boilerplate

No official downloadable starter template. The Playground lets you test scrape and crawl without writing code.

Safety and ToS notes

Firecrawl is a managed service. It routes requests through its own infrastructure ("Fire-engine"). Respect the robots.txt and terms of service of target sites you scrape. Do not use Firecrawl to harvest data in violation of a site's terms. The free tier enforces rate limits automatically; production use requires a paid plan or your own self-hosted instance.