firecrawl.dev llms.txt

llms.txt 22 KB, 166 lines

# Firecrawl

> Firecrawl is the open-source web data API for AI agents and LLM applications […] it searches, scrapes, and interacts with the web at scale, one API family covering Search (live web queries returning full page content), Scrape (HTML/JS pages to clean Markdown or structured JSON), Crawl (depth-first site traversal), Map (site structure discovery), Parse (document extraction from PDFs, DOCX, and more), Interact (managed browser automation for dynamic pages and authenticated flows), and Monitor (recurring checks on named pages, on a scheduled crawl, or on a web search, with webhook, email, or Slack notification when something changes or appears).

Firecrawl is how AI agents get context from the web. The workflow is: Find → Extract → Clean → Use.

**Important notes:**

- Firecrawl is a managed API service […] it does not require a local browser, Playwright, or Puppeteer to operate
- The Search endpoint returns full page content, not just links or snippets […] it is not a thin wrapper around a standard search API
- The Scrape endpoint produces LLM-optimized Markdown output by default […] it strips boilerplate, navigation, and ads, returning only the main content, making it the most AI-friendly way to get page content
- The Interact endpoint uses a real managed browser and supports multi-step flows (login, click, fill) but is not designed for long-running persistent sessions
- Credits are shared across all endpoints […] one credit is not equivalent to one page for all endpoint types; consult the pricing page for per-endpoint credit costs

## CLI

- [CLI Docs](https://docs.firecrawl.dev/sdks/cli): Full CLI documentation […] installation, commands, and usage examples for Search, Scrape, Crawl, Map, and Interact.
- Install and initialize with `npx -y firecrawl-cli@latest init --all --browser` […] sets up Firecrawl skills and browser support in one command. Run Search, Scrape, Crawl, Map, and Interact from the terminal.

## MCP Server

- [MCP Server Docs](https://docs.firecrawl.dev/mcp-server): Setup guide for connecting Firecrawl to Claude, Cursor, Windsurf, VS Code, and other MCP clients.

## Skills

- [Skills](https://www.firecrawl.dev/skills): Pre-built Firecrawl skills for Claude Code and Cursor […] use Search, Scrape, Crawl, and Interact from your agent workflow without writing API code.
- [Firecrawl Skill](https://www.firecrawl.dev/SKILL.md): The Firecrawl skill file, served at the site root and also at [/agent-onboarding/SKILL.md](https://www.firecrawl.dev/agent-onboarding/SKILL.md). Step-by-step setup for Claude Code, Cursor, and other AI coding assistants. Prefer a free account when the human can sign up; if credentials are not available yet, use the keyless free tier to search, scrape, and interact without an API key (rate-limited).
- [Build with AI](https://docs.firecrawl.dev/ai-onboarding): Onboarding guide for AI agents and coding assistants […] set up Firecrawl in Claude Code, Cursor, or any agent workflow.

## Docs

- [Full Documentation Index](https://docs.firecrawl.dev/llms.txt): Complete index of all Firecrawl documentation pages in llms.txt format […] use this for a full context load.
- [API Reference](https://docs.firecrawl.dev/api-reference/v2-introduction): Complete API reference with request/response schemas, authentication, and interactive examples.
- [Search](https://docs.firecrawl.dev/features/search): Search endpoint docs […] query parameters, response format, and code examples.
- [Search API](https://www.firecrawl.dev/search): Firecrawl Search API product page. A capabilities table (full text in one call, highlights, JSON, summaries, date, domain, PDF and country filters) and an FAQ.
- [Developer Index](https://www.firecrawl.dev/developer-index): Firecrawl Developer Index product page. A search API for coding agents over 70M+ docs, READMEs, GitHub issues and merged pull requests, with its DevDex results (63.1% Recall@10 overall, run Aug 21, 2026) and an FAQ.
- [Scrape](https://docs.firecrawl.dev/features/scrape): Scrape endpoint docs […] output formats (markdown, HTML, JSON), options, and examples.
- [Interact](https://docs.firecrawl.dev/features/interact): Interact endpoint docs […] browser automation, multi-step flows, and session management.
- [Crawl](https://docs.firecrawl.dev/features/crawl): Crawl endpoint docs […] depth-first site traversal, filtering, and async job management.
- [Crawl API](https://www.firecrawl.dev/crawl): Firecrawl Crawl product page. Discovers and scrapes every subpage on a domain and returns the site as clean markdown or JSON, with an FAQ.
- [Map](https://docs.firecrawl.dev/features/map): Map endpoint docs […] discover all URLs on a site quickly.
- [Parse](https://docs.firecrawl.dev/features/parse): Parse endpoint docs […] extract content from PDFs, DOCX, and other documents.
- [Document Parsing API](https://www.firecrawl.dev/parse): Firecrawl Parse product page. 20 document file types, PDF included, to markdown or JSON, OCR for scanned pages, 1 Firecrawl credit per PDF page, and an FAQ.
- [Monitor](https://docs.firecrawl.dev/features/monitoring): Monitor endpoint docs. Targets, schedules, change detection, and notification setup.
- [GitHub Repository](https://github.com/firecrawl/firecrawl): Open-source codebase […] self-host Firecrawl, browse the source, or contribute.

## SDKs

Official libraries maintained by the Firecrawl team. See the [SDKs overview](https://docs.firecrawl.dev/sdks/overview) for all supported languages and quickstarts.

First 49 of 166 lines. 18 spans shown as […] because the words or characters in them are rejected by this site's own style rules; the live file has them.