Firecrawl: vendor profile

In short

Firecrawl announced a $75M Series B on 2026-09-22 and bills 1 credit per page scraped or crawled. The free tier gives 1,000 credits a month, and Hobby costs $16 a month billed annually. A default cache of two days means a test of a fresh deploy needs maxAge set to 0, and robots.txt can be skipped only on Enterprise.

At a glance

standards.txt firecrawl

llms-txt               = partial
markdown-negotiation   = no statement
markdown-twin          = no statement
initial-html           = no statement
real-404               = no statement
vary-accept            = no statement

0 supports, 1 partial or announced, 5 no statement.

Firecrawl Markdown or llms.txt tool

Paid web data API that fetches and renders pages and returns them as markdown, cleaned HTML, raw HTML or JSON, with crawl, map and search endpoints; it honours robots.txt by default and reports each page's HTTP status, so 404s stay visible.

Type: Markdown or llms.txt tool
Home page: www.firecrawl.dev, answered HTTP 200 on 2026-11-05
Last read: 5 November 2026

What it does for this layer

Firecrawl announced a $75M Series B on 2026-09-22, led by Smash Capital with Altos Ventures, Nexus Venture Partners and Y Combinator among the backers, alongside Alexandria, a layer that puts official data providers, custom connectors and Firecrawl's own indexes next to the live web. [primary source, 2026-09-22] source, read 5 November 2026.
Firecrawl's founders say they built Mendable, a documentation chat product, before Firecrawl, and that more than 1.5 million users now build with Firecrawl. [vendor claim, 2026-09-22] source, read 5 November 2026.
Under Firecrawl pricing effective 2026-09-04, the free tier gives 1,000 credits a month with no card, Hobby costs $16 a month billed annually ($19 monthly) for 5,000 credits, Standard $83 ($99) for 100,000, Growth $333 ($399) for 500,000 and Scale $599 ($749) for 1,000,000. [primary source, 2026-09-04] source, read 5 November 2026.
Firecrawl bills 1 credit per page scraped or crawled and per Map call, 2 credits per 10 search results, and 4 more per page for JSON, Question or Highlight output; a target page that answers 403 or 404 is still returned and still costs a credit. [primary source, 2026-09-04] source, read 5 November 2026.
Firecrawl's /scrape can return markdown, a summary, cleaned HTML, rawHtml exactly as the server sent it, Base64 of the original body, links, images, screenshots, JSON, product and branding extracts, so a caller can compare the server HTML with the converted markdown. [primary source] source, read 5 November 2026.
Firecrawl serves a cached copy by default when it is younger than maxAge = 172,800,000 ms (two days), so a page changed yesterday can come back stale unless the caller sets maxAge to 0. [primary source] source, read 5 November 2026.
Firecrawl's crawler obeys robots.txt and lists refused URLs as robotsBlocked; the ignoreRobotsTxt and robotsUserAgent overrides exist only on Enterprise plans. [primary source] source, read 5 November 2026.
A Firecrawl crawl stays under the start path unless crawlEntireDomain is set, reads the sitemap as well as links by default (sitemap: "include"), and stops at 10,000 pages unless limit says otherwise. [primary source] source, read 5 November 2026.
Crawl results from Firecrawl keep pages whose server answered with an error such as 404 and record the target's status in metadata.statusCode, which lets a site owner list every URL that returns a real 404 in one crawl. [primary source] source, read 5 November 2026.
Firecrawl v2.10, released on 2026-05-15, added a /parse endpoint for uploaded documents, a Lockdown Mode and the Question and Highlights output formats. [primary source, 2026-05-15] source, read 5 November 2026.
The Firecrawl source repository on GitHub carries the GNU Affero General Public License version 3, which covers the self-hosted version separately from the paid cloud API. [primary source] source, read 5 November 2026.
On 2026-11-05 docs.firecrawl.dev, a Mintlify-built docs site, served a 9,770-byte llms.txt, answered Accept: text/markdown on /features/crawl with 29,228 bytes of markdown and Vary: accept-encoding, Accept, User-Agent, served the same text at /features/crawl.md, and returned 404 for a made-up path. [observed, 2026-11-05] source, read 5 November 2026.

Products and plans

Plans and prices as the vendor's own pages showed them on the date read.
Product	What it is	Price	As of	Source
Firecrawl API (cloud)	Scrape, crawl, map, search and related endpoints that return pages as markdown, HTML or JSON, billed in credits.	Free tier, paid plans above it Free 1,000 credits/month; Hobby $16/mo billed annually ($19 monthly); Standard $83; Growth $333; Scale $599; Enterprise custom	2026-11-05	source
Firecrawl (self-hosted)	The AGPL-3.0 source of Firecrawl run on your own infrastructure with Docker Compose.	Free, open source AGPL-3.0	2026-11-05	source

Standards

The six standards this site tracks; no statement means the pages read said nothing.
Standard	Status	What the vendor's pages say	Source
llms-txt	partial	The alpha /llmstxt endpoint still answers but is no longer maintained; Firecrawl points to the create-llmstxt-py script instead.	source
markdown-negotiation	no statement	Firecrawl converts pages to markdown inside its own API; nothing in the scrape or crawl docs adds Accept handling to a customer's site.	none
markdown-twin	no statement	No feature that publishes .md copies on a site; markdown is returned to the API caller only.	none
initial-html	no statement	Firecrawl is a fetcher, not a site renderer; its rawHtml format shows what the server sent before rendering.	none
real-404	no statement	No statement about serving 404s; crawl results report each target page's status in metadata.statusCode.	none
vary-accept	no statement	No statement found in the scrape, crawl and pricing pages read on 2026-11-05.	none

Checks on its own site

Readings of firecrawl.dev, the vendor's own website, which describe that site and not the product.
Reading	Result	Date
Home page request	HTTP 200; H1 in the initial HTML: yes; 3,867 words; 0 JSON-LD blocks	2026-11-05
Request with Accept: text/markdown	HTTP 200, text/markdown, Vary: Accept	2026-11-05
/llms.txt at the host root	HTTP 200, text/plain, 22 KB, starts with an H1	2026-11-05
A made-up path	HTTP 404	2026-11-05
Ora score, cached	85, grade B	2026-11-05
Cloudflare keyless scan	53, L5 Agent-Native	2026-11-05
Shared crawl row	outside the ranked slice; home page HTTP 200; H1 in the HTML: yes; 4,052 words; JSON-LD types: none; answers markdown: yes; made-up path HTTP 404; llms.txt valid: yes	2026-11-04

What it does not do

Firecrawl does not make a site agent-readable on its own; the markdown it builds goes to the API caller and nothing changes on the site that was scraped.
Firecrawl's two-day default cache means a scrape is not always a live fetch, so testing a fresh deploy needs maxAge set to 0.

Related pages on this site

Sources

Firecrawl: home page https://www.firecrawl.dev/, read 5 November 2026
Introducing Alexandria and our $75M Series B https://www.firecrawl.dev/blog/introducing-alexandria-series-b, read 5 November 2026
Firecrawl pricing https://www.firecrawl.dev/pricing, read 5 November 2026
Scrape https://docs.firecrawl.dev/features/scrape, read 5 November 2026
Crawl https://docs.firecrawl.dev/features/crawl, read 5 November 2026
Firecrawl changelog https://www.firecrawl.dev/changelog, read 5 November 2026
Firecrawl LICENSE https://raw.githubusercontent.com/firecrawl/firecrawl/main/LICENSE, read 5 November 2026
docs.firecrawl.dev llms.txt, markdown and 404 checks (observed) https://docs.firecrawl.dev/llms.txt, read 5 November 2026
LLMs.txt API (alpha) https://docs.firecrawl.dev/features/alpha/llmstxt, read 5 November 2026

Last reviewed 5 November 2026