--- title: "Firecrawl: vendor profile: Agent Ready HTML" description: "Firecrawl, markdown tooling: 0 of 6 standards stated as supported, 12 sourced facts, dated plans and what it does not do." canonical: https://agentreadyhtml.com/vendors/firecrawl/ site: "Agent Ready HTML" last_updated: 2026-11-05 section: "Vendors" --- # Firecrawl: vendor profile: Agent Ready HTML > Firecrawl, markdown tooling: 0 of 6 standards stated as supported, 12 sourced facts, dated plans and what it does not do. Source: https://agentreadyhtml.com/vendors/firecrawl/ Site: Agent Ready HTML Last reviewed: 2026-11-05 # Firecrawl: vendor profile In short Firecrawl announced a $75M Series B on 2026-09-22 and bills 1 credit per page scraped or crawled. The free tier gives 1,000 credits a month, and Hobby costs $16 a month billed annually. A default cache of two days means a test of a fresh deploy needs maxAge set to 0, and robots.txt can be skipped only on Enterprise. ## At a glance **standards.txt** firecrawl ```text llms-txt = partial markdown-negotiation = no statement markdown-twin = no statement initial-html = no statement real-404 = no statement vary-accept = no statement ``` 0 supports, 1 partial or announced, 5 no statement. **Firecrawl** Markdown or llms.txt tool Paid web data API that fetches and renders pages and returns them as markdown, cleaned HTML, raw HTML or JSON, with crawl, map and search endpoints; it honours robots.txt by default and reports each page's HTTP status, so 404s stay visible. Type [Markdown or llms.txt tool](https://agentreadyhtml.com/vendors/by-type/markdown-tooling/) Home page [www.firecrawl.dev](https://www.firecrawl.dev/), answered HTTP 200 on 2026-11-05 Last read 5 November 2026 ## What it does for this layer - Firecrawl announced a $75M Series B on 2026-09-22, led by Smash Capital with Altos Ventures, Nexus Venture Partners and Y Combinator among the backers, alongside Alexandria, a layer that puts official data providers, custom connectors and Firecrawl's own indexes next to the live web. \[primary source, 2026-09-22\] [source](https://www.firecrawl.dev/blog/introducing-alexandria-series-b), read 5 November 2026. - Firecrawl's founders say they built Mendable, a documentation chat product, before Firecrawl, and that more than 1.5 million users now build with Firecrawl. \[vendor claim, 2026-09-22\] [source](https://www.firecrawl.dev/blog/introducing-alexandria-series-b), read 5 November 2026. - Under Firecrawl pricing effective 2026-09-04, the free tier gives 1,000 credits a month with no card, Hobby costs $16 a month billed annually ($19 monthly) for 5,000 credits, Standard $83 ($99) for 100,000, Growth $333 ($399) for 500,000 and Scale $599 ($749) for 1,000,000. \[primary source, 2026-09-04\] [source](https://www.firecrawl.dev/pricing), read 5 November 2026. - Firecrawl bills 1 credit per page scraped or crawled and per Map call, 2 credits per 10 search results, and 4 more per page for JSON, Question or Highlight output; a target page that answers 403 or 404 is still returned and still costs a credit. \[primary source, 2026-09-04\] [source](https://www.firecrawl.dev/pricing), read 5 November 2026. - Firecrawl's /scrape can return markdown, a summary, cleaned HTML, rawHtml exactly as the server sent it, Base64 of the original body, links, images, screenshots, JSON, product and branding extracts, so a caller can compare the server HTML with the converted markdown. \[primary source\] [source](https://docs.firecrawl.dev/features/scrape), read 5 November 2026. - Firecrawl serves a cached copy by default when it is younger than maxAge = 172,800,000 ms (two days), so a page changed yesterday can come back stale unless the caller sets maxAge to 0. \[primary source\] [source](https://docs.firecrawl.dev/features/scrape), read 5 November 2026. - Firecrawl's crawler obeys robots.txt and lists refused URLs as robotsBlocked; the ignoreRobotsTxt and robotsUserAgent overrides exist only on Enterprise plans. \[primary source\] [source](https://docs.firecrawl.dev/features/crawl), read 5 November 2026. - A Firecrawl crawl stays under the start path unless crawlEntireDomain is set, reads the sitemap as well as links by default (sitemap: "include"), and stops at 10,000 pages unless limit says otherwise. \[primary source\] [source](https://docs.firecrawl.dev/features/crawl), read 5 November 2026. - Crawl results from Firecrawl keep pages whose server answered with an error such as 404 and record the target's status in metadata.statusCode, which lets a site owner list every URL that returns a real 404 in one crawl. \[primary source\] [source](https://docs.firecrawl.dev/features/crawl), read 5 November 2026. - Firecrawl v2.10, released on 2026-05-15, added a /parse endpoint for uploaded documents, a Lockdown Mode and the Question and Highlights output formats. \[primary source, 2026-05-15\] [source](https://www.firecrawl.dev/changelog), read 5 November 2026. - The Firecrawl source repository on GitHub carries the GNU Affero General Public License version 3, which covers the self-hosted version separately from the paid cloud API. \[primary source\] [source](https://raw.githubusercontent.com/firecrawl/firecrawl/main/LICENSE), read 5 November 2026. - On 2026-11-05 docs.firecrawl.dev, a Mintlify-built docs site, served a 9,770-byte llms.txt, answered Accept: text/markdown on /features/crawl with 29,228 bytes of markdown and Vary: accept-encoding, Accept, User-Agent, served the same text at /features/crawl.md, and returned 404 for a made-up path. \[observed, 2026-11-05\] [source](https://docs.firecrawl.dev/llms.txt), read 5 November 2026. ## Products and plans | Product | What it is | Price | As of | Source | | --- | --- | --- | --- | --- | | Firecrawl API (cloud) | Scrape, crawl, map, search and related endpoints that return pages as markdown, HTML or JSON, billed in credits. | Free tier, paid plans above it; Free 1,000 credits/month; Hobby $16/mo billed annually ($19 monthly); Standard $83; Growth $333; Scale $599; Enterprise custom | 2026-11-05 | source | | Firecrawl (self-hosted) | The AGPL-3.0 source of Firecrawl run on your own infrastructure with Docker Compose. | Free, open source; AGPL-3.0 | 2026-11-05 | source | ## Standards | Standard | Status | What the vendor's pages say | Source | | --- | --- | --- | --- | | llms-txt | partial | The alpha /llmstxt endpoint still answers but is no longer maintained; Firecrawl points to the create-llmstxt-py script instead. | source | | markdown-negotiation | no statement | Firecrawl converts pages to markdown inside its own API; nothing in the scrape or crawl docs adds Accept handling to a customer's site. | none | | markdown-twin | no statement | No feature that publishes .md copies on a site; markdown is returned to the API caller only. | none | | initial-html | no statement | Firecrawl is a fetcher, not a site renderer; its rawHtml format shows what the server sent before rendering. | none | | real-404 | no statement | No statement about serving 404s; crawl results report each target page's status in metadata.statusCode. | none | | vary-accept | no statement | No statement found in the scrape, crawl and pricing pages read on 2026-11-05. | none | ## Checks on its own site | Reading | Result | Date | | --- | --- | --- | | Home page request | HTTP 200; H1 in the initial HTML: yes; 3,867 words; 0 JSON-LD blocks | 2026-11-05 | | Request with Accept: text/markdown | HTTP 200, text/markdown, Vary: Accept | 2026-11-05 | | /llms.txt at the host root | HTTP 200, text/plain, 22 KB, starts with an H1 | 2026-11-05 | | A made-up path | HTTP 404 | 2026-11-05 | | Ora score, cached | 85, grade B | 2026-11-05 | | Cloudflare keyless scan | 53, L5 Agent-Native | 2026-11-05 | | Shared crawl row | outside the ranked slice; home page HTTP 200; H1 in the HTML: yes; 4,052 words; JSON-LD types: none; answers markdown: yes; made-up path HTTP 404; llms.txt valid: yes | 2026-11-04 | ## What it does not do - Firecrawl does not make a site agent-readable on its own; the markdown it builds goes to the API caller and nothing changes on the site that was scraped. - Firecrawl's two-day default cache means a scrape is not always a live fetch, so testing a fresh deploy needs maxAge set to 0. ## Related pages on this site - [Markdown and llms.txt tooling in this section](https://agentreadyhtml.com/vendors/by-type/markdown-tooling/) - [Firecrawl llms.txt generator: platform profile and rubric scores](https://agentreadyhtml.com/platforms/firecrawl-llmstxt/) - [The Plain Fetch viewer](https://agentreadyhtml.com/viewer/) ## Sources 1. [Firecrawl: home page](https://www.firecrawl.dev/) https://www.firecrawl.dev/, read 5 November 2026 2. [Introducing Alexandria and our $75M Series B](https://www.firecrawl.dev/blog/introducing-alexandria-series-b) https://www.firecrawl.dev/blog/introducing-alexandria-series-b, read 5 November 2026 3. [Firecrawl pricing](https://www.firecrawl.dev/pricing) https://www.firecrawl.dev/pricing, read 5 November 2026 4. [Scrape](https://docs.firecrawl.dev/features/scrape) https://docs.firecrawl.dev/features/scrape, read 5 November 2026 5. [Crawl](https://docs.firecrawl.dev/features/crawl) https://docs.firecrawl.dev/features/crawl, read 5 November 2026 6. [Firecrawl changelog](https://www.firecrawl.dev/changelog) https://www.firecrawl.dev/changelog, read 5 November 2026 7. [Firecrawl LICENSE](https://raw.githubusercontent.com/firecrawl/firecrawl/main/LICENSE) https://raw.githubusercontent.com/firecrawl/firecrawl/main/LICENSE, read 5 November 2026 8. [docs.firecrawl.dev llms.txt, markdown and 404 checks (observed)](https://docs.firecrawl.dev/llms.txt) https://docs.firecrawl.dev/llms.txt, read 5 November 2026 9. [LLMs.txt API (alpha)](https://docs.firecrawl.dev/features/alpha/llmstxt) https://docs.firecrawl.dev/features/alpha/llmstxt, read 5 November 2026 Last reviewed 5 November 2026 Edition Crawl 2026-11-04 Page updated 2026-11-05 Method How the viewer reads a page, how a file is graded and how a platform is scored: [the method page](https://agentreadyhtml.com/method/).