Question 1

What is llms.txt?

Accepted Answer

A markdown file at /llms.txt proposed by Jeremy Howard on 2024-09-03. It holds an H1 with the site name (the only required part), an optional blockquote summary, optional detail text, and H2 sections listing links to LLM-friendly pages. Version 2 dates from August 2026.

Question 2

What changed in llms.txt version 2?

Accepted Answer

v2 adds rel="alternate" type="text/markdown" and rel="describedby" link relations, allows page.md as well as page.html.md, defines subpath files (the most specific applies), removes the llms_txt2ctx tooling, and drops the special tool meaning of the Optional section.

Question 3

Is llms-full.txt part of the llms.txt proposal?

Accepted Answer

No. The llmstxt.org text read on 2026-11-04 does not define llms-full.txt. It is a convention platforms added: Mintlify, GitBook, Scalar and Shopify all serve it, usually as the full site markdown in one file.

Question 4

Does llms.txt help me get cited by ChatGPT?

Accepted Answer

No published evidence shows that. Ora's lab found (2026-10-19) that llms.txt changed nothing about whether a plain-fetch or JavaScript client retrieved an answer already in the HTML. Agents did use the file when linked: 36 percent of runs that reached it grounded the answer in its content.

Question 5

Does Google use llms.txt?

Accepted Answer

Google Search says it does not need llms.txt and that it will not help or hurt rankings; its AI features guide says no new machine-readable files are required. Chrome Lighthouse separately audits llms.txt in its Agentic Browsing category.

Question 6

Where should llms.txt be linked from?

Accepted Answer

From pages agents already read: the footer, a link rel element in the head, or an HTTP Link header. In the Vercel and Ora data, 86 percent of llms.txt fetches arrived through a link, not a guessed path.

Question 7

What does "answer in the initial HTML" mean?

Accepted Answer

The text that answers the question must be in the HTML the server returns, before scripts run. In Ora's controlled test, hiding the answer in JavaScript left the plain-fetch client with 0 of 5 answers.

Question 8

How do I see my page the way an agent does?

Accepted Answer

Fetch it without JavaScript: curl -iL -A "Claude-User/1.0" https://example.com/page. The response should end in 200 and the body should contain the answer. Check server logs too, because bot protection judges source IPs, not only user agents.

Question 9

Which agents ask for markdown?

Accepted Answer

In Checkly's February 2026 test, Claude Code, Cursor and OpenCode sent Accept: text/markdown; OpenAI Codex, Gemini CLI, GitHub Copilot and Windsurf did not. Ora's harnesses requested markdown on 65.1 percent of fetches.

Question 10

How many sites support markdown negotiation?

Accepted Answer

Cloudflare Radar found 3.9 percent of scanned top sites passing markdown negotiation at the April 2026 launch of isitagentready.com. The crawl for this site asked 8,802 home pages that answered 200 for markdown, and 798 of them (9.1 percent) sent it (edition 2026-11-04).

Question 11

What headers should a markdown response carry?

Accepted Answer

Content-Type: text/markdown; charset=utf-8 and Vary: Accept so caches keep HTML and markdown apart. If markdown is unavailable, return HTML with its own content type.

Question 12

Is serving markdown to agents cloaking?

Accepted Answer

Content negotiation returns the same content in another format for the same URL, which is standard HTTP. Serving different content to AI user agents is a different practice; Scrunch documents that its AXP version can be seen by spoofing a targeted user agent.

Question 13

Does Cloudflare Markdown for Agents work on the Free plan?

Accepted Answer

The documentation read on 2026-11-04 lists Pro, Business and Enterprise plans and SSL for SaaS customers, at no extra cost. Conversion is limited to 6 MiB of decompressed HTML.

Question 14

Does Cloudflare keep JSON-LD when converting to markdown?

Accepted Answer

Yes. JSON-LD blocks are appended at the end of the markdown in one fenced json block, and all other script and style content is stripped.

Question 15

Do agents read JSON-LD?

Accepted Answer

Converters such as Cloudflare keep it, but the Vercel and Ora study did not measure an accuracy gain from JSON-LD and recommends repeating critical facts in visible text because some converters drop script blocks.

Question 16

How often do agents fetch sitemap.xml?

Accepted Answer

Directly in 4 percent of runs (43 of 1,033) in the Vercel and Ora data. Sitemaps still help crawlers and search systems list canonical pages.

Question 17

How often do agents fetch openapi.json?

Accepted Answer

On seven sites that shipped one, agents fetched it in 112 of 541 runs, or 21 percent, and 97 percent of attributed fetches arrived through a link.

Question 18

Why should a missing page return 404?

Accepted Answer

A 200 for a missing page removes the signal that separates missing content from a live page. Google also skips rendering for most non-200 pages and documents ways to avoid soft 404s in single-page apps.

Question 19

Which docs platforms generate llms.txt automatically?

Accepted Answer

Mintlify, GitBook, ReadMe, Fern, Redocly and Scalar Docs all generate it by default per their documentation read on 2026-11-04. Docusaurus needs a community plugin; no llms.txt was found on Stoplight docs.

Question 20

Does Webflow generate llms.txt?

Accepted Answer

No. Webflow accepts an uploaded UTF-8 file under 100 KB in Site settings, SEO, serves it at the root and keeps it out of search indexes. Markdown conversion for agents is an Enterprise setting since 2026-04-02.

Question 21

Does Framer serve markdown?

Accepted Answer

Yes. Framer serves a markdown version of every page on the same URL when the request sends Accept: text/markdown, and in a browser with a md query parameter, with no setup.

Question 22

Does Shopify have llms.txt?

Accepted Answer

Shopify manages /agents.md for every store and mirrors it at /llms.txt and /llms-full.txt. Theme developers can override with agents.md.liquid or llms.txt.liquid.

Question 23

How does Yoast pick pages for llms.txt?

Accepted Answer

By default the 5 most recently updated posts, pages or custom post types, with posts only if published in the last 12 months. The file is refreshed weekly, and an existing llms.txt on the server blocks generation.

Question 24

Is the Firecrawl llms.txt API still maintained?

Accepted Answer

Firecrawl marks its /llmstxt endpoint as deprecated: still active, but not maintained after 2025-06-30. It points users to the open-source create-llmstxt-py script.

Question 25

How big should llms.txt be?

Accepted Answer

Small enough to fit an agent context. Mintlify splits generated indexes above 100,000 characters, and Cloudflare publishes one llms.txt per docs directory because one file for 5,000+ pages would exceed context windows.

Question 26

Should every page link to its markdown twin?

Accepted Answer

Linking helps discovery. llms.txt v2 defines rel="alternate" type="text/markdown" for that, in HTML or an HTTP Link header, so a CDN can add it without changing markup.

Question 27

Do answer assistants use markdown twins?

Accepted Answer

One operator log study (4 to 18 September 2026) saw ChatGPT, Claude on the web and Perplexity fetch HTML over 13,000 times and markdown once while answering people, while coding agents asked for markdown about 22,000 times.

Question 28

What does a JavaScript-only home page look like to a plain fetch?

Accepted Answer

A title and a nearly empty body with a root div. In the crawl, 307 of 8,816 home pages that answered 200 (3.5 percent) had fewer than 40 words and an app root div (edition 2026-11-04).

Question 29

What does Ora check on this layer?

Accepted Answer

Ora's Access layer is worth 30 of 100 points and runs 42 checks on whether an agent can access and understand a site. Cloudflare isitagentready.com and mintlify.com/score cover overlapping ground.

Question 30

How can I test readability in CI?

Accepted Answer

Fetch key URLs without JavaScript and assert the answer text, a 404 on a made-up path, and text/markdown on an Accept request. AFDocs ships a vitest helper for docs sites.

Questions and answers

llms.txt and llms-full.txt