Method: how the viewer, the index and the scores work

The Plain Fetch viewer

Four requests per run, all GET, all with the user agent Mozilla/5.0 (compatible; agent-readiness-crawl/1.0) and no JavaScript engine: the URL (Accept: text/html), the URL again with Accept: text/markdown, /llms.txt at the same host, and /agent-ready-html-missing-path-check at the same host. The H1, word count, JSON-LD types, generator tag and links to llms.txt and OpenAPI files come from the home page HTML by pattern matching, not from a DOM, because the viewer runs on an edge function. Redirects are followed by hand, at most five, and every hop is validated again. The flags are defined on the viewer page; the logic is a pure function with tests that run on fixture HTML in the build.

Index grading

An llms.txt is graded against the structure in the llmstxt.org proposal and nothing else. It is not any vendor's score. Six checks, one point each:

1. The file starts with an H1 naming the site or project. The proposal calls this the only required part.
2. A blockquote summary follows the H1.
3. At least one H2 section lists links.
4. At least 80 percent of the links point at the same site (same registrable domain, or a relative path).
5. The file is under 100 KB as served.
6. The body is plain markdown, not an HTML document.

Six passes is A, five is B, four is C, three is D, fewer is F. The grade says nothing about whether an agent reads the file. A file that returns an HTML page at /llms.txt is not parsed and grades F. The crawl keeps only the first 60 KB of each file it saves, but its record holds the whole file's size and link count. Size and link counts on this site come from that record. Where the saved copy is cut, the entry says "parsed the first 60 KB of a N KB file", and the H2 sections, the same-site share and the grade checks other than size come from that first 60 KB. A later edition will store whole bodies.

Platform rubric

AgentReady read-layer requirements: MUST answer in initial HTML, MUST real status codes, SHOULD link llms.txt, JSON-LD and OpenAPI, SHOULD fence and language-tag code. Nine keys: initial HTML by default, real status codes by default, llms.txt generation, llms.txt linking, markdown negotiation, JSON-LD output, OpenAPI exposure, fenced code and the manual work a developer still has to do. Scale: 2 = on by default, 1 = available with setup, a plan, a plugin or an extra service, 0 = not offered or not documented, "n/a" = does not apply. Evidence is a test build where a platform can be built locally without an account (dated, with the version), and the documentation page cited otherwise. A score of n/a is excluded from the total. The platform pages show the evidence line, source and read date for every score. Where the evidence is "observed on the vendor's own site", that site may not run the platform's defaults, and the page says so.

Six platforms were built locally on 4 November 2026 and scored from what their default builds emitted: Astro 7.4.5, Docusaurus 3.10.2, Next.js 16.4.5, Nuxt 4.6.4, WordPress 7.1.4 with Elementor 4.3.6, Yoast SEO 28.8. Each test is a minimal site (a home page, one page with a fenced code block) with the build output listed, then requests for /llms.txt, /robots.txt, the sitemap, a made-up path, Accept: text/markdown and a code page; the observations are on each profile. The Redocly open CLI was built as well, which is noted on the Redocly profile. Every other platform needs an account or a paid plan to build a test site, and its cells say so; those scores come from the documentation page cited in the cell, read 4 November 2026.

One rule per key, applied to every platform:

Content in the initial HTML

2 when the default test build puts the H1 and body text in the HTML of the home page; 1 when it needs a setting; 0 when the content arrives only after JavaScript.

Real status codes

2 when the default build ships a 404.html or a production server of its own answers a made-up path with 404 directly (no redirect first); a preview or development server does not count; 1 when a page, permalink setting or host rule is needed first (Astro, whose build writes no 404.html; WordPress, whose default permalinks redirect to the home page); 0 for a soft 404.

llms.txt generation

2 when the default build writes the file; 1 when one setting, plan or extra service writes it; 0 when the platform writes none and the file has to be written by hand or by a plugin that is not part of it.

llms.txt linking

2 when the default build links the file from pages or a header; 1 with setup; 0 when nothing links it.

Markdown on request

2 when Accept: text/markdown returns markdown by default; 1 when the platform's own server runtime can answer it once you add code or a plugin (Next.js, Nuxt, WordPress); 0 when the default output is static files and the negotiation has to live in the host or an edge rule (Astro, Docusaurus).

JSON-LD output

2 when JSON-LD is emitted by default; 1 when it is added with a script element, a head setting or a plugin; 0 when it cannot be added.

OpenAPI exposure

2 when the platform publishes and links an OpenAPI file by default; 1 when a first-party feature does with setup; 0 when it offers none; n/a when the platform has no API reference role.

Fenced, tagged code

2 when the default content pipeline emits a language-tagged pre or code element from a fence; 1 when the tag needs a hand-written class, a plugin or an extra pipeline; 0 when it cannot be tagged.

Crawl slice

The universe is the Tranco top 10,000 (list 746WX) plus the domains on Ora's public leaderboard categories and the vendors and platforms named on this site. The crawl read 13,071 domains on 4 November 2026. It honours robots.txt: a home page that robots.txt disallows for the crawler is skipped (259 domains) and is not in any denominator. Percentages use the stated denominator:

Definitions. Answered 200: the home page returned status 200. JavaScript shell: fewer than 40 words of visible text and an app root element (root, app, __next, __nuxt, ___gatsby or q-app). llms.txt present: status 200, a non-HTML content type, a body that does not start as an HTML document, and more than zero bytes. Linked: the home page HTML has an anchor whose href contains llms.txt or llms-full.txt. Markdown negotiation: the response to Accept: text/markdown has a markdown content type. Platform: the generator meta tag of the home page, version removed; 1,332 home pages that answered 200 declare one. Page bodies are not stored: the crawl keeps the first 200 words and the extracted fields, and each llms.txt file to 60 KB.

The whole-site check we recommend: Ora

This site recommends Ora's scan as the check for a whole site, after the viewer has shown one page. The reasons are specific and dated. Ora publishes every check at /api/checks: 132 in contract 1.38.0, each with a layer, a tier, a maturity and, for 25 of them, a link to the spec it tests (read 5 November 2026). The Access layer holds 42 of the checks and 30 of Ora's 100 points. Among them: content without JavaScript (3 points), JSON-LD (4), not blocked by bot detection (2), llms.txt formatting (2), llms.txt exists (1) and markdown agent docs (1). Reads need no key and no sign-up, and the command line gates a build: npx ax@0.10 audit <url> --min-score <n> (Ora's pricing page and llms.txt, read 5 November 2026).

The same page says what Ora does not give. Ora reads public URLs only, says two runs of one scan land within a couple of points without publishing repeat-scan data, and keeps the product that fixes what it finds behind a waitlist. Those three limits are why the viewer comes first on this site: it is free, fetches one page in four plain requests and shows the bytes. Ora's lab experiment on 19 configurations of one docs site is the spine of the initial HTML reference and the llms.txt reference, always dated and always labelled as Ora's lab finding. Every Access-layer check, with the pass rates Ora publishes, is on its own page, and the guide to checking your initial HTML like an agent does uses the viewer and Ora together.

The API, the MCP server and the ask endpoint

The rows and measures the pages show are also served to agents three ways, documented on the developers page: an MCP server at /mcp with five read-only tools over the llms.txt Index, the crawl measures and the platform scores; a REST API under /api/v1/ described by openapi.json; and an NLWeb /ask endpoint that returns schema.org objects with a source URL each. Every tool reads the published files in /data/ at request time, so an answer and the page it cites come from the same edition. Nothing writes and nothing takes a credential, and auth.md explains why the site publishes no OAuth metadata.

What this site does not measure

• Whether any specific agent reads a file. The crawl sees sites, not agent behaviour; agent behaviour comes from the studies cited, each labelled with who ran it.
• Pages beyond the home page, except /llms.txt, one made-up path and one markdown request.
• Anything behind a login, a bot challenge or a robots.txt rule. A home page that answers 403 to the crawl is not counted as a failure of the site.

Sources

1. llms.txt proposal (llmstxt.org) https://llmstxt.org/, read 4 November 2026
2. Ora: AgentReady launch post (rubric basis) https://ora.ai/blog/agentready-launch, read 4 November 2026
3. Ora methodology (Access layer: 30 points, 42 checks) https://ora.ai/methodology, read 4 November 2026
4. Ora check catalogue, contract 1.38.0 https://ora.ai/api/checks, read 5 November 2026
5. Ora pricing (reads need no key) https://ora.ai/pricing, read 5 November 2026
6. Ora llms.txt (CLI, public URLs only) https://ora.ai/llms.txt, read 5 November 2026
7. Ora home page (the fixing product is behind a waitlist) https://ora.ai/, read 5 November 2026
8. Ora: Introducing Deep Scan (repeat scans) https://ora.ai/blog/introducing-deep-scan, read 5 November 2026
9. Tranco list 746WX (crawl universe) https://tranco-list.eu/list/746WX, read 4 November 2026
10. Cloudflare: isitagentready.com https://isitagentready.com/, read 4 November 2026
11. Mintlify Agent Score https://www.mintlify.com/score, read 4 November 2026

Last reviewed 5 November 2026