Agent Ready HTML

API and MCP server

In short

The llms.txt Index, the crawl measures and the platform scores are also data an agent can query: an MCP server at /mcp with five read-only tools, a REST API under /api/v1 described by openapi.json, and an NLWeb /ask endpoint. Every answer names the page it comes from, and nothing needs a credential.

Connect the MCP server

The server speaks the Streamable HTTP transport at https://agentreadyhtml.com/mcp. Send JSON-RPC 2.0 messages with POST and read the JSON answer. There is no session to open and no credential to send, so one request gets one answer. An MCP client that takes a URL connects with that address alone; the server card and the manifest say the same thing in the forms clients read.

curl -s https://agentreadyhtml.com/mcp -H 'Content-Type: application/json' -H 'Accept: application/json, text/event-stream' -d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-06-18","capabilities":{},"clientInfo":{"name":"my-agent","version":"1.0"}}}'
curl -s https://agentreadyhtml.com/mcp -H 'Content-Type: application/json' -H 'Accept: application/json, text/event-stream' -d '{"jsonrpc":"2.0","id":2,"method":"tools/call","params":{"name":"get_llms_entry","arguments":{"domain":"supabase.com"}}}'

A tool answers with the result as structuredContent and the same JSON as text. A wrong argument answers isError: true with the parameter list, and an unknown tool answers JSON-RPC error -32602. The server also lists five resources with resources/list: the page index, the methodology, this page and two data files, readable with resources/read.

The five tools

Every tool is read-only (readOnlyHint true, destructiveHint false) and answers from the data files of edition 2026-11-04.
ToolWhat it returnsParameters
search_llms_indexLists llms.txt files from the index (domain, Tranco rank, category, platform, grade A to F, size, H1, link counts, whether the home page links it, page URL). Filter by words in the domain or H1, grade, category, platform, minimum links or linked_from_home. Call it to find which sites serve an llms.txt and how it grades; use get_llms_entry for one domain in full.q?: Words to look for in the domain or the H1 of the file; every word must appear, for example "stripe" or "developer docs".
grade?: Structure grade: A passes all six checks of the llmstxt.org structure, F two or fewer.
category?: Category of the domain as the index lists it, for example "Developer Tools". grade_distribution with group_by category lists them all.
platform?: Platform the home page declares in its generator tag, for example WordPress, Framer or Astro.
min_links?: Keep files that list at least this many links.
linked_from_home?: true keeps files the home page links; false keeps files it does not link. Files whose home page was not read are left out of both.
get_llms_entryReturns one domain from the index: its row (grade, checks passed, size, H1, links, whether the home page links the file), what the grade means, and, when the domain has an entry page, the parsed summary, H2 sections, link hosts and the six structure checks with their detail. Call it after search_llms_index, or with any domain you are checking.domain: A domain such as stripe.com, or a URL on it; a leading www. is ignored if the domain is not listed with it.
grade_distributionReturns how many llms.txt files earned each grade (A to F), overall and grouped by category or by platform, with the share that earned an A. Call it for adoption and quality questions across the index, or to list the category and platform names that search_llms_index accepts.group_by?: category (default), platform, or none for the overall counts only.
min_files?: Leave out groups with fewer files than this; platform names read from generator tags are many and small, so 5 is a useful floor.
get_crawl_measureLists the crawl measures (count, denominator and percent) for home pages that answered 200: headline figures, by category and by platform, for example the share that serve llms.txt, answer Accept: text/markdown with markdown or return 200 for a made-up path. Filter by group_type, group and words of the measure. Call it for how common a practice is.group_type?: headline is the whole crawl; category and platform split it.
group?: The group name, for example "Developer Tools" (a category) or "WordPress" (a platform); case-insensitive. Omit group_type to match the name in any type.
measure?: Words from the measure name; every word must appear, for example "llms.txt" or "markdown" or "soft 404".
list_platformsLists the 23 platforms (docs platforms, site builders, frameworks, edge products) with their total on the nine-key rubric and each key score (2 on by default, 1 with setup, 0 not offered, n/a). With id, returns one platform with the evidence, source URL and read date for every key; with key, adds that key's evidence to each platform. Call it to compare platforms or to cite one score.id?: Platform id for one platform in full, for example astro, nextjs-app-router or mintlify.
type?: Keep one kind of platform.
key?: A rubric key to rank by and to show the evidence for, for example markdown_negotiation.

search_llms_index, get_crawl_measure, list_platforms are paginated: they take limit (1 to 100, default 20) and the next_cursor of the previous page as cursor, and answer with items, total and next_cursor.

The REST API

Each tool is also one GET endpoint under /api/v1/, with the parameters in the query string and the same JSON in the answer. /api is the index. /openapi.json describes every operation with its parameters, response schema and the Error schema, and /.well-known/api-catalog links that description the way RFC 9727 asks. The examples below return real results.

curl -s 'https://agentreadyhtml.com/api/v1/get_llms_entry?domain=supabase.com' | jq '{grade, links, linked_from_home}'

Ask in plain words

The /ask endpoint follows NLWeb 0.55. It searches the llms.txt entries and rows, the crawl measures, the platform scores and every page listed in /llms.txt, and returns schema.org objects, each with a grounding.source URL to cite. GET takes query; POST takes the NLWeb request object. Add streaming=true, or prefer.streaming in the body, for server-sent events named start, result and complete. mode=summarize adds one sentence assembled from the results, never written by a model.

curl -s 'https://agentreadyhtml.com/ask?query=markdown+negotiation+on+Next.js'
curl -s https://agentreadyhtml.com/ask -H 'Content-Type: application/json' -d '{"query":{"text":"markdown negotiation on Next.js"},"prefer":{"mode":"list"},"meta":{"version":"0.55"}}'

A query that matches nothing answers _meta.response_type: failure with code NO_RESULTS, and a missing query answers 400 with INVALID_QUERY. A context object is accepted and ignored, because the answers come from the data and not from a conversation.

Data files behind the tools

The tools read these published files at request time, so a tool answer and the page it cites come from the same edition. A reader who wants everything can download the files whole instead of paging the API.

Discovery files

  • /openapi.json: OpenAPI 3.1 for the REST API and /ask, linked from every page as rel=service-desc and from the Link header.
  • /.well-known/api-catalog: an RFC 9727 linkset that points to the descriptions of the API, the MCP server and the ask endpoint.
  • /mcp/server-card: the MCP server card with name, version, remote URL and protocol versions, with a copy at /.well-known/mcp/server-card.json.
  • /.well-known/mcp.json: the manifest with the endpoint, the transport and the tools with their input schemas.
  • /.well-known/ai-catalog.json: AI Catalog 1.0 with entries for the MCP card, the OpenAPI document, the ask endpoint and every document; the same file is at /.well-known/ard.json.
  • /schemamap.xml: the schema feeds, which carry the entries, rows, platforms and measures as JSON-LD lines under /feeds/ and are named in robots.txt.

Access, limits and errors

Credential
None. The OpenAPI document declares security: [] with no security schemes, and auth.md says the same. The site publishes no OAuth metadata because no endpoint takes a token; metadata for an authorization server that does not exist would point agents at a door that is not there.
Rate limit
120 requests a minute per client address, counted per running instance and announced in RateLimit and RateLimit-Policy headers. A 429 carries Retry-After, and the host adds limits of its own above that. The whole index is one download from /data/, so bulk readers never need the API.
Errors
JSON, never HTML: {"error":{"code","message","hint","status","docs"}} with the codes invalid_params, not_found, method_not_allowed, rate_limited and internal. A missing endpoint under /api answers 404 and lists the endpoints in the hint.
Writes
There are none, so every call is safe to retry and no idempotency key exists. Every call answers in one response, so there is no job to poll.
CORS
Every answer carries Access-Control-Allow-Origin: *, so an agent that runs in a browser can call the API and the MCP server.

Versioning

The path prefix /api/v1 is the contract. A breaking change, such as a field removed or renamed or a tool removed, ships under /api/v2 while /api/v1 keeps answering for at least 180 days, announced on this page and in the info.version of openapi.json. Additive changes, such as a new field or a new tool, keep the prefix. The MCP server follows the same rule and carries the same version number in its card and in serverInfo.version. The data behind every answer is edition 2026-11-04; the edition is in the X-Dataset-Edition header and in every tool result.

Cite the data

Every result names the page or file it comes from; cite that URL. The llms.txt Index and the crawl datasets are licensed CC BY 4.0 under the terms, and the other text and data may be quoted with attribution and a link to the page. The citation line for the data is on the data page.

What the surface does not do

It does not write, hold accounts, fetch other sites or generate text with a model. For a live look at one URL, use the Plain Fetch viewer; the tools answer from the crawl of 2026-11-04, which can differ from what a site serves today. The pages stay the record, and the methodology explains how the rows are made.

Sources

  1. MCP specification, Streamable HTTP transport https://modelcontextprotocol.io/specification/2025-06-18/basic/transports, read 4 November 2026
  2. NLWeb specification v0.55 https://github.com/nlweb-ai/website/blob/main/NLWEBSPEC.md, read 4 November 2026
  3. RFC 9727, api-catalog https://www.rfc-editor.org/rfc/rfc9727, read 4 November 2026
  4. OpenAPI 3.1.0 https://spec.openapis.org/oas/v3.1.0, read 4 November 2026
  5. Schema Feeds specification 0.1 https://github.com/nlweb-ai/website/blob/main/SCHEMA_SPEC.md, read 4 November 2026

Last reviewed 4 November 2026