Agent Ready HTML

The Plain Fetch viewer

In short

The viewer requests a URL the way a plain-fetch agent does: no JavaScript, one request for HTML, one for markdown, one for llms.txt and one for a path that should not exist. It shows the initial HTML on the left, the readable text on the right, and seven flags underneath.

Pre-loaded with www.wikipedia.org, read 4 November 2026. Type any public URL to run your own.

GET https://rapid7.com/ answered 200 with text/html, 733 KB, 1 redirect, read 9 November 2026, 14:07 UTC. Final URL: https://www.rapid7.com/.

Permalink: /viewer/?url=https://rapid7.com/

The page answers a plain fetch: 4 of 6 checks found.

Initial HTML what a plain fetch receives

<!DOCTYPE html>
<html lang="en">
<head>
<meta charSet="utf-8"/>
<meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=2, user-scalable=yes"/>
[... 46 more head elements folded ...]
<title>Rapid7 Managed Cybersecurity: Outpace Attackers
</title>
<meta name="description" content="Outpace attackers with the only endpoint to cloud, unified cybersecurity platform. Build business resilience with expert-led 24/7 MDR."/>
[... 3 more head elements folded ...]
<meta name="generator" content="Next.js"/>
[... 2 more head elements folded ...]
<meta name="robots" content="index, follow"/>
[... 2 more head elements folded ...]
<link rel="canonical" href="https://www.rapid7.com/"/>
<link rel="alternate" type="application/rss+xml" href="https://www.rapid7.com/rss.xml" title="RSS Feed"/>
[... 13 more head elements folded ...]
<link href="https://www.rapid7.com/" hrefLang="en" rel="alternate"/>
<link href="https://www.rapid7.com/de/" hrefLang="de" rel="alternate"/>
<link href="https://www.rapid7.com/ja/" hrefLang="ja" rel="alternate"/>
[... 2 more head elements folded ...]
</head>
<body class="inter_980ebfdc-module__g2y1eq__className inter_980ebfdc-module__g2y1eq__variable antialiased">
<script>[... 527 characters folded ...]
</script>
<script id="ld-home" type="application/ld+json">[... 763 characters folded ...]
</script>
<div class="flex h-full flex-1 flex-col homepage">
<script data-promo-banner-early-state="" data-promo-banner-expiry-date="2026-11-14" data-promo-banner-publish-time="2026-10-21T12:56:31.308Z" data-promo-banner-title="The Quarterly Threat Landscape Report is out. See wha [... 64 characters folded ...]>[... 952 characters folded ...]
</script>
<div data-header-chrome-root="true" class="fixed top-0 z-50 w-full flex-col transition-transform duration-300 ease-out">
<div class="w-full overflow-hidden bg-gray-slate" id="promo-banner" style="height:auto;opacity:1">
<div class="mx-auto flex items-center px-5 py-[6px] md:py-[10px]">
<div class="hidden lg:block lg:flex-1">
</div>
<div class="flex min-w-0 flex-grow items-center space-x-3 lg:justify-center"><img alt="ico-alert-white.svg" loading="lazy" width="30" height="30" decoding="async" data-nimg="1" class="hidden flex-shrink-0 md:block" style="color:transparent" src="https://www.rapid7.com/cdn/images/bltb045a64c5450e057/69b2b07db4b5c8232aa26f9d/ico-alert-white.svg"/>
<div class="min-w-0 flex-grow overflow-hidden">
<div class="flex min-w-0 items-center">
<div class="flex min-w-0 flex-wrap items-center space-x-[10px]"><span class="min-w-0 flex-grow overflow-hidden text-ellipsis text-[14px] md:text-[16px] whitespace-normal font-normal leading-[120%] text-white md:leading-[160%]">The Quarterly Threat Landscape Report is out. See what attackers are targeting now.<a class="hover:text-white-pearl/90 flex-shrink-0 text-[14px] font-normal md:text-[16px] pl-1 leading-[160%] text-white underline transition-colors" data-gtm-cta="read-report" data-gtm-location="top-promo" rel="noopener" target="_self" href="/research/report/quarterly-threat-landscape-report/">Read report </a></span>
</div>
</div>
</div>
</div>
<div class="ml-2 flex-shrink-0 md:ml-0"><button aria-label="Dismiss the promotional banner" class="rounded-full transition-colors hover:bg-white/20">
<svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-x h-[30px] w-[30px] font-normal text-white"><path d="M18 6 6 18"></path><path d="m6 6 12 12"></path>
</svg></button>
</div>
</div>
</div>
<div class="navigation:hidden">
<header class="flex w-full flex-col navigation:hidden" data-testid="main-mobile-header-navigation">
<nav aria-label="Primary navigation" class="z-[1] flex flex-col bg-white">
<div class="flex w-full items-center px-5 h-[48px]"><span aria-hidden="true" class="invisible h-6 w-6 py-[14px]"></span><span class="contents"><a class="flex flex-shrink-0 items-center justify-center mx-auto h-full px-[14px] hover:opacity-80" data-gtm-cta="rapid7-logo" data-gtm-location="menu" target="_self" rel="noopener" href="/"><img alt="Rapid7" draggable="false" width="122" height="20" decoding="async" data-nimg="1" style="color:transparent" src="/_next/static/media/rapid7-logo.2ueoguhn_hq76.svg"/></a></span><button aria-controls="mobile-navigation-menu" aria-expanded="false" aria-label="Toggle menu" class="header-burger-icon py-[14px]" data-state="closed" data-testid="mobile-toggle-menu-button"><span></span><span></span><span></span></button>
</div>
<div class="flex flex-col overflow-hidden invisible pointer-events-none py-0" data-state="closed" id="mobile-navigation-menu" style="height:0px;opacity:0">
<div class="px-5 py-[10px]"><button class="whitespace-nowrap text-[15px] font-medium normal-case transition duration-500 ease-out border-solid disabled:cursor-not-allowed disabled:opacity-50 text-gray-slate hover:border-gray-charcoal hover:text-bla [... 411 characters folded ...]>
<div class="flex items-center gap-2">
<svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-search h-6 w-6"><circle cx="11" cy="11" r="8"></circle><path d="m21 21-4.3-4.3"></path>
</svg><span class="text-[18px] leading-[21.6px] text-black-rich">Search</span>
</div></button>
</div>
<div class="flex h-full max-h-[78dvh] w-full flex-col overflow-y-auto">
<div class="flex flex-col">
<div data-orientation="vertical">
<div data-state="closed" data-orientation="vertical" class="m-0 flex list-none flex-col w-full">
<h3 data-orientation="vertical" data-state="closed" class="sticky top-0 z-[2] flex bg-white" data-component="accordion"><button type="button" aria-expanded="false" data-state="closed" data-orientation="vertical" id="radix-_R_mnahklukivb_" class="group sticky top-0 flex w-full items-center justify-between gap-[10px] p-5 no-underline font-p [... 207 characters folded ...]><span>Platform</span>
<svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-chevron-right h-5 w-5 text-gray-cadet duration-200 group-data-[state=open]:rotate-90"><path d="m9 18 6-6-6-6"></path>
</svg></button>
</h3>
<div data-state="closed" id="radix-_R_4mnahklukivb_" role="region" aria-labelledby="radix-_R_mnahklukivb_" data-orientation="vertical" class="group/mobile-accordion isolate overflow-hidden bg-white-flash data-[state=clos [... 186 characters folded ...]>
<div class="grid duration-500 ease-[cubic-bezier(0.4,0,0.2,1)] group-data-[state=closed]/mobile-accordion:grid-rows-[0fr] group-data-[state=open]/mobile-accordion:grid-rows-[1fr] transition-[grid-template-rows,opacity] g [... 106 characters folded ...]>
<div class="min-h-0 overflow-hidden">
<div class="flex w-full flex-col pb-5 pt-5">
<div class="flex h-full w-full flex-col"><a class="h-fit no-underline" data-gtm-cta="platform-explore-platform" data-gtm-location="menu" target="_self" rel="noopener" href="/platform/">
<div class="flex h-fit flex-col gap-3 rounded-lg hover:bg-navigation-hover px-5 py-4 lg:px-[10px] lg:py-[6px]">
<div class="flex flex-col gap-[6px]">
<div class="text-[26px] font-semibold leading-[28.6px] text-gray-charcoal lg:text-[28px] lg:leading-[30.8px]">The Command Platform
</div>
<div class="flex flex-col [&amp;_*+h2]:mt-[50px] [&amp;_h2]:mb-[18px] [&amp;_*+h3]:mt-[20px] [&amp;_h3]:mb-[12px] [&amp;_*+h4]:mt-[20px] [&amp;_h4]:mb-[12px] [&amp;_*+h5]:mt-[20px] [&amp;_h5]:mb-[12px] [&amp;_p:not(:last [... 1045 characters folded ...]>
<p><span style="color:rgb(59, 69, 74);font-size:undefined"></span>
</p>
<p><span style="color:rgb(59, 69, 74);font-size:undefined">Respond faster and outpace attackers. Taking command of your attack surface starts here. </span>
</p>
</div>
</div>
<div class="group flex w-full items-center">
<div data-testid="desktop-navigation-main-item_cta_default_link_0" class="mt-0 inline-flex items-center border-[1.5px] border-transparent cta gap-2 rounded-md pb-2 pt-2 text-gray-charcoal transition-all duration-500 ease [... 881 characters folded ...]><span class="flex items-center gap-x-[8px] leading-[normal]">Explore platform
<svg xmlns="http://www.w3.org/2000/svg" width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-arrow-right" aria-hidden="true" focusable="false"><path d="M5 12h14"></path><path d="m12 5 7 7-7 7"></path>
</svg></span>
</div>
</div>
</div></a><span class="my-5 h-[1px] px-5"><hr class="h-4 w-full text-gray-silver-sand"/></span>
</div>
<div class="flex h-full w-full flex-col">
<div class="h-full w-full relative flex flex-col">
<div class="eyebrow sticky top-[-1] z-[1] bg-white-flash text-gray-cadet px-5 py-[14px]">Capabilities
</div>
<ul class="m-0 flex list-none flex-col px-1 py-[2px]" data-testid="header-submenu-column-Capabilities">
<li><a class="p-4 flex items-center justify-between group relative rounded-lg border border-transparent no-underline transition-all duration-200 ease-in-out hover:bg-navigation-mobile-hover overflow-hidden" data-gtm-cta="pla [... 144 characters folded ...]>
<div class="flex flex-1 flex-col gap-1 overflow-hidden truncate">
<div class="flex items-center gap-1 overflow-hidden truncate">
<div class="truncate font-semibold leading-[17.6px] transition-colors duration-200 ease-in-out text-gray-charcoal group-hover:text-black-rich" data-testid="header-submenu-title">Attack Surface Management
</div>
</div>
<div class="text-xs leading-[14.4px] text-black-rich group-hover:opacity-100 transition-opacity duration-200 ease-in-out opacity-50" data-testid="header-submenu-title">Reveal &amp; protect coverage gaps
</div>
</div>
<svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-chevron-right h-5 w-5 text-gray-cadet"><path d="m9 18 6-6-6-6"></path>
</svg></a>
</li>
<li><a class="p-4 flex items-center justify-between group relative rounded-lg border border-transparent no-underline transition-all duration-200 ease-in-out hover:bg-navigation-mobile-hover overflow-hidden" data-gtm-cta="pla [... 128 characters folded ...]>
<div class="flex flex-1 flex-col gap-1 overflow-hidden truncate">
<div class="flex items-center gap-1 overflow-hidden truncate">
<div class="truncate font-semibold leading-[17.6px] transition-colors duration-200 ease-in-out text-gray-charcoal group-hover:text-black-rich" data-testid="header-submenu-title">Exposure Management
</div>
</div>
<div class="text-xs leading-[14.4px] text-black-rich group-hover:opacity-100 transition-opacity duration-200 ease-in-out opacity-50" data-testid="header-submenu-title">Identify &amp; remediate at-risk data
</div>
</div>
<svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-chevron-right h-5 w-5 text-gray-cadet"><path d="m9 18 6-6-6-6"></path>
</svg></a>
</li>
<li><a class="p-4 flex items-center justify-between group relative rounded-lg border border-transparent no-underline transition-all duration-200 ease-in-out hover:bg-navigation-mobile-hover overflow-hidden" data-gtm-cta="pla [... 131 characters folded ...]>
<div class="flex flex-1 flex-col gap-1 overflow-hidden truncate">
<div class="flex items-center gap-1 overflow-hidden truncate">
<div class="truncate font-semibold leading-[17.6px] transition-colors duration-200 ease-in-out text-gray-charcoal group-hover:text-black-rich" data-testid="header-submenu-title">Vulnerability Management
</div>
</div>
<div class="text-xs leading-[14.4px] text-black-rich group-hover:opacity-100 transition-opacity duration-200 ease-in-out opacity-50" data-testid="header-submenu-title">Pinpoint &amp; eliminate risk
</div>
</div>
<svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-chevron-right h-5 w-5 text-gray-cadet"><path d="m9 18 6-6-6-6"></path>
</svg></a>
</li>
<li><a class="p-4 flex items-center justify-between group relative rounded-lg border border-transparent no-underline transition-all duration-200 ease-in-out hover:bg-navigation-mobile-hover overflow-hidden" data-gtm-cta="pla [... 107 characters folded ...]>
<div class="flex flex-1 flex-col gap-1 overflow-hidden truncate">
<div class="flex items-center gap-1 overflow-hidden truncate">
<div class="truncate font-semibold leading-[17.6px] transition-colors duration-200 ease-in-out text-gray-charcoal group-hover:text-black-rich" data-testid="header-submenu-title">CyberGRC
</div>
</div>

Shown with 430 long blocks (scripts, styles, long tags) folded into a marker that says how many characters they hide; line breaks added before block tags because the source is minified; the pane stops after the first 200 words; the page is 733 KB.

Reading view the text a plain-fetch client can extract

title: Rapid7 Managed Cybersecurity: Outpace Attackers

The Preemptive MDR Leader that Outpaces Attackers

The Quarterly Threat Landscape Report is out. See what attackers are targeting now. Read report Search Platform The Command Platform Respond faster and outpace attackers. Taking command of your attack surface starts here. Explore platform Capabilities Attack Surface Management Reveal & protect coverage gaps Exposure Management Identify & remediate at-risk data Vulnerability Management Pinpoint & eliminate risk CyberGRC Manage governance, risk & compliance Next-Gen SIEM Investigate with AI-driven detections Threat Intelligence Platform Recognize & mitigate adversaries AI-Powered Security Operations Automate & optimize workflows Cloud-Native Application Protection Discover & defend sensitive data Application Security Scan & simplify dev workflows Explore the platform Talk to an Expert Product Tours Customer Stories All Products Services MDR Services Managed Detection and Response Preemptive MDR that disrupts attackers MDR for Microsoft Maximize and protect your Microsoft investment MDR for Enterprise Get customized protection for enterprise systems MDR ROI Calculator See how MDR can boost your ROI Rapid7 MDR vs. Competitors Decide which solution fits Exposure Management Managed Vulnerability Management Get help pinpointing exposures Continuous Red Teaming Get help validating exposures Managed Application Security Get help securing dev workflows Penetration Testing Services Get help with defense assessment All services Intelligence Intelligence Rapid7 Intelligence tracks adversaries, shares

1476 more words follow in the initial HTML.

4 found, 1 missing, 2 notes, from 7 checks run

  • H1Found
  • 200 wordsFound
  • llms.txtNote
  • MarkdownMissing
  • Missing pathFound
  • JSON-LDFound
  • OpenAPINote
What a plain fetch found on rapid7.com
CheckResultDetail
H1 in the initial HTMLFound one H1First H1: The Preemptive MDR Leader that Outpaces Attackers
First 200 words in the initial HTMLFound 200 of 200 wordsThe initial HTML carries 1676 words of visible text; the first 200 are marked in the left pane.
llms.txt present and linkedNote present, not linked2271 bytes, 16 links. Structure grade A (6 of 6). Nothing on the home page points to it. Ora's lab found 86 percent of llms.txt fetches arrived through a link.
Markdown on requestMissing 200 text/htmlAccept: text/markdown returned 200 with text/html; charset=utf-8. The server ignores the header.
Status of a made-up pathFound 404/agent-ready-html-missing-path-check returned 404: a real status code for a missing page.
JSON-LD typesFound EntryPoint, ImageObject, Organization, SearchAction and 1 more5 schema.org types in JSON-LD blocks on the home page: EntryPoint, ImageObject, Organization, SearchAction, WebSite.
OpenAPI linkNote none on the home pageNo OpenAPI link on the home page. It matters only for a site that ships an API.

Run the full check

The viewer read one page with four requests. The full check we recommend next is Ora's scan. Its Access layer runs 42 checks worth 30 of its 100 points, among them content without JavaScript, bot detection, llms.txt, markdown on request, JSON-LD and the sitemap (Ora's check catalogue, contract 1.38.0, read 5 November 2026). Open Ora's report for rapid7.com.

Limits: Ora reads public URLs only, says two runs of one scan land within a couple of points without publishing repeat-scan data, and keeps the product that fixes what it finds behind a waitlist. Cloudflare's scanner at isitagentready.com is a quick free alternative, and Mintlify's score covers documentation.

What the viewer does

It sends four plain GET requests with the user agent Mozilla/5.0 (compatible; agent-readiness-crawl/1.0): the URL itself, the same URL with Accept: text/markdown, /llms.txt on the same host, and a made-up path at the root. It runs no JavaScript. The left pane is the first bytes of the HTML as text, with long scripts and styles folded into a marker that says how many characters they hide. The H1 and the first 200 words of visible text are marked in both panes. The flags under the panes say what the four responses added up to.

Limits

  • Ten seconds for all four requests together, 1.5 MB read from the home page, at most five redirects, each one checked again.
  • http and https only, default ports only. Private, local and internal addresses and names are refused.
  • Ten runs per address per minute. The counter lives in memory on the edge function, so it is per instance rather than global. Past ten you get a 429 with a Retry-After header.
  • Results are cached for 24 hours by the response headers. Nothing is written to a database: the viewer keeps the first 200 words and the extracted fields for the length of the request and keeps no page body beyond that.

What it does not tell you

A plain fetch from this server is not a real agent. Bot protection weighs the source address more than the user agent, so a page that answers 200 here can still refuse an agent, and the reverse. Ora's lab found (2026-10-19) that returning 403 to agent user agents and hiding the answer in JavaScript were the only two of 19 configurations that stopped retrieval; the viewer can see the second, and can only hint at the first. The initial HTML reference covers both. The server logs and the published crawler address ranges settle the first.

The seven flags

H1 in the initial HTML
Whether the HTML the server sends already contains an H1. It is the cheapest sign that the page does not depend on scripts to say what it is.
First 200 words in the initial HTML
Whether at least 200 words of visible text arrive in the initial HTML. Under 40 words with an app root element is flagged as a JavaScript shell.
llms.txt present and linked
Whether /llms.txt answers with a plain-text file, whether the home page links it (an anchor, a link element or a Link header), and the structure grade the index uses.
Markdown on request
Whether a request with Accept: text/markdown comes back as text/markdown. Cloudflare Radar found 3.9 percent of scanned top sites passing this at its April 2026 launch.
Status of a made-up path
Whether a path that should not exist returns 404 or 410. A 200 is a soft 404 and reads as a live page.
JSON-LD types
The schema.org types found in JSON-LD blocks on the page. Ora's lab found JSON-LD made no difference when the answer was already in the HTML.
OpenAPI link
Whether the page links an OpenAPI file, by an anchor or a service-desc link relation. It matters only for a site that ships an API.

Sources

  1. Ora: AgentReady launch post (Ora's lab findings) https://ora.ai/blog/agentready-launch, read 4 November 2026
  2. Ora check catalogue, contract 1.38.0 (132 checks, 42 in the Access layer) https://ora.ai/api/checks, read 5 November 2026
  3. Ora home page (the fixing product is behind a waitlist) https://ora.ai/, read 5 November 2026
  4. Ora: Introducing Deep Scan (repeat scans) https://ora.ai/blog/introducing-deep-scan, read 5 November 2026
  5. Ora llms.txt (public URLs only) https://ora.ai/llms.txt, read 5 November 2026
  6. Vercel: Make your site readable by AI agents https://vercel.com/kb/guide/make-your-site-readable-by-ai-agents, read 4 November 2026
  7. llms.txt proposal (llmstxt.org) https://llmstxt.org/, read 4 November 2026

Last reviewed 5 November 2026