Markdown Rendering — Overview

What it is

A pattern where your origin returns a markdown-formatted version of a page when, and only when, the request comes from a verified LLM bot. All other clients (humans, generic crawlers) continue to see the regular HTML response.

Why it exists

LLM crawlers don't render JavaScript, don't paint pixels, and don't care about your design system. When they fetch a typical modern web page, 80–95% of the bytes are noise: scripts, styles, font files, third-party tags, image markup. The signal — the actual prose, headings, and structured content — is buried.

Serving markdown when a bot asks for it dramatically improves the signal-to-bytes ratio:

• Faster crawl.
• Less load on your infrastructure.
• Higher-quality content lands in the model's training and retrieval layers.
• More reliable mention behavior in downstream LLM responses.

How it works (in one paragraph)

A small router sits in front of your application. For every incoming request, it checks the User-Agent header. If the value matches a known LLM bot allowlist and a feature flag is enabled, the router internally rewrites the request to a markdown rendering endpoint, which produces a clean markdown representation of the same URL. Everyone else gets the normal HTML response. The end-user experience is unchanged.

What you change in production

• Add one router (middleware, URL rewrite rule, or edge worker — your stack determines the form).
• Add one rendering endpoint that converts your existing HTML to markdown server-side.
• Add one feature flag.

Nothing else. No CMS migration. No design system change. No public URL changes. No SEO risk.

What you do NOT change

• Your HTML templates.
• Your tracking and analytics.
• Your authentication flows.
• Your database schema.
• Your CDN configuration (unless you choose to push the routing to the edge).

Stack-specific guides

• SharePoint Client Site (IIS-hosted) — Microsoft stack with client-side rendering
• Header-based routing pattern (coming soon)
• Next.js (coming soon)
• .NET / ASP.NET Core (coming soon)

Effort

For a single application with a coherent template system, a competent backend engineer ships this in 15–25 hours of focused work. The largest variable is your HTML's cleanliness — heavily nested or div-soup pages take longer to convert cleanly.

Reversibility

Every implementation Maya documents is gated by a feature flag. Flipping the flag off restores the original behavior in seconds. No data migration, no rollback complexity, no risk to user-facing traffic.

Common questions

Will this affect SEO? No. The router fires only for the LLM bot allowlist. Googlebot fetches the normal HTML by default. (You may opt to route Googlebot to markdown if you want; we recommend starting without that.)

Will this break our analytics? No. Bots don't execute JavaScript, so they were never running your analytics tags. Removing analytics tags from the markdown response is correct, not a regression.

Will this work with our CDN? Yes. The router can run at the origin or at the edge. The IIS-based example in the SharePoint guide assumes origin routing.

Will this expose internal pages? No, when implemented correctly. The endpoint refuses paths that resolve to authenticated areas, and only renders content from the same set of public URLs your normal site already serves.

Can we limit which sections get markdown? Yes. The router can match on path prefix as well as user-agent. You can roll this out one section at a time.