In September 2024, Jeremy Howard (of fast.ai) proposed a new convention: a file at /llms.txt, written in plain Markdown, that gives AI assistants a curated summary of a website and links to its most important pages. The idea spread fast through developer and SEO circles, several major platforms and documentation sites adopted it within months, and by 2026 it's a question that comes up in nearly every "how do we show up in ChatGPT" conversation. It's also, worth saying upfront, not a confirmed standard that any major AI lab has officially committed to using.

What llms.txt actually is

Structurally, it's about as simple as a web standard gets: a Markdown file at the root of your domain (yoursite.com/llms.txt), starting with an H1 for your site or company name, a short blockquote summary, and then a handful of linked sections pointing to key pages with a one-line description of each.

# Zelvorm

> A London digital studio building websites, apps and digital products.

## Docs

- [Services](https://www.zelvorm.com/services): What we build and for whom
- [Work](https://www.zelvorm.com/work): Case studies and results

## Optional

- [Journal](https://www.zelvorm.com/articles): SEO, design and build guides

The intent is for an AI assistant — or honestly, a human skimming quickly — to understand what the site is and where to find the parts that matter in a few seconds, without parsing a full HTML page or a 10,000-URL sitemap.

Why it exists at all

Large language models have a real, practical constraint that traditional search crawlers don't: context window size. A search engine can index and store the full text of billions of pages; a model answering a single question can only hold a limited amount of text in its working context at once. llms.txt is a response to that constraint — a condensed, high-signal pointer to what's actually worth reading, instead of forcing a model (or whatever retrieval system sits in front of it) to parse an entire site's navigation, legal pages, and marketing chrome to find the two pages that matter.

It's also, less formally, a bet on where AI-assisted browsing is heading: as more traffic arrives via an AI assistant that's read a page on a user's behalf rather than a human clicking a blue link, having a clean, curated entry point for that assistant becomes a reasonable piece of insurance.

Should you actually add one?

For most small-to-medium business websites, the honest answer is: it's a low-cost addition worth doing, but not a priority over fixing actual technical SEO and content issues. The cases where it matters more:

  • ◆Documentation and developer-tool sites, where AI assistants are very frequently asked to help someone use or integrate the product — this is where llms.txt adoption is strongest and most clearly useful.
  • ◆Sites with large, noisy navigation where the pages that actually matter are buried under hundreds of thumbnails, archive pages, or legal boilerplate that would otherwise dilute what an AI system reads.
  • ◆Businesses actively investing in AI-search visibility as a channel — if you're already doing the work to be well-cited in AI answers, llms.txt is a few minutes of additional effort with no real downside.

Where it matters less: a small brochure site with five pages doesn't need a curated index of five pages — the full site is already small enough for any crawler or assistant to understand directly.

Tip Keep it genuinely short. The entire value proposition is "here's what actually matters," so a 200-line llms.txt listing every blog post you've ever written defeats the point. Three to eight linked pages with honest one-line descriptions is usually plenty.

What to put in it

Lead with a clear, factual summary of what your company does — not marketing copy, just the plain facts an assistant would need to describe you accurately to someone else. Then link your most important pages: core service or product pages, your best case studies or proof points, and anything genuinely informational (documentation, a pricing page, a well-written guide). Leave out legal pages, individual blog posts (unless one is truly foundational), and anything behind a login.

Generating one

You don't need to hand-write the Markdown structure from scratch. Our free llms.txt generator takes your site details and key pages and produces a correctly formatted file you can drop at your domain's root in a couple of minutes — and because nothing is fetched or changed on your live site by using it, there's no risk in trying it even if you're still deciding whether to publish.

llms.txt is one small piece of a much bigger shift — see our deeper look at how AI Overviews are changing SEO and what AI visibility (GEO) actually means for the fuller picture of how to show up in AI-generated answers, not just traditional search. If you want your whole AI-readiness setup reviewed — crawler access, structured data, llms.txt and content — that's exactly what our AI automation services cover.