What an llms.txt file is

An llms.txt file is a plain text file written in Markdown that sits at the top level of a website, at an address like example.com/llms.txt. It was suggested in 2024 as a simple way for site owners to point large language models (LLMs) at the pages that matter most.

Think of it as a reading list rather than a full map. A sitemap lists every address for search engines; llms.txt names your site, explains it in one sentence, and links to a hand-picked set of pages with a short note on each.

Why it can help your website

AI assistants answer questions by reading web pages, and they can only take in so much text at once. A short, well-labelled index makes it easier for them to find your pricing page, your documentation or your main services instead of guessing from navigation menus and footers.

Support for llms.txt varies between AI companies, so treat it as a low-effort extra rather than a guaranteed way to be cited. It takes minutes to publish and does not affect your normal search results.

How Crawllyt builds your file

• We read up to 500 of your pages, following internal links from the address you enter, or we take the page list from your sitemap.
• Every page is checked the same way as in our Site Audit, so robots.txt rules are respected and pages marked noindex are skipped.
• Your homepage title supplies the site name, and its meta description supplies the summary. You can type your own in Settings.
• Pages are grouped by the first folder in their address. Single pages go into Main pages, and legal pages go into Optional, a group AI tools can skip when they are short on space.

Example of the finished format

# Example Bakery

> Small-batch sourdough bakery in Pune with online ordering and local delivery.

## Main pages

- [Order online](https://example.com/order): Choose loaves and a delivery slot
- [Our story](https://example.com/about): Who bakes your bread and how

## Recipes

- [Starter guide](https://example.com/recipes/starter): Keep a sourdough starter healthy

## Optional

- [Privacy policy](https://example.com/privacy): How customer data is handled

Publishing your llms.txt

• Save the file with the exact name llms.txt, in plain text with UTF-8 encoding.
• Upload it to the top folder of your website using your host’s file manager or FTP, the same place robots.txt lives.
• Open yourdomain.com/llms.txt in a browser to confirm it loads as text, not as a web page.
• Run it through the Crawllyt LLMs.txt Validator and update it whenever you add important pages.