Skip to main content
Schjønhaug AS
← Guides

What is llms.txt – and does your website need one?

llms.txt is a small text file that tells language models and AI agents what your website is about and where the most important content is. Here is what it actually does today, what it does not do, and how to create one.

Published

In short

  • llms.txt is a Markdown file at the root of a domain, such as https://www.yourcompany.com/llms.txt. It gives a short description of the business and a list of links to the most important pages.
  • Google Search does not use it. Google has said clearly that the file has no effect on indexing, ranking, or AI features in Search.
  • Lighthouse checks for it. Since version 13.3 (July 2026), Lighthouse has an Agentic Browsing category, and llms.txt is one of its audits.
  • It takes less than an hour to create. The benefit is modest, but the file can help AI agents and developer tools find their way.
  • It does not make your website usable for agents. Whether an agent can order, book, or submit a form depends on the pages themselves, not on llms.txt.

What is llms.txt?

The proposal came in autumn 2024 from Jeremy Howard at Answer.AI and is described at llmstxt.org. The problem it addresses is that ordinary web pages are built for people: navigation, cookie banners, scripts, and layout take up a lot of space, and language models have limited room for text. An llms.txt file gives the model a tidy starting point: who you are, what you offer, and which pages are worth reading.

llms.txt is easy to confuse with robots.txt, but they do different jobs. robots.txt says what crawlers may fetch. llms.txt says what is worth reading. It grants no access and blocks nothing.

What the file looks like

The format is ordinary Markdown in a fixed order:

  1. A level-1 heading with the name of the website or business (the only required part).
  2. A short summary in a blockquote (>).
  3. Optionally, a few paragraphs of further explanation.
  4. Sections with level-2 headings, each containing a list of links in the form [title](url): description.

This is how our own llms.txt begins:

# Schjønhaug AS

> Software and product studio in Oslo, Norway, founded in 2014. We build
> AI and automation, web and mobile apps, and backends and integrations ...

## English: Services

- [AI and automation](https://www.schjonhaug.com/en/services/ai-and-automation/):
  We identify where AI can create measurable value ...

We did not write the file by hand. It is built from the same content as the rest of the website, so new projects and services are included automatically, and the file cannot go out of date unless the pages do too.

Who reads llms.txt in 2026?

Google Search: no. Google has said that Search does not use llms.txt, and lists the file among the things you do not need to appear in Google’s AI features.

AI crawlers: rarely. None of the large AI companies has said that its crawlers use llms.txt from other websites. Server log analyses show that the crawlers mostly fetch ordinary HTML pages.

Lighthouse and Chrome: yes. Lighthouse 13.3 added the Agentic Browsing category, which assesses how well a website works for AI agents that use the browser. Its main audits are these:

  • llms.txt: does the file exist, and is it valid?
  • WebMCP: does the page offer tools and forms that agents can use directly?
  • Accessibility for agents: do buttons, links, and form fields have names a machine can read?
  • Layout stability: does content move while the page loads, so the agent risks clicking the wrong thing?

We go through each audit, and what counts towards the score, in Lighthouse Agentic Browsing: how to read the results.

Developer tools and documentation: this is where the file is used most. Many API and developer tool providers publish llms.txt for their documentation so that coding assistants can look things up.

What Lighthouse checks in llms.txt

The audit is simple and is about form, not content. The file must

  • have a level-1 heading (# Name)
  • be longer than 50 characters
  • contain at least one link in Markdown format: [text](https://...)

Plain URLs without Markdown syntax do not count as links, even if they work. If the file does not exist (404), the audit is marked “not applicable” and does not count against you. If the server returns another error, the audit is flagged.

How to create an llms.txt

  1. Write the summary first. One to three sentences on who you are, what you offer, and where you are based. Write it the way you would explain it to a new customer.
  2. Choose the pages. Include your services, prices or offers, the contact page, and a few good examples of your work. Skip privacy policies, tags, and archive pages.
  3. Write one line per link. Use [title](full URL): what the page is about, with full URLs including https://.
  4. Put the file at the root. It belongs at /llms.txt, not in a subfolder, and should be served as plain text.
  5. Keep it up to date. Generate the file from your publishing system if you can. Many plugins for WordPress and other platforms do this, and on custom-built websites it is a small job.
  6. Use a tool if you like. With our free llms.txt generator, you fill in the name, description, and links, and get a finished file checked against the requirements.
  7. Check it. Run Lighthouse in Chrome and look at the Agentic Browsing category, or open /llms.txt in the browser and read it as if you were a new customer.

Some websites also publish llms-full.txt with all the content in a single file, or a Markdown version of every page. That is mostly useful for documentation. For most business websites, a single llms.txt is enough.

Does your website need one?

If yours is an ordinary business website, the answer is: it does no harm, it takes little time, and you should not expect more traffic from Google because of it. It is a small, cheap measure that makes the website slightly easier to understand for the agents and tools that do look for the file. It is also one of the Lighthouse audits, which may be reason enough.

If you publish documentation, an API, or a developer tool, the benefit is clearer. That is where the file is actually read today.

What llms.txt does not do is let an agent do anything on the website. The file can say that you accept orders, but not whether an agent can actually fill in the order form. That depends on how the form, the buttons, and the flow are built – the same things Lighthouse looks at in the other audits. We have collected the most important points in a checklist for websites AI agents can use.

Want to know whether an agent actually gets the job done?

An agent test is part of Foretrukket (in Norwegian), our AI-visibility service for Norwegian businesses. It helps them get recommended when customers ask ChatGPT and Google.