Understand

What is llms.txt?

A Markdown file at /llms.txt. This page reads the one this site serves, next to robots.txt and the sitemap.

The proposal

llmstxt.org is a 2024 proposal by Jeremy Howard. A site may publish Markdown at /llms.txt so a model can start with a name, a summary, and a few chosen links.

The usual shape is one H1, a blockquote summary, optional notes, then H2 sections of markdown links. An optional /llms-full.txt can expand those pages into one long document. This origin does not serve /llms-full.txt.

Which AI tools read the file is unconfirmed, and the public evidence is limited. Publishing it is cheap. It is a speculative hint, not a ranking lever. A crawler that requests the URL has fetched a file. That request does not show the file changed an answer.

Access

robots.txt

Tells a polite crawler which paths it may fetch. It does not describe the site.

User-Agent: *
Allow: /
Disallow: /account
Disallow: /admin
Disallow: /api
Disallow: /white-paper/download
Disallow: /white-paper/thanks

Sitemap: https://www.blackboxpersonalization.com/sitemap.xml

Discovery

sitemap.xml

45 public URLs. A list for crawlers, not a reading guide.

/live
/blog
/white-paper
/about
/contact

Guidance

llms.txt

A short brief. The links are chosen, not every URL in the sitemap.

# Black Box Personalization

> Opening the black box of personalization. A public site by Neil Black about how digital metrics are collected in the browser, on the network, and at the server, and how those signals can personalize a page.

The public origin is https://www.blackboxpersonalization.com. Pages teach with the live mechanism. This file is a curated brief. It does not replace robots.txt or the sitemap.

## Start here

- [Home](https://www.blackboxpersonalization.com/): Where a metric is read in the browser, on the network, and in the stored beacon.
- [Lab](https://www.blackboxpersonalization.com/lab): Collection, understanding, and personalization, with the rule on the page.
- [About](https://www.blackboxpersonalization.com/about): Neil Black.
- [Notes](https://www.blackboxpersonalization.com/blog): Short essays.

This site’s file

Open /llms.txt

The lines below are the body served at /llms.txt. Each note names the part of the proposal that line fills.

1 · H1
```
# Black Box Personalization
```
One H1. The proposal uses it as the project or site name.
2 · Blank
```
 
```
A blank line between parts.

3 · Summary

> Opening the black box of personalization. A public site by Neil Black about how digital metrics are collected in the browser, on the network, and at the server, and how those signals can personalize a page.

A blockquote. A short summary a model can read before it follows a link.

4 · Blank
```
 
```
A blank line between parts.

5 · Detail

The public origin is https://www.blackboxpersonalization.com. Pages teach with the live mechanism. This file is a curated brief. It does not replace robots.txt or the sitemap.

Optional prose. The proposal allows notes between the summary and the link lists.

6 · Blank
```
 
```
A blank line between parts.
7 · Section
```
## Start here
```
An H2. Links under it are one group.
8 · Blank
```
 
```
A blank line between parts.

9 · Link

- [Home](https://www.blackboxpersonalization.com/): Where a metric is read in the browser, on the network, and in the stored beacon.