The llms.txt Convention

The llms.txt convention is a method for website administrators and community professionals to create a public file that guides Large Language Model (LLM) crawlers towards high-value content and away from low-value or irrelevant sections of a site. Functioning as a companion to, but distinct from, the `robots.txt` file, `llms.txt` is not primarily about blocking access. Instead, its purpose is to provide signals about which content is most suitable and valuable for the purpose of training generative AI models. By creating an `llms.txt` file, organisations can proactively influence how their community's collective knowledge is ingested, understood, and ultimately represented by AI systems, making it a foundational tool in the practice of Generative Engine Optimisation (GEO).

Why it matters — The llms.txt Convention