Added llms.txt and llms-full.txt to My Hugo-built Website


This article is a translation of https://masutaka.net/2025-05-18-1/.
What is llms.txt?
llms.txt
is a Markdown-formatted text file proposed to address the limitation that LLMs have small context windows and cannot process entire websites.
It was proposed on September 3, 2024, by Jeremy Howard of Answer.AI at https://llmstxt.org/. It is not defined in an RFC like robots.txt.
The format includes certain specifications such as the website name in an H1 section, a brief summary, and a list of links in H2 sections. Example: https://llmstxt.org/llms.txt
There's also llms-full.txt
, which contains all website information. Example: https://developers.cloudflare.com/llms-full.txt
There are directory sites for llms.txt as well:
The llms.txt and llms-full.txt I Added
I've added both:
How to Configure in Hugo
The necessary work involves modifying or creating just three files:
config.toml (modify)
layouts/index.llms.txt (create new)
layouts/index.llmsfull.txt (create new)
config.toml
I added the following to config.toml:
Defining outputFormats for llms
and llmsfull
while adding them to the default value of outputs.home, which is ["html", "rss"]
.
[outputs]
home = ["html", "rss", "llms", "llmsfull"]
[outputFormats.llms]
baseName = "llms"
isPlainText = true
mediaType = "text/plain"
rel = "alternate"
root = true
[outputFormats.llmsfull]
baseName = "llms-full"
isPlainText = true
mediaType = "text/plain"
rel = "alternate"
root = true
layouts/index.llms.txt
I created layouts/index.llms.txt
as a template for llms.txt
:
# {{ .Site.Title }}
> {{ .Site.Params.Description }}
## Articles
{{ $yearMonthGroups := slice -}}
{{ range where (where (sort (.Site.GetPage "/posts/").Pages "Date" "desc") "Draft" "ne" true) "Sitemap.Disable" "ne" true -}}
{{ $yearMonth := .Date.Format "2006/01" -}}
{{ if not (in $yearMonthGroups $yearMonth) }}
* {{ $yearMonth -}}
{{ $yearMonthGroups = $yearMonthGroups | append $yearMonth -}}
{{ end }}
* [{{ .Title }}]({{ .Permalink }})
{{- end }}
## Others
{{- $baseURL := .Site.BaseURL | strings.TrimSuffix "/" }}
* [Device]({{ $baseURL }}/device/): 所有デバイス
* [History]({{ $baseURL }}/history/): このサイトの歴史
* [About]({{ $baseURL }}/about/): 自己紹介
* [Privacy Policy]({{ $baseURL }}/privacy/): 当サイトの広告、アフィリエイト、プライバシーポリシー
Since Hugo site structures can vary widely, this is just one example. My blog has both blog posts and other articles, and since other articles don't change much, I hardcoded them in the Others section.
layouts/index.llmsfull.txt
I created layouts/index.llmsfull.txt
as a template for llms-full.txt
:
# {{ .Site.Title }}
> {{ .Site.Params.Description }}
{{/* Articles */}}
{{ range where (where (sort (.Site.GetPage "/posts/").Pages "Date" "desc") "Draft" "ne" true) "Sitemap.Disable" "ne" true }}
--------------------------------------------------------------------------------
title: "{{ .Title }}"
date: "{{ .Date.Format "2006-01-02" }}"
--------------------------------------------------------------------------------
{{ replaceRE "{{<\\s*comment\\s*>}}(.|\n)*?{{<\\s*/comment\\s*>}}" "" .RawContent -}}
{{ end -}}
{{/* Others */}}
{{ range slice "device.md" "history.md" "about.md" "privacy.md" -}}
{{ with site.GetPage . -}}
--------------------------------------------------------------------------------
title: "{{ .Title }}"
lastmod: "{{ .Date.Format "2006-01-02" }}"
--------------------------------------------------------------------------------
{{ replaceRE "{{<\\s*comment\\s*>}}(.|\n)*?{{<\\s*/comment\\s*>}}" "" .RawContent -}}
{{ end -}}
{{ end -}}
I made the delimiter longer because some articles contained ---
.
Also, I made sure comments like the following wouldn't be included in llms-full.txt:
{{<comment>}}
This is a Hugo comment. It won't be output as an HTML comment either.
{{</comment>}}
nginx Configuration
Since llms.txt and llms-full.txt were showing character encoding issues, I added the following to the server directive:
location ~ "^/llms(-full)?\.txt" {
root /usr/share/nginx/html;
charset UTF-8;
}
Conclusion
I added llms.txt and llms-full.txt on my Hugo-built blog:
Regardless of whether there's demand for it, I'm satisfied with the result.
As a side note, I initially found a method on Hugo Discourse that used resources.ExecuteAsTemplate
in layouts/robots.txt
to generate llms.txt. However, I decided not to use this approach since including llms.txt in robots.txt is not yet common practice.
Another side note: in the past, my blog was created by converting a single ChangeLog-formatted file into HTML using a tool called chalow. Creating llms-full.txt reminded me of those days. It feels like I've come full circle.
Subscribe to my newsletter
Read articles from Takashi Masuda directly inside your inbox. Subscribe to the newsletter, and don't miss out.
Written by
