Building Your First LLMs.txt in Kentico

May 8, 2025

Author

John Flores

Ever feel like your brilliantly crafted Kentico site content isn't getting the love it deserves from the latest AI models? Like they're wandering around your digital masterpiece, missing the really important bits? You're not alone! While robots.txt has been our go-to for telling traditional crawlers where to go (and where not to go), the world of Large Language Models (LLMs) needs a slightly different map. In this post, I will show you how to build a llms.txt to help AI agents to crawl your website data efficiently and faster.

Getting Started

Think of llms.txt as your website’s personal concierge for AI. Unlike traditional crawlers that rely on robots.txt to navigate your site, large language models (LLMs) need their own guide. That’s where llms.txt comes in. This structured text file helps LLMs locate and understand your most valuable content—whether it’s API docs, product details, policies, or anything else you want them to find. By giving AI agents a curated map, you make their interactions with your site faster, more accurate, and much more useful.

How to Structure Your llms.txt File

Like any file meant for a program, llms.txt needs a specific structure. It uses a Markup Language (which looks suspiciously like Markdown, making it nice and readable for us humans too!). The basic layout is hierarchical:

# Site Title
Site Summary

## Section Name 
- [Name of the page](URL): Short Summary

## Other
- [Name of the page](URL): Short Summary

Example

Let's build an example llms.txt file. In this case, we have a fictional school Called "example.com" which specializes in .NET learning, where we have inscription information, prices, guidelines, curriculums, etc.

Bringing it to Life in Kentico CMS

Okay, time to put on our developer hats! Manually creating and updating that file could be a pain, right? Especially on a dynamic Kentico site. The good news is we can automate this using the Kentico API!

Here's a simple example of a method that pulls a list of "Article" pages from your site and formats them for your llms.txt file

public string GetArticlesForLLMS()
  {
   StringBuilder sb = new StringBuilder();
   List<TreeNode> result;
   result = pageRetriever.RetrieveMultiple(query => query
                    	.WithPageUrlPaths()
                    	.Published()
                    	.Types("Article")
                    	.WithCoupledColumns()
                    	.OrderByDescending("DocumentCreatedWhen"))
                    	.ToList();

   sb.AppendLine("# SITE-NAME");
   sb.AppendLine("SITE-DESCRIPTION\n");
   sb.AppendLine("## Articles \n");

   foreach (var page in result)
   {
   	var typedPage = page.Map<Article>();
   	var summary = typedPage.Summary;
   	var headline = typedPage.Headline;
   	var url = SiteContext.CurrentSite.SitePresentationURL + "/" + page.GetPageUrlPath().FullPath.ToLower();

   	if (!string.IsNullOrEmpty(headline))
       	sb.AppendLine(string.Format("- [{0}]({1}){2}", headline, url, string.IsNullOrEmpty(summary) ? "" : ": " + summary));
   }

	return sb.ToString();
}

This example just pulls "Article" pages. As the sources mention, you can easily include additional page types and concatenate them to build a comprehensive llms.txt for your whole site. You'd just write similar methods for other page types (like Products, FAQs, Documentation, etc.) and then combine the results.

Once you have the generated string, you'll need to decide where to serve it from. The standard location for AI models to look is at /llms.txt on your domain. You could set up a custom route in your Kentico application that generates and serves this file dynamically.

Conclusion

Building the basis for your llms.txt in Kentico isn't so scary. With a bit of Kentico API magic, you can automate this process and provide LLMs with a clear, structured map to your site's most valuable content. This helps ensure your important information is easily discoverable and accurately interpreted by AI models.

This example is simple, but creating a reliable llms.txt file for a complex site can be more challenging. You might need to handle multiple page types, optimize performance, and set up a reliable way to serve the file—all of which can get tricky.. If you find yourself hitting roadblocks or just want expert assistance to get this implemented in your Kentico project, don't hesitate to reach out! We specialize in making Kentico development smoother and helping you leverage We specialize in making Kentico development smoother and implementing forward-looking features like llms.txt to support AI models.

Ready to get started? Reach out to [A]—we’ll help you move faster and smarter.