Milaaj Editorial / Research Insights

Someone on LinkedIn told you llms.txt is "the new robots.txt." A vendor offered to add one to your site for a fee. Now you are wondering whether you are about to miss the biggest shift in search, or about to pay for a file nobody reads.
Fair question. So, what is llms.txt, and does your business actually need it?
This guide gives you the plain answer, what Google itself says, and a simple way to decide.
llms.txt is a proposed Markdown file, placed on your website, that gives AI tools a short, curated summary of your site and links to your key pages. Google says Google Search does not use it, and you do not need it to appear in Google. For most UAE business websites it is optional and low priority.
llms.txt is a text file written in Markdown, saved at the root of your website (for example, yourdomain.com/llms.txt). It acts like a short guide for large language models: who you are, and which pages matter most.
The idea is simple. AI tools can struggle to pull key facts from cluttered web pages full of menus, scripts and banners. A tidy summary with direct links could save them the effort.
The proposal's format is straightforward. Only the title line is required:
# Example Business Name
> One or two sentences explaining what the business does.
## Services
- [Web design](https://example.com/web-design/): What we build and for whom
- [SEO](https://example.com/seo/): How our SEO work is structured
## Optional
- [Blog](https://example.com/blog/): Guides and articles
The required part is the H1. The summary quote, body text and link lists are optional. An H2 section named "Optional" holds secondary links that can be skipped.
These three files are often confused. They do different jobs.
File | What it does | Who reads it |
|---|---|---|
robots.txt | Tells crawlers which parts of your site they may access | Search and other crawlers |
sitemap.xml | Lists your URLs so search engines can find them | Search engines |
llms.txt | Offers a curated Markdown summary and key links | AI tools (proposed) |
So llms.txt does not block or allow anything. It is a reading guide, not a rulebook.
The idea comes from the llms.txt proposal, which lists Jeremy Howard as its author. It says thousands of sites publish the file, and that some documentation platforms generate it automatically.
The same page also says the specification is open for community input. In other words, it is a proposal that people are trying out, not a rule that search engines or AI companies must follow.
Chrome's Lighthouse tool now includes an audit for it. Chrome's documentation calls llms.txt an "emerging convention" and says that providing the file is optional at the moment. That audit concerns how ready a site is for browser-based AI agents. It says nothing about rankings.
No. Google's guide to AI features in Search says you do not need new machine-readable files, AI text files, markup or Markdown to appear in Google Search. It says Google Search ignores llms.txt files, though it is fine to create one if you want.
So adding llms.txt will not help you rank in Google or appear in AI Overviews. What Google does ask for is simple: make sure your content is crawlable.
This is the claim sellers lean on, so be careful with it.
I could not find a verified source showing that ChatGPT, Perplexity, Claude or any other AI assistant reads llms.txt files, or that having one earns more citations. Anyone who promises that is guessing.
What you can say honestly:
If a provider announces support in future, the picture could change. For now, treat llms.txt as an experiment, not a growth strategy.
It depends on what your site does. Use this table as a guide.
Type of site | Should you add llms.txt? |
|---|---|
Software, API or developer documentation site | Reasonable, since agents may read long technical docs |
Large content or knowledge-base site | Worth a small experiment |
Typical UAE service business (clinic, agency, trading company) | Optional and low priority |
Site with crawlability or content problems | Fix those first |
For most readers of this blog, the honest answer is: you can add one, but do not spend real budget on it. At Milaaj Brandset, we would put your time into fixing weak pages and unclear answers before touching a new file.
If you decide to try it, the work is small.
Make sure it returns a normal page rather than an error. Chrome's audit flags server errors when it tries to fetch the file. Your developer can handle the upload, and a web development team in Dubai can add it to your site or CMS quickly.
Google's guidance stresses crawlable content. These basics help your site whichever tool reads it:
Our guide to AI search optimization strategies for businesses walks through these priorities step by step. If you only have time for one improvement this month, start there.
llms.txt is a proposed Markdown file placed at the root of a website. It gives AI tools a short summary of the site and links to its most useful pages. It is an open proposal, not an official standard.
No. Google says Google Search ignores llms.txt files and that you do not need special AI files or markup to appear in Search. You may still create one for other purposes.
No. Robots.txt tells crawlers what they may or may not access. llms.txt is a reading guide that summarises your content. It does not allow or block anything.
There is no verified evidence that it will. I could not find a source showing that major AI assistants read the file or cite sites more because of it. Focus on clear, crawlable content instead.
Write a Markdown file with your business name as the H1, an optional short summary, and lists of links to your key pages. Save it as llms.txt and upload it to your site root.
So, what is llms.txt? A small, optional file with an interesting idea behind it and no proven payoff for most business websites. Google says it ignores it, and AI platforms have made no commitments I could verify.
You can try it in an afternoon, but your real gains come from clear content, a crawlable site and pages that answer customer questions well. If you want a team to look at that bigger picture, our SEO services in Dubai cover technical foundations and AI search visibility together.
Good decisions here are boring ones: check what is proven, ignore the hype and fix the basics first. You are already ahead of most people asking this question.