What an assistant answers with when someone asks about you
When a customer asks an AI assistant whether your company also works on Saturdays, the answer is assembled from whatever the system managed to read. Your service pages, your business listing, a directory someone added years ago, a page a former employee copied. The machine has no way of knowing which of those sources is right. If your site says “appointments only” and a listing says “9 to 5”, it has two versions of your company and no reason to prefer yours.
This is the part website owners underestimate. Conversations about AI visibility usually start with “how do we get cited”, and a smaller question has to come first: does a consistent, machine-readable version of your company exist at all?
The page a machine fetches is not the page your visitor sees
Most AI fetchers read raw HTML and never run the scripts that make a modern site look finished. Menus, tabs and service lists that only appear after JavaScript executes can be invisible to them. The same applies to text baked into images and to links that are really click handlers. A website can look perfect in a browser and still hand a machine an empty shell.
That makes readable, server-rendered HTML a build decision, not a cleanup task. View source is the honest test: if a page’s main content — what you do, for whom, where, at what price — is not in the HTML the browser downloads first, a fetcher may never see it. Headings that form a real outline, real anchors instead of scripted clicks and alt text on images close the gap between what people see and what machines receive.
This is also where AI-generated sites most often go wrong. Generation optimises for what looks right on screen, not for what survives being read line by line by something that never renders it.
Structured data: your facts in the format machines expect
Structured data is a short description of your page that sits next to the visible text, written for machines to parse. The vocabulary is schema.org, created in 2011 by Google, Bing and Yahoo so that one website could describe itself once for every system. Google recommends the JSON-LD format and states that it uses the markup to understand page content and to gather information about companies, people and places.
For a business website a small set covers most of the value: the organisation with its name, address, phone number, opening hours and area served; the individual services; the articles you publish; and breadcrumbs so a machine can tell where a page sits. Every one of those facts is something an assistant can be asked about.
Two honest limits. First, markup does not buy citations. There is no published evidence that adding schema makes AI answers link to you more often, and anyone promising that is selling something. What the markup removes is ambiguity: a system that can read your hours as data does not have to guess them out of a sentence. Second, machines compare the markup with the page. When structured data claims something a visitor cannot see, search systems discount it, and contradictory signals make a business less quotable, not more. Generate it from the same source as your visible content, and validate it with a structured data testing tool before launch.
Who is allowed to read: the robots.txt decision nobody makes
Access for AI systems is settled in robots.txt, a convention from the 1990s formalised as RFC 9309 in 2022 and honoured by every major crawler. The entries that matter now include answer crawlers such as OAI-SearchBot and PerplexityBot, training crawlers such as GPTBot, ClaudeBot and CCBot, and Google-Extended, which governs how Google’s Gemini uses your content.
Two mistakes are common here. Blocking Google-Extended does not remove a site from Google’s AI summaries — those run on the regular search index — so it buys nothing there. Blocking an answer crawler can remove you from exactly the surface where your customers now ask questions, while blocking a training crawler only affects future model training. Most websites simply inherit whatever the host or template shipped with. The policy should be a decision someone wrote down, with a date, at launch.
The fashionable shortcut: what llms.txt does not do
llms.txt is the file most AI-visibility offers are currently built on, and the evidence behind it is thin. Proposed in September 2024 as a markdown map of a site for language models, it is still a proposal rather than a standard: Google has said on the record that it is not needed for Search or its AI features, and log studies covering hundreds of millions of AI crawler requests find answer bots fetching it in well under one percent of visits.
That does not make it worthless — several AI vendors publish one themselves, and agent tooling does read it when someone points it at the file. But it takes an hour to write and cannot substitute for the unglamorous work above. Paying for it as an AI-visibility fix is buying the visible part and skipping the part that works.
What belongs in the brief
Everything above is decided while the website is being created, so it belongs in the requirements rather than in a later optimisation phase. Four things worth putting in writing with your builder: the main content of every key page must be present in the raw HTML; the core facts — name, address, hours, prices, area served — must be identical on the site, on your business listing and in the directories; structured data must be generated from the content visitors actually read and pass validation; and the robots.txt policy must be an explicit decision, recorded with reasons. None of it is expensive. All of it is expensive to retrofit.
How we build this into the project
Machine readability is part of the same pass we make for speed and accessibility, because it has the same property: it is cheap to decide early and awkward to add later. In practice that means server-rendered pages, one source of truth for your business facts, structured data generated from that source instead of typed by hand, and a crawler policy written down at launch.
A website is now read twice — once by a person deciding whether to trust you, and once by a system deciding what to say about you. Build it so both readings come out the same.
Related service: Web design and content strategy