What makes a B2B website show up in AI search?

What does Google say you need to do?

Less than the industry suggests. Google says there are no additional requirements to appear in AI Overviews or AI Mode, and no special optimisations necessary. ¹ A page must be indexed and eligible to be shown in Google Search with a snippet. ¹

Google also says you do not need to create new machine-readable files, AI text files or markup to appear in these features, and that no special schema.org structured data is needed. ¹ It recommends ordinary fundamentals instead: allow crawling in robots.txt, use internal links, offer a good page experience, and keep important content in text form. ¹

It adds that meeting the requirements does not guarantee crawling, indexing or serving. ¹ So be sceptical of anyone selling a guaranteed AI ranking, or a special format that promises to fix the problem.

Can the AI crawlers reach your site?

Check this first, because it is the one thing you can confirm today. The main AI companies run separate crawlers for search and for training, and your robots.txt can treat them differently.

Decide your policy for search and for training separately. Many B2B companies will want to be found, and may feel differently about training. Then look beyond robots.txt, because a firewall or bot protection rule at your host can block these crawlers without anyone noticing. Check your server logs, or your host's analytics, to see whether they actually arrive.

Fetchers acting for a user follow different rules. OpenAI says robots.txt rules may not apply to ChatGPT-User, and Perplexity says Perplexity-User generally ignores them. ^{2 4}

• OpenAI uses OAI-SearchBot to surface sites in ChatGPT search, and says sites that opt out will not be shown in ChatGPT search answers, although they may still appear as navigational links. ²
• OpenAI's GPTBot crawls content that may be used for training, and it can be blocked independently of search. ²
• Anthropic's ClaudeBot collects content that may be used for training, Claude-SearchBot crawls to improve search results, and Claude-User fetches pages when a user asks. Anthropic says its bots honour robots.txt. ³
• PerplexityBot surfaces and links sites in Perplexity's results, and Perplexity says it is not used to crawl for AI foundation models. ⁴
• Google-Extended controls whether content is used for training and grounding in other Google systems. Google points to nosnippet, max-snippet and noindex for limiting how content appears in search. ¹

Is your content really in the page?

Make sure your important content arrives as text in the page itself. Google recommends keeping it in text form. ¹ A sales page that needs JavaScript to show anything, or that hides its service descriptions in images and PDFs, gives crawlers less to work with.

A simple test is to view the page source, or switch off JavaScript, and check that your headline, service description and prices or terms are still there. Server rendering or static generation solves this for most modern sites.

How should the pages be written?

So that a stranger can tell what you do, for whom, and on what terms. In our view, the writing habits that help a buyer are the same ones that help a machine summarise you accurately.

Google's guidance on helpful content asks you to show who created a page, how it was created and why. It looks for first-hand expertise and depth of knowledge, and warns against content made primarily to attract search traffic. ⁵ For a B2B firm, that favours named deliverables, a clear process and honest limits over generic articles written to catch keywords.

• Answer first. Open each page with a one or two sentence direct answer to its main question.
• Use question-led headings where the page answers something buyers actually ask.
• Be specific. Say what you deliver, how long it takes, how it is priced, and where you work.
• Give each service its own page, so there is one clear place to cite.
• State your location and sectors in plain words, especially if you sell locally.
• Keep your company description the same everywhere it appears.

Does evidence help?

It helps people trust you, and it gives anyone summarising your site something solid to repeat. Cite primary sources for your claims, and link to them.

Avoid invented statistics and vague superlatives. A claim such as "delivered in about two weeks" with a defined scope is worth more than "best in Ireland". Real case notes, used with permission, are valuable if you have them. If you do not, say what you do and how you do it, and let the specifics carry the page. Pages like this one, with numbered sources, also make it easy for a reader to check the facts.

Does structured data help?

It still has value, but it is no shortcut. Google says no special schema is needed for its AI features, and that existing structured data should match the visible text on the page. ¹

Keep Organization, Service, Article and Breadcrumb markup accurate because it clarifies what a page is. schema.org documents these types. ⁶ Be careful with advice from a few years ago. Google's documentation records that the FAQ rich result is no longer shown in Google Search, ⁷ so do not add FAQ markup expecting that result, though it can still describe your page to other systems.

Should you add an llms.txt file?

It is cheap, optional and unproven. The llms.txt site describes it as a proposal, published in September 2024 to help agents use a website, and it remains open for community input. ⁸ Google says you do not need AI text files to appear in its features. ¹

We publish one on this site because it costs little when generated automatically from the same content as the pages, and so it cannot drift. Do not expect it to change your visibility, and do not pay much for it. If you add one, keep it accurate and generate it from your content.

Does speed still matter?

Yes, for people first. Google recommends a good page experience. ¹ Core Web Vitals set out the targets: Largest Contentful Paint of 2.5 seconds or less, Interaction to Next Paint of 200 milliseconds or less, and Cumulative Layout Shift of 0.1 or less, measured at the 75th percentile of page loads. ⁹

Measure on mobile as well as desktop, because the targets are checked separately for each. ⁹ A buyer who arrives from an AI answer has already been given a summary, and if the page is slow or unclear, there is little reason to stay.

What about other sites mentioning you?

Treat this as ordinary reputation work. Keep your details consistent in directories and on your company profiles, and earn mentions by publishing something worth citing.

Being cited elsewhere is hard to engineer, so we would not promise it. Accurate listings, a clear LinkedIn company page and useful guides are all under your control, and they help whether or not any AI system picks them up.

How do you measure it?

Imperfectly, so combine a few signals and be wary of precise claims. Check your web analytics for referrals from AI tools, and your logs for crawler visits. Then ask the tools directly.

Answers vary between runs and between people, so look for a trend and do not celebrate or panic over a single result.

• Keep a list of ten or so questions your buyers would really ask.
• Run them each month in the main AI tools, and note whether you appear and what is said.
• Correct your own pages if the summary is wrong.
• Use Search Console for Google, which remains your most reliable data on search visibility. ¹⁰

Frequently asked questions

Sources

1. Google Search Central: AI features and your website
2. OpenAI: Overview of OpenAI crawlers
3. Anthropic: Does Anthropic crawl data from the web?
4. Perplexity: Perplexity crawlers
5. Google Search Central: Creating helpful, reliable, people-first content
6. schema.org: Article
7. Google Search Central: FAQ structured data
8. llms.txt: The proposal
9. web.dev: Web Vitals
10. Google Search Central: Search Essentials