AI bots get in
We check robots.txt for three kinds of bots: the indexes AI answers are built from (Google, Bing for ChatGPT and Copilot, Claude, Perplexity), bots that fetch your page live when a client clicks a link in a chat, and training bots. A blocked index means the company is invisible in that assistant; blocking training bots is your call.