# Block AI Training Crawlers in robots.txt

> Adds rules to your virtual robots.txt that ask AI training bots such as GPTBot, ClaudeBot, CCBot and Google-Extended not to crawl your site.

- Type: PHP
- Runs: Run everywhere
- Category: [SEO](https://wpide.com/library/category/seo/)
- Tags: robots-txt, ai, crawlers
- Author: WPIDE Team
- Installs: 0
- Updated: 2026-10-28
- Page: https://wpide.com/library/block-ai-crawlers-robots-txt/

## Code

```php
// AI crawler user agents to disallow.
$wpide_ai_bots = array( 'GPTBot', 'ChatGPT-User', 'ClaudeBot', 'anthropic-ai', 'CCBot', 'Google-Extended', 'Applebot-Extended', 'PerplexityBot', 'Bytespider', 'meta-externalagent' );

add_filter( 'robots_txt', function ( $output, $public ) use ( $wpide_ai_bots ) {
    if ( ! $public ) {
        return $output;
    }

    $rules = "\n# Block AI training crawlers\n";

    foreach ( $wpide_ai_bots as $bot ) {
        $rules .= 'User-agent: ' . $bot . "\nDisallow: /\n\n";
    }

    return rtrim( $output ) . "\n" . $rules;
}, 20, 2 );
```

## Notes

robots.txt is a request, not an enforcement. A physical robots.txt file in the site root overrides the virtual one. Edit the list to allow bots you want.

## How to use it

Install the free WPIDE plugin (https://wordpress.org/plugins/wpide/), find "Block AI Training Crawlers in robots.txt" in its snippet library and add it in one click; or create a new PHP snippet in WPIDE (or any code snippets plugin) and paste the code above. More snippets: https://wpide.com/library/