Block AI Training Crawlers in robots.txt
Adds rules to your virtual robots.txt that ask AI training bots such as GPTBot, ClaudeBot, CCBot and Google-Extended not to crawl your site.
Snippet Library
WordPress code snippets, ready to use
3,040 free snippets from the WPIDE team and the community. Copy the code, or add any snippet to your site in one click with WPIDE.
Block Bad Bots by User Agent
Returns 403 Forbidden to aggressive SEO crawlers and scrapers that waste server resources, matched by their user agent.
Bot Trap Link That Blocks Misbehaving Crawlers
Adds an invisible link that real visitors never see and well-behaved search engines are told to skip; any client that requests it anyway is blocked from the site for 24 hours.
Browser and CDN Caching for Feeds and Sitemaps
Sends cache headers on RSS feeds and XML sitemaps so feed readers, crawlers and CDNs can reuse them for an hour instead of making WordPress rebuild them on every request.
HTML Sitemap Shortcode
Adds an [html_sitemap] shortcode that lists all your pages and your posts grouped by category, giving visitors and crawlers one page that links to everything.
Keep Bots Out of Redirection's 404 Log
Stops the Redirection plugin from logging 404s caused by crawlers and vulnerability scanners (wp-login probes, .env and .php file hunts), so the log shows only real broken links.
Last-Modified Header and 304 Responses for Posts
Sends a Last-Modified header with each post's update time and answers repeat visits from crawlers and browsers with "304 Not Modified", saving bandwidth and crawl budget.
LearnDash: Stop Search Engines Indexing Lessons
Tells search engines not to index LearnDash lessons, topics, quizzes and certificates, which are usually locked and only show a login message to crawlers.
Leave Old Posts Out of the Yoast Sitemap
Removes posts that have not been updated for a number of years from the Yoast SEO post sitemap, so crawlers spend time on fresher content.
LifterLMS: Stop Search Engines Indexing Lessons and Quizzes
Adds a noindex instruction to LifterLMS lessons, quizzes and certificates, which are usually locked and only show crawlers a restriction message.
Markdown Versions of Posts for AI Crawlers, Plus llms.txt
Serves any post or page as clean Markdown when .md is added to its address (or ?format=markdown), and publishes an /llms.txt index of your pages and recent posts for AI tools.
Noindex URLs With Sorting and Filter Parameters
Adds noindex to pages opened with sorting or filter parameters (such as ?orderby= or ?filter_color=), so crawlers do not index endless variations of the same list.
Redirect Visitors by Country
Sends visitors from the countries you choose to a regional site or page, using the country header added by Cloudflare, CloudFront or your host, with search engine crawlers left alone.
Remove ?replytocom From Comment Reply Links
Stops comment reply links from creating ?replytocom= URLs that crawlers treat as duplicate pages, and redirects any such URL to the clean post address.
Stop Crawlers Paging Endlessly Through Old Archives
Returns a "not found" page for archive pages beyond page 50 (such as /page/873/) for logged-out visitors, which keeps bots from hammering your database with very slow deep pagination queries.
Tutor LMS: Noindex for Lessons, Quizzes and Assignments
Adds a noindex instruction to Tutor LMS lessons, quizzes and assignments, which are normally locked and only show crawlers a login wall.
rel="prev" and rel="next" Links on Paginated Archives
Adds rel prev and next link tags to paginated blog, category and archive pages, which Bing and other crawlers use to understand page series.