Add Rules to the Virtual robots.txt
Adds Disallow rules for internal search result URLs to the robots.txt file that WordPress generates.
Snippet Library
WordPress code snippets, ready to use
3,040 free snippets from the WPIDE team and the community. Copy the code, or add any snippet to your site in one click with WPIDE.
Allow Full Text and Video Previews in Search Results
Adds max-snippet:-1 and max-video-preview:-1 to the robots meta tag, letting search engines show longer text snippets and video previews for your pages.
Block AI Training Crawlers in robots.txt
Adds rules to your virtual robots.txt that ask AI training bots such as GPTBot, ClaudeBot, CCBot and Google-Extended not to crawl your site.
Exclude Specific Posts From the Rank Math Sitemap
Leaves the posts and pages you list out of Rank Math's XML sitemaps without changing their robots settings.
Keep Feeds Out of Search Results
Sends an X-Robots-Tag header that asks search engines not to index your RSS feeds, so feed URLs do not compete with your real pages.
Keep Filtered and Sorted Shop Pages Out of Search Engines
Adds noindex to shop pages that have sorting, price or attribute filters in the URL, so search engines index your real category pages instead of thousands of variations.
Keep REST API Responses Out of Search Results
Sends an X-Robots-Tag: noindex header with every REST API response so search engines do not index raw JSON from /wp-json/ next to your real pages.
Noindex Page 2 and Beyond With Yoast
Tells search engines not to index paginated archive pages (page 2, 3 and so on) while still following their links, using Yoast SEO's robots meta.
Noindex Paginated Archives With All in One SEO
Adds noindex to page 2 and later of archives through All in One SEO's robots meta while keeping links followed.
Noindex Paginated Archives With Rank Math
Sets page 2 and later of archives to noindex, follow through Rank Math's robots meta, keeping the first page indexable.
Noindex Posts in Selected Categories
Keeps posts from the categories you choose, such as internal news or short announcements, out of search engine results.
Noindex Search Result Pages
Asks search engines not to index your internal search result pages, which are thin, endless and a favorite target for spam links.
Noindex Selected Posts and Pages
Asks search engines not to index the posts and pages you list, such as thank-you, login or internal pages, while still following their links.
Noindex Staging and Development Sites
Adds noindex, nofollow to every page when WP_ENVIRONMENT_TYPE is not production, so staging copies never end up in search results.
Noindex Subsites With a Staging Path
Adds a noindex robots directive to any subsite whose address contains "staging", "dev" or "test", so preview copies inside a network never show up in search results.
Noindex Thin Archive Pages
Adds noindex, follow to date archives, attachment pages and page 2+ of archives, so search engines focus on your real content.