Introduction
You spent weeks writing product descriptions, fine-tuning your pricing, and building a catalog your competitors can’t easily replicate. Then an AI training bot crawls your entire store overnight – product titles, descriptions, images, prices, and reviews – and feeds it into a dataset used to train a model that powers a competitor’s storefront.
This isn’t a hypothetical. It’s happening to WooCommerce stores every day in 2026.
AI bot traffic grew dramatically through 2025, and the crawlers have evolved far beyond the well-behaved bots that politely respect your robots.txt file. Competitor intelligence scrapers now use headless browsers, rotate IP addresses through residential proxies, and increasingly use AI agents that can solve CAPTCHAs and mimic human browsing behaviour. Meanwhile, established AI crawlers like GPTBot, ClaudeBot, Bytespider, and Meta-ExternalAgent are hitting WooCommerce product and checkout pages dozens of times per week.
The good news: there are practical, layered defences you can put in place today – without being a developer – that make your store a significantly harder target than the next one.
This guide covers what AI scrapers are actually taking from your store, how to detect whether you’re being scraped right now, what robots.txt can and can’t do, and how to enforce real protection using a dedicated bot protection plugin.
What AI Scrapers Are Actually Taking from Your WooCommerce Store
Before choosing a defence strategy, it helps to understand what different types of bots are after. Not all scrapers have the same goal.
Training crawlers fetch your content to feed into AI model training datasets. GPTBot (OpenAI), ClaudeBot (Anthropic), Google-Extended, CCBot (Common Crawl), and Bytespider (ByteDance) are the most active examples. They’re primarily interested in your product descriptions, blog posts, FAQ pages, and any original prose on your store. Most of these crawlers are supposed to respect your robots.txt – though compliance is inconsistent, and some well-known crawlers have been caught bypassing it entirely.
Competitor intelligence scrapers are the more commercially damaging threat. These are custom-built bots – often using headless Chrome or Playwright — that impersonate real browsers to harvest your prices, inventory levels, review counts, and product images. They ignore robots.txt, rotate IPs, and are explicitly designed to evade detection. A competitor using one of these can monitor your pricing in near-real-time and undercut you automatically.
Bandwidth-draining crawlers don’t necessarily steal content, but they consume server resources at scale. A single AI shopping agent can visit 40 or more product pages in minutes while converting zero times. These sessions are often counted as real visitors in GA4, inflating your traffic numbers while quietly skewing your conversion rate downward and increasing server load – especially on uncached product pages.
Understanding which type of scraper is targeting you determines which layer of defence you prioritise.
How to Detect Whether Your Store Is Being Scraped Right Now
The most reliable way to confirm scraping is to check your server access logs. Most hosting providers give you access to raw logs – look for repeated requests from user-agent strings containing GPTBot, ClaudeBot, Bytespider, PerplexityBot, Meta-ExternalAgent, or CCBot. High request volume from a single IP in a short window, especially to product and catalog pages, is a strong signal.
A few other warning signs to watch for:
Unexplained bandwidth spikes – particularly during off-peak hours when human traffic is low. If your hosting dashboard shows bandwidth usage that doesn’t correlate with traffic, bots are a likely cause.
Conversion rate drops with no change to your store – if your WooCommerce conversion rate has fallen since early 2026 and nothing about your store changed, AI bot sessions being counted as real traffic in GA4 can artificially inflate the denominator, making your funnel look worse than it is.
Competitors launching with suspiciously similar product copy – if a competitor’s store appears with product descriptions that closely mirror yours, price-scraping or content scraping is worth investigating.
Slow product page load times during off-peak hours – aggressive bots hitting uncached WooCommerce product URLs generate cache misses, forcing PHP execution and database queries for every bot request.
What Robots.txt Can and Can’t Do in 2026
The first instinct for most store owners is to add AI bots to their robots.txt file. It’s a reasonable starting point – but it has significant limits in 2026 that are worth understanding before you rely on it alone.
Robots.txt works on an advisory basis. It signals your preferences to well-behaved crawlers, and many of the major AI operators – OpenAI, Anthropic, Google, Apple – do generally respect it. Adding Disallow: / for GPTBot and ClaudeBot will reduce your exposure to those specific crawlers.
However, robots.txt does nothing against the scrapers that pose the biggest commercial risk – competitor intelligence bots that explicitly ignore it, impersonate browsers, and rotate their IP addresses to avoid detection. A security researcher studying scrapers found that several of the most active crawlers in 2026 never check robots.txt at all, despite being among the highest-volume bots on the open web.
There’s also a nuance worth understanding before you block everything: not all AI bots are harmful to your business. AI search bots – the ones that retrieve your pages in real-time so an AI assistant can cite your store when a shopper asks a question – are increasingly a source of referral traffic. Blocking GPTBot (which trains ChatGPT) is different from blocking ChatGPT-User or OAI-SearchBot (which drive AI-referred shoppers to your product pages).
The smarter approach is a layered defence: use robots.txt and noai meta directives to signal your preferences to compliant bots, then enforce real blocking at the server and application level for everything else.
How to Block AI Scrapers from Your WooCommerce Store
The most effective protection for a WooCommerce store in 2026 combines three layers: bot blocking at the server level, rate limiting at the application level, and behavioural detection for scrapers that evade both. ScraperBlock by WindCodex handles all three in a single plugin.
Here’s how each layer works and when to use it.
Layer 1: Bot signature blocking
The first and most immediate defence is blocking known bot user-agents at the server level – before WordPress and WooCommerce even load, so no server resources are wasted serving content to the bot.
ScraperBlock includes a database of 50+ AI and scraper bot signatures and blocks them via two mechanisms:
robots.txt directives – ScraperBlock automatically writes the appropriate Disallow entries to your robots.txt file for all covered bots, signalling to compliant crawlers that your content is off-limits.
.htaccess blocking – For Apache-based hosting, ScraperBlock can add user-agent blocking directives directly to your .htaccess file. This blocks matched bots at the server level before PHP executes, making it significantly more efficient than application-level blocking.
To enable, go to WooCommerce → ScraperBlock → Settings → Core Protection and toggle on Enable Protection. Turn on the robots.txt and .htaccess options from the same panel.
Layer 2: Rate limiting
Signature-based blocking catches known bots, but scrapers that rotate user-agents or mimic browser behaviour slip through. Rate limiting catches them by behaviour – too many requests in too short a window triggers a block, regardless of what the bot claims to be.
ScraperBlock’s default global rate limit is 120 requests per minute (RPM), which is configurable to suit your server capacity. For WooCommerce stores specifically, it includes a stricter catalog limiter – a separate, tighter RPM threshold applied to your product and category pages, where scraping is most damaging.
Configure rate limits under WooCommerce → ScraperBlock → Settings → Core Protection → Rate Limiting. Start with the defaults and tighten them if your logs show bots slipping through.
Layer 3: Behavioural detection and advanced deception (Pro)
For scrapers sophisticated enough to rotate IPs and mimic browser patterns, ScraperBlock Pro adds two more powerful layers.
Behavioural detection analyses the patterns of incoming requests – request timing, page sequences, interaction signals – and flags automation patterns that bypass traditional signature-based filters. A bot visiting 40 product pages in 90 seconds with no JavaScript execution will be detected and blocked regardless of what user-agent string it presents.
Content poisoning is ScraperBlock’s most distinctive Pro feature. Instead of simply blocking a matched bot with a 403 error – which tells the bot operator they’ve been detected – content poisoning returns fake, decoy data. The bot thinks it scraped your real product prices and descriptions. It actually received deliberately corrupted data. This wastes the scraper’s resources and makes any data it collected commercially useless, while giving you no indication of your detection in server logs.
Honeypot traps place invisible links on your pages – invisible to real visitors but visible to bots that parse your HTML. Any request that follows a honeypot link is immediately identified as automated and can be blocked or fed poisoned content.
Layer 4: noai meta directives
Beyond blocking bots at the server level, ScraperBlock outputs noai and noimageai meta directives on your pages – signals that tell AI training crawlers your content and images are not available for use in training datasets. These are increasingly being recognised by AI operators as the standard way to communicate content licensing preferences.
This is especially important for product images. Your original photography is the hardest thing to replace if a competitor scrapes and re-uses it.
Access Control: IP and Geo Blocking
In addition to bot-signature and behavioural detection, ScraperBlock Pro gives you two direct access control tools.
IP allowlist and blocklist – If you identify specific IPs or CIDR ranges that are scraping your store, you can add them directly to ScraperBlock’s blocklist. Manual CIDR support means you can block entire ASN ranges used by known scraper hosting providers. An allowlist ensures your own monitoring tools, trusted partners, and internal IPs are never caught by rate limiting.
Geo-based blocking – Block access to your store data from specific countries at the ScraperBlock level. This complements GeoBlock’s product-level country restrictions – ScraperBlock’s geo blocking can be applied to specific endpoints or your entire store to reduce high-volume scraper traffic originating from specific regions.
Block scheduling – Pro users can set time-based schedules for blocking rules. If your logs show scraping spikes at consistent times (common with automated crawlers on fixed schedules), you can activate stricter rules during those windows without affecting daytime shopping traffic.
Monitoring: Real-Time Threat Feed and Email Alerts
Blocking bots is only half the picture – knowing what was blocked and when is what allows you to tune your rules over time.
ScraperBlock Pro includes a real-time threat feed in your WordPress dashboard: a live event log showing each blocked request, the source IP, the detected bot signature, and the action taken (blocked, rate-limited, or poisoned). This gives you immediate visibility into what’s hitting your store and whether your rules are catching it.
Email alerts notify you when blocking activity spikes above a threshold – useful for catching a new scraping campaign before it consumes significant bandwidth.
The analytics dashboard surfaces trends: which rules are firing most often, which IPs or user-agents are responsible for the most blocked traffic, and how protection activity has changed over time.
Free vs. Pro: What You Get Without Paying
ScraperBlock has a free version available on WordPress.org that covers the core bot signature database (50+ AI bots), robots.txt directives, noai meta tags, per-page control, rate limiting, and .htaccess blocking. For most small to medium WooCommerce stores, the free version meaningfully reduces scraper exposure.
The Pro version adds the features needed to catch sophisticated scrapers that evade signature-based detection: behavioural detection, honeypot traps, content poisoning, real-time threat feed, email alerts, geo-based blocking, IP allowlists and blocklists, block scheduling, WooCommerce endpoint protection, and multisite network support.
Pro licences start at $99 for a single site with a 14-day money-back guarantee. See ScraperBlock Pro →
Frequently Asked Questions
Will blocking AI bots hurt my search rankings?
Blocking AI training bots (GPTBot, ClaudeBot, Google-Extended) does not affect your Google search rankings – Googlebot and AI crawlers are separate systems that don’t interact. The bots to be careful about blocking are AI search bots (OAI-SearchBot, PerplexityBot, Claude-SearchBot) – these are the ones that route AI-referred shoppers to your store. ScraperBlock targets training crawlers and malicious scrapers while leaving search-oriented bots configurable.
Doesn’t robots.txt already block AI crawlers?
robots.txt signals your preference to compliant bots, and major AI operators largely respect it. But it has no enforcement power against scrapers that ignore it – and many commercial scraper bots do exactly that. Blocking at the .htaccess level and via behavioural detection is the only way to enforce rules against non-compliant bots.
How is ScraperBlock different from Cloudflare’s AI bot blocking?
Cloudflare’s one-click AI bot toggle blocks known AI crawlers at the CDN level, which is effective for declared bots. ScraperBlock works at the WordPress and WooCommerce application level and adds capabilities Cloudflare doesn’t have: WooCommerce-specific rate limiting, content poisoning, honeypot traps, and per-page noai directives. They complement each other well – Cloudflare handles declared bots at the edge; ScraperBlock handles sophisticated scrapers and undeclared bots at the application level.
Can scrapers get around rate limiting by using multiple IPs?
Yes – IP rotation is a common scraper technique. Rate limiting per IP is one layer of defence, not the only one. ScraperBlock Pro’s behavioural detection catches scrapers that rotate IPs by analysing request patterns across the session rather than relying on IP-based signals alone. Content poisoning provides a fallback even when detection is uncertain – if a scraper does get through, it receives corrupted data rather than your real content.
Will ScraperBlock slow down my WooCommerce store?
No. ScraperBlock’s .htaccess-level blocking stops matched bot requests before PHP executes, which actually reduces server load rather than adding to it. Rate limiting and behavioural detection add minimal overhead – far less than the load generated by the bot traffic they prevent.
Does ScraperBlock work with caching plugins like WP Rocket?
Yes. ScraperBlock is designed to work alongside common caching setups. .htaccess-level blocking fires before the cache layer, so blocked bots never consume cached page resources.
Wrapping Up
AI scrapers in 2026 range from well-behaved training crawlers that respect robots.txt to sophisticated competitor intelligence bots that actively evade detection. A single layer of defence – robots.txt alone, or a rate limiter alone – won’t stop all of them.
The most effective approach is layered: bot signature blocking at the server level to catch declared crawlers, rate limiting to catch high-volume automated traffic, behavioural detection for scrapers that rotate IPs and mimic browsers, and content poisoning to neutralise the ones that slip through everything else.
ScraperBlock gives you all of these in a single WooCommerce plugin, with a free version that handles the basics and a Pro tier that adds the advanced detection and deception tools needed for serious protection.
Install ScraperBlock free from WordPress.org →
Explore ScraperBlock Pro →