A T-shirt in six colours and five sizes is one product - but for search engines it quickly becomes around 30 nearly identical URLs. This is exactly where duplicate content arises in an online shop: product variants create near-duplicate pages that differ only in colour, size or material and spread the ranking value of the actual product page across many weak copies. The solution is not a block but an order: one canonical master URL, correctly set canonical tags and clean internal linking. A single figure shows how underrated the topic is: only around 65% (Web Almanac 2024) of all analysed pages set a canonical tag at all - in an analysis of 16.9 million websites (Web Almanac 2024). This article shows how to consolidate variants in search engine optimisation without degrading usability - and clearly distinguishes the topic from filter and facet URLs.
Why product variants become near-duplicate URLs
In e-commerce a variant is one expression of the same base product: the same T-shirt in black, blue and red, each in sizes S to XXL. Conceptually this is one item with options. Technically, however, many shop systems create a separate, retrievable URL for every selectable combination - such as /tshirt-basic/black/ or /tshirt-basic/?color=blue&size=l. One product thus becomes two dozen URLs whose content is identical apart from the colour or size label: the same description, largely the same image gallery, the same shipping and returns information, the same reviews.
The pattern scales unpleasantly. A fashion shop with 2,000 products carrying on average 6 colours and 5 sizes mathematically generates 60,000 variant URLs - for an assortment of 2,000 genuinely different items (illustrative example). Each of these URLs potentially competes with the others for the same search terms. Google groups such content under duplicate content: content that appreciably matches other content within or across domains, or differs only slightly (Google Search Central).
Duplicate content on a site is not grounds for action on that site unless it appears that the intent of the duplicate content is to be deceptive and manipulate search engine results.
Google Search Central, Duplicate content and Google Search
There is no classic duplicate content penalty - the widespread myth has been debunked (Google Search Central). The damage arises differently: when 30 variant URLs compete for the same rankings, links, clicks and relevance signals spread across many weak pages instead of one strong page. Google does consolidate these signals onto a chosen URL (Google Search Central) - but only if it chooses the right one. And you are better off making that choice yourself than leaving it to the algorithm.
Variants are not a facet problem - the distinction
Product variants are often lumped together with facet and filter URLs, but they are a different problem with a different solution. Filter URLs arise at category level through the combinatorics of filters and sort orders - a few thousand products turn into millions of theoretical URLs, and the answer is crawl control. That is precisely what the article on faceted navigation and crawl budget control covers in detail. Variants, by contrast, arise at product level: not millions of combinations, but a manageable, defined set of expressions of the same item - and the answer is consolidation, not blocking.
| Attribute | Facet / filter URLs | Product variants |
|---|---|---|
| Level of origin | Category / listing | A single product |
| Cause | Combinatorics of filters | Options like colour, size, material |
| Order of magnitude | Millions of theoretical URLs | A handful to a few dozen per product |
| Primary risk | Wasted crawl budget | Diluted ranking signals |
| Primary solution | robots.txt disallow, crawl control | Canonical master URL, canonical tag |
The distinction matters in practice because the wrong solution does damage: blocking variants wholesale via robots.txt denies Google the chance to read the canonical signal at all. Conversely, fighting filter URLs only with canonical tags saves no crawl budget. Both topics belong in a shared e-commerce SEO architecture, but each with its own tool.
What Google does with duplicates - and what it does not
Google does not assess every URL in isolation. The search process groups pages recognised as equivalent into a cluster and selects a canonical URL from it that is indexed and ranked on behalf of the group (Google Search Central). The remaining URLs of the cluster are consolidated onto that canonical version - including the signals pointing to them. That is fundamentally helpful. It becomes problematic when you send no clear signal about which URL should be the canonical one.
If you don't explicitly tell Google which URL is canonical, Google will make the choice for you, or might consider them both of equal weight, which might lead to unwanted behaviour.
Google Search Central, How to specify a canonical URL
This is exactly where the risk with variants lies. Without a clear specification, Google may pick /tshirt-basic/blue/ as the canonical version because that variant happens to have more internal links or clicks. Your generic, option-agnostic product page - the one that should actually rank for T-shirt Basic - then drops out of the index or is treated as secondary. The rel=canonical link is a strong signal, but not a directive (Google Search Central): Google additionally considers redirects, internal linking, the sitemap and further hints, and may decide against your specification in case of conflict.
Only around 65% of all pages set a canonical tag at all (Web Almanac 2024). The rest leaves the choice of the canonical version entirely to Google. For products with many variants that is an avoidable risk: a self-referencing canonical on the master URL and a canonical from the variants to that same master URL make the intent unambiguous.
Defining the canonical master URL
The first step is a decision, not a tag: which URL is the canonical one per product? The robust ground rule is: the option-agnostic product page is the master URL - that is /tshirt-basic/ without a preselected colour or size. It is the place where all variants are reachable, where the full description lives, and where category links, internal linking and the sitemap point. All variant URLs reference this one page via canonical.
Two implementation variants are clean. First: variants get no indexable URL of their own and are switched client-side on the master URL - the simplest and most robust solution. Second: variants have their own URLs for technical reasons but each carries a canonical to the master URL. Only where a variant has demonstrable search demand of its own does it become its own indexable page - more on that below.
| Approach | Recommendation | When suitable |
|---|---|---|
| Variants without own URL, client-side switch | Default | Colour/size without own search demand |
| Variant URL with canonical to master | Solid | System technically creates own URLs |
| Variant as its own indexable page | Selective | Proven search demand, sufficient depth |
| Block variants via robots.txt | Not recommended | Prevents reading the canonical |
| Variant via noindex | Only temporarily | Cleaning up already indexed URLs |
Which page is the master URL should not be a byproduct of click order but should follow from the data structure. In a well-maintained PIM system the base product is the leading entity and the variants hang beneath it as expressions. Whoever organises product data sovereignty cleanly has already defined the canonical URL unambiguously, before the first tag is set.
Setting canonical tags correctly
A canonical tag is quick to set and just as quick to set wrongly. For Google to consider the signal at all, the canonical URL must be reachable, indexable and free of contradictions. The following rules decide whether the tag takes effect or is ignored:
- Self-referencing on the master URL: the product page points to itself via canonical - not to the category.
- Variants point to the master URL: every variant URL names the same canonical master URL, consistently across all expressions.
- Use absolute URLs: fully with
https://and domain, not as a relative path. - Only one canonical per page: multiple
rel=canonicaldeclarations are ignored by Google (Google Search Central). - The canonical URL returns status 200 and is indexable: a canonical to a page blocked by
noindexorrobots.txtis contradictory. - Consistency with sitemap and internal linking: sitemap, navigation and canonical must not point to different URLs.
<!-- On every variant URL: canonical to the canonical master URL -->
<link rel="canonical" href="https://example.com/tshirt-basic/">
<!-- Wrong: canonical to the category instead of the product -->
<link rel="canonical" href="https://example.com/t-shirts/">
<!-- Wrong: relative path with leftover parameter, not the absolute master URL -->
<link rel="canonical" href="/tshirt-basic/?color=blue">The last item on the checklist is the most common finding in our audits: a canonical that contradicts a structure saying the opposite. If the sitemap lists /tshirt-basic/blue/, the navigation links there, but the canonical points to /tshirt-basic/, the shop sends conflicting signals - and Google decides for itself, often differently than intended (Google Search Central). How to systematically track down such contradictions is shown in the SEO audit guide for online shops.
When a variant deserves its own URL
Consolidating all variants wholesale is the safe default but not always the most rewarding choice. Some variants cover real, independent search demand and deserve an indexable landing page. A red summer dress gets searched, a summer dress in size 38 sorted by price does not. The assignment should not be made by gut feeling but by three testable criteria - the same ones that apply to category landing pages.
Own search demand
Is the variant searched as an independent term? Colour often has demand in fashion (black leather jacket), size practically never. Evidence comes from keyword data and the analysis of the internal shop search.
Sufficient assortment depth
A landing page needs stable results. A variant that is frequently sold out or contains a single product does not carry its own indexable page (project experience).
Standalone content
Is there anything to say beyond the colour option - material, care, styling tips? Without it the page remains a duplicate with no justification of its own in the index.
If a variant meets all three criteria it gets a descriptive path URL with a self-referencing canonical - for instance /dresses/red/ as its own category-like page, not as a product-detail variant. If it does not meet the criteria it remains part of the master URL. This separation prevents the most common mistake: indexing hundreds of thin colour pages that nobody searches for, in the hope of long-tail traffic. The hope is rarely fulfilled, the dilution effect reliably sets in.
A common reflex is to make every size variant indexable. In our experience this practically never pays off: T-shirt size L is not an independent search intent but a selection within an already found product. Sizes therefore belong in the master URL, not in the index. The case is different for special segments such as plus sizes, which are indeed searched as a category - but then as an editorially maintained category, not as a product variant.
Structured data for variants (ProductGroup)
Consolidating variants does not mean hiding the relationship between base product and expressions from Google - quite the opposite. Google supports dedicated structured data for this: the ProductGroup type bundles related variants, hasVariant names the individual expressions and variesBy the distinguishing attributes such as colour and size (Google Search Central). This lets Google understand that it is dealing with one product with options and not with dozens of separate items.
{
"@context": "https://schema.org",
"@type": "ProductGroup",
"name": "T-Shirt Basic",
"productGroupID": "TSHIRT-BASIC",
"variesBy": ["https://schema.org/color", "https://schema.org/size"],
"hasVariant": [
{
"@type": "Product",
"sku": "TSHIRT-BASIC-BK-M",
"color": "Black",
"size": "M",
"offers": { "@type": "Offer", "price": "19.90", "priceCurrency": "EUR" }
},
{
"@type": "Product",
"sku": "TSHIRT-BASIC-BL-L",
"color": "Blue",
"size": "L",
"offers": { "@type": "Offer", "price": "19.90", "priceCurrency": "EUR" }
}
]
}Structured variant data has a double benefit: it supports consolidation onto the master URL and at the same time enables variant-accurate offers in the search results - such as the correct price or availability of a specific expression. The prerequisite is clean, complete product data per variant - connecting that data to the front end is part of our custom development.
Internal linking and the variant switcher
Crawlers discover URLs through links - which is why internal linking helps decide how many variant duplicates reach the index at all. The classic mistake sits in the variant switcher: if every colour and size selection is delivered as a crawlable <a href> link, the shop creates exactly the duplicates it then has to catch again via canonical. It is cheaper not to let the URLs arise in the first place.
<!-- Problematic: every switcher is a crawlable link to a duplicate -->
<a href="/tshirt-basic/black/">Black</a>
<a href="/tshirt-basic/blue/">Blue</a>
<!-- Better: one indexable page, switching without new crawlable URLs -->
<button type="button" data-variant="black">Black</button>
<button type="button" data-variant="blue">Blue</button>- Prominently link only the master URL - from category, navigation and editorial content.
- Deliver switching without new crawlable URLs where variants have no search demand of their own.
- Include only master URLs in the XML sitemap, no variant parameters.
- Indexable variants (if any) should be linked cleanly like their own pages and canonicalised self-referencingly.
- After every template release, check whether new links point to variant duplicates.
Internal linking also distributes link equity to the pages that should rank. If everything points to the master URL, authority concentrates there - instead of scattering across 30 variants. A side effect concerns performance: fewer crawlable duplicates mean fewer unnecessary server requests - relevant when you are decluttering third-party scripts to improve shop performance anyway. For the user side of variant selection - how customers find the right expression - it is worth additionally looking at an AI product advisor with guided selling.
Implementation in Shopware and other systems
Most shop systems ship with a variant concept. In the Shopware Community Edition, for instance, variants hang as expressions beneath a main product, defined via properties such as colour and size; the system can generate variant URLs and set canonical declarations. What matters is not the presence of the feature but its correct configuration: which page is the master URL, whether variants get their own URLs, and where their canonical points. In practice these switches are often left at their defaults - and the default rarely fits the individual assortment and search-demand situation.
- Inventory: run a full crawl and record the actually reachable variant URL patterns - not the planned ones, the real ones.
- Define the master URL: per product, set the option-agnostic page as the canonical URL, anchored in the product data.
- Check demand: per variant dimension, prove whether there is search demand of its own - line up keyword data against internal search queries.
- Set the canonical: self-referencing on the master URL, consolidating on all variant URLs.
- Add structured data: ProductGroup with hasVariant and variesBy for the variant relationship.
- Close the linking: switcher without crawlable duplicates, sitemap with master URLs only.
- Clean up already indexed duplicates: noindex temporarily, wait for de-indexing, then consolidate.
- Measure again: observe index coverage and the chosen canonical URL in the Search Console over several weeks.
A realistic horizon is several weeks until index coverage and canonical assignment settle again - depending on shop size, crawl frequency and starting point. For shops with grown variant logic it is worth looking at the whole e-commerce architecture, because variants rarely arise in isolation. We support the implementation in the front end and in the product-data pipeline as part of our Shopware development.
Variant URLs as ranking capital instead of dilution
Product variants are not a problem you lock away. They are an ordering task. The catalogue already contains the right structure - a base product with expressions - it just often translates technically into a URL space where one strong page competes against thirty weak copies. The task is to concentrate the authority: one canonical master URL that draws all signals to itself, and only there indexable variants of their own where genuine search demand carries them.
The lever is manageable and the effect lasting. While only around 65% of all pages set a canonical at all (Web Almanac 2024), every shop that sets up variant consolidation cleanly wins twice: clearer rankings for the product pages that count, and less pointless crawl and server load. We support the analysis and implementation as part of our SEO services - from taking inventory of the variant URL space to rolling out the canonical architecture. Talk to us if you want to know how many of your product pages currently compete against their own variants.
This article draws on data from: Web Almanac 2024 (HTTP Archive), SEO chapter (share of pages with a canonical tag of around 65%, data base of 16.9 million analysed websites), Google Search Central - Duplicate content and Google Search (definition of duplicate content, no ranking disadvantage without deceptive intent), Google Search Central - Consolidate duplicate URLs / How to specify a canonical URL (clustering and choice of the canonical URL, consolidation of signals, rel=canonical as a strong signal rather than a directive, only one canonical per page, automatic choice without an explicit specification) and Google Search Central - Product (ProductGroup) structured data (ProductGroup, hasVariant, variesBy for product variants). Illustrative examples are marked as such and serve to clarify; statements marked as project experience are based on implementation projects. The figures cited relate to the respective collection periods and may vary by industry, shop size and point in time.
No, there is no classic duplicate content penalty for ordinary variants. Google explicitly states that duplicate content is not grounds for action as long as there is no deceptive intent (Google Search Central). The damage is different: when many nearly identical variant URLs compete for the same search terms, ranking signals spread across many weak pages. Google does consolidate these signals onto a chosen canonical URL (Google Search Central), but you should steer that choice yourself via a clean canonical.
Usually not. The safe default is to consolidate all variants onto one canonical master URL. An indexable URL of its own typically pays off only when a variant has demonstrable search demand, offers sufficient assortment depth and carries standalone content. Colour meets this in some industries, size practically never. The decision should be made per dimension based on keyword data, not wholesale.
Filter URLs arise at category level through the combinatorics of filters and turn a few thousand products into millions of theoretical URLs - here it is about crawl budget and the right answer is crawl control. Variants arise at product level and comprise a manageable set of expressions of the same item - here it is about diluted ranking signals and the right answer is consolidation via canonical tags. Details on filter URLs are covered in the article on faceted navigation.
A robots.txt disallow is typically counterproductive for variants. If Google may not crawl the URL, it cannot read the canonical tag in the source - the consolidation signal is lost. For variants the canonical tag is usually the right tool, complemented by internal linking that keeps duplicates from becoming crawlable in the first place. A robots.txt disallow stays reserved for the category and filter level.
They do not disappear the instant a canonical is set, because rel=canonical is a signal and not an immediate directive (Google Search Central). A sensible order in our experience: first set the consolidating canonical and give Google several weeks to re-evaluate. If unwanted duplicates remain in the index, a temporary noindex on the variant URLs can speed up de-indexing before switching to canonical only.
Two sources suffice for a first assessment. First the Search Console: under index coverage, check which URL Google has chosen as canonical per product - if it deviates from your master URL, there is work to do. Second a site search for the product name: if several variant URLs of the same product appear, that points to missing or contradictory canonicals. The systematic approach is described in the SEO audit guide for online shops.