Semantic Product Search 2026: Vector Search for Shops

A substantial share of product searches in online shops returns no result - even though the requested product exists in the catalog. Classical keyword search based on BM25 fails at synonyms, intent variants and long-tail queries. Semantic product search with vector embeddings, an HNSW index and cross-encoder reranking closes this gap on a technical level. In Microsoft's Azure AI Search evaluation, hybrid retrieval with a semantic ranker on top reaches NDCG@3 60.1 against 43.8 for pure vector search (Microsoft Azure AI Search). For online shops and Shopware projects, a modern hybrid search architecture is therefore a direct lever on revenue, AOV and catalog utilisation.

Where classical keyword search fails

BM25 and its variants have been the standard for full-text retrieval for more than 30 years. They weight terms by frequency, document length and inverse document frequency - and deliver solid baseline results in many e-commerce scenarios. But as soon as a query deviates from the vocabulary stored in the catalog, result quality drops sharply. Synonyms, typos, size and quantity terms and descriptive searches without a product name regularly end on empty result pages. How large that share is in your own shop is something only your own search statistics can show: zero-result rate, search exit rate and the list of the most frequent empty queries belong in every shop report.

• Synonyms and jargon: 'trainer' vs. 'sneaker' vs. 'running shoe' - BM25 treats these as unrelated tokens.
• Intent variants: 'running shoes for asphalt' and 'road running shoes' describe the same thing but barely share terms.
• Long-tail queries: Natural phrasing like 'comfortable waterproof shoes for autumn hiking' rarely matches 1:1.
• Error tolerance: typos, alternative spellings, German compound words.
• Multilingual catalogs: international shops with queries across languages need cross-language representations.
• Attribute semantics: a 'machine washable' query misses products whose description only mentions '30-degree wash'.

The consequences are measurable: anyone who cannot find a result leaves the site or abandons the cart. The Baymard Institute averages 50 studies into a global cart abandonment rate of 70.22% (Baymard Institute) - bad search is one of the drivers. Anyone serious about search quality is intervening directly in the same KPIs addressed in checkout optimization.

Vector embeddings: semantics as numerical space

An embedding is a dense numerical representation of a text, image or mixed object. A language model maps a product title like 'Waterproof trail running shoes men size 43' to a vector with typically 384 to 2,048 dimensions - voyage-3-large covers the range from 256 to 2,048 dimensions (Voyage AI). Products with similar meaning - even with different wording - receive vectors that sit close together in space. Semantic search exploits this property: the query is embedded into the same vector space and compared to all product vectors via a distance metric (cosine, dot product, L2).

This covers synonyms, paraphrases and linguistic intent variants implicitly - without a hand-maintained dictionary. Embedding models are typically pretrained on MS MARCO, BEIR or domain-specific e-commerce datasets and can be fine-tuned on shop-specific product language.

Embedding models compared

The model landscape is broad: Sentence-BERT variants on MS MARCO, the E5 family (small/base/large), multilingual-e5-large for cross-language, and commercial APIs such as OpenAI text-embedding-3-small or voyage-3-large with 2,048 dimensions (Voyage AI). The right choice depends on catalog size, languages, latency budget and hosting model. Compact models such as E5-small (118M parameters) keep memory footprint and response time low, while large and multilingual models cover heterogeneous catalogs better - which combination holds up only shows in a benchmark on your own catalog data.

When latency is business-critical, distribution matters more than mean values: a hosted embedding service responds quickly on average, but the outliers at the top end decide how live search feels. Anyone not measuring p95 and p99 is optimising past the user experience. Self-hosted models on your own GPU take that uncertainty out of the chain and turn the one-time indexing of large catalogs into a predictable batch job - in exchange, operations and capacity planning move into the shop. Make-or-buy depends on volume, the latency requirement and compliance rules.

HNSW index and vector databases

A linear nearest-neighbour scan across millions of product vectors cannot meet millisecond budgets. This is where HNSW (Hierarchical Navigable Small World) comes in - a graph-based approximate nearest-neighbour index that navigates hierarchically across several layers and finds relevant neighbours in logarithmic time. Key parameters are graph connectivity (M), the build-time parameter (efConstruction) and the query-time parameter (efSearch or num_candidates), which trades recall against latency.

• OpenSearch: HNSW as a knn_vector field with tight BM25 integration inside the same query.
• Qdrant: Rust-based vector engine with payload filters, quantisation and hybrid search primitives.
• Weaviate: schema-driven vector DB with integrated modules for generative search.
• pgvector (PostgreSQL): HNSW and IVFFlat indexes directly inside the relation - attractive when the shop already runs on PostgreSQL.
• Milvus: scales to billions of vectors with strong quantisation options (PQ, SQ, BBQ).
• Lucene-based unified indexes: combine BM25 and HNSW in one segment - one index, one query, no separate synchronisation.

The database choice is less about recommendations and more about fit with the existing stack - all systems listed are production-ready. Four points decide: memory per vector, behaviour on filtered queries, the effort for index updates during live operation, and who runs it. For new programming projects, a small proof of concept on real catalog data beats synthetic benchmarks - a third-party benchmark reflects neither your catalog nor your query mix.

Hybrid search: BM25 + dense + RRF

Pure dense search degrades on exact product IDs, SKUs, brand and measurement terms: someone typing 'ISO 9001 stainless steel 304' does not want semantically similar products but exact term matches. Pure sparse search (BM25) degrades on synonyms and natural language. The answer is hybrid search: both retrievers run in parallel, and their result lists are fused.

The most robust fusion mechanism is Reciprocal Rank Fusion (RRF): the rank position of a document in both lists is added (inversely weighted) - without normalising score scales. In Microsoft's Azure AI Search evaluation, hybrid retrieval outperforms the single methods across all query types: on the customer datasets, pure keyword search reaches NDCG@3 40.6, pure vector search 43.8 and hybrid 48.4; with a semantic ranker on top the value rises to 60.1 - a lead of 11.7 points over hybrid without reranking (Microsoft Azure AI Search).

{
  "query": {
    "hybrid": {
      "queries": [
        {
          "multi_match": {
            "query": "waterproof running shoes men size 43",
            "fields": ["title^3", "brand^2", "description", "attributes.*"],
            "type": "best_fields",
            "fuzziness": "AUTO"
          }
        },
        {
          "neural": {
            "embedding": {
              "query_text": "waterproof running shoes men size 43",
              "model_id": "e5-small-v2",
              "k": 50
            }
          }
        }
      ]
    }
  },
  "search_pipeline": "hybrid-rrf-pipeline",
  "size": 20
}

The trick lies in sensible field weighting: title and brand via BM25 with higher boost, description and attributes primarily through the embedding. For a technical view on the data model, see the article on AI-optimised product data.

Robustness also benefits from query routing: for a pure SKU, an exact-match path with keyword boost takes over. For very short queries (1-2 tokens), BM25 carries more weight; for long, natural language queries, the dense side contributes more relevance. This switching is typically driven by a heuristic or a small classifier before retrieval and prevents hybrid search from delivering uniformly 'average' results instead of cleanly serving the query class at hand.

Cross-encoder reranking as a second stage

Hybrid retrieval typically returns 50-200 candidates. The cross-encoder is a second, more precise model that jointly encodes each query-product pair and produces a relevance score. Unlike a bi-encoder (one vector per side), the cross-encoder sees query and document simultaneously and reaches significantly higher precision - at the price of higher compute. That is why it is applied only to the top-K candidates from the first stage.

from sentence_transformers import CrossEncoder

reranker = CrossEncoder("cross-encoder/ms-marco-MiniLM-L-6-v2")

def rerank(query: str, candidates: list[dict], top_n: int = 20) -> list[dict]:
    pairs = [(query, c["title"] + " " + c["description"]) for c in candidates]
    scores = reranker.predict(pairs, batch_size=32)
    for c, s in zip(candidates, scores):
        c["rerank_score"] = float(s)
    return sorted(candidates, key=lambda x: x["rerank_score"], reverse=True)[:top_n]

Cross-encoders typically add 20-80 ms of latency, depending on model size and candidate count. With GPU inference or quantised models, the overall pipeline still stays within the live-search budget. The quality gain is largest on ambiguous queries and fine intent differences - exactly where classical search breaks down.

In practice a two-model setup works well: a small reranker (such as MiniLM-L-6) on the hot path with strict latency limits and a larger model (such as MonoT5 or bge-reranker-base) on the async path for recommendation lists, category pages or SEO programmes like programmatic SEO. Cross-encoders also benefit from feature enrichment: instead of encoding only the title and description, brand, main category and one or two key attributes can be passed as a text prefix. This measurably lifts NDCG without swapping the model.

Query expansion with LLMs

Before retrieval, an LLM can reformulate, expand or structure the query: generate synonyms, extract implicit attributes, fix spelling errors, or split a natural language query into a structured filter part plus a free-text part. This is particularly valuable for voice commerce and chat interfaces, covered in more depth in the article on voice commerce.

{
  "system": "You are an e-commerce query parser. Extract structured filters and a cleaned free-text query. Respond with JSON only.",
  "user": "comfortable waterproof running shoes men size 43 for autumn jogging",
  "expected_output": {
    "freetext": "waterproof running shoes autumn",
    "filters": {
      "category": "running shoes",
      "gender": "men",
      "size_eu": 43,
      "feature": ["waterproof", "comfort"]
    },
    "synonyms": ["running sneakers", "jogging shoes", "trail runners"]
  }
}

Combined with a synonym graph derived from catalog and search-log data, this builds a query-understanding stage that maps natural user language onto the catalog's domain vocabulary - without brittle rule systems. Caveat: LLM calls add latency and cost; for high-frequency queries, caching on the normalised query level is worthwhile.

Another building block is retrieval-augmented generation (RAG) for advisory queries: questions such as 'Which running shoe works for overpronation?' are answered against the top-K results plus relevant guide content. The model explains and points to concrete products - especially useful in advice-heavy categories. Thematically this connects to the article on generative engine optimization, which covers the SEO side of the same development.

Quantisation: cutting memory cost

A Float32 vector takes 4 bytes per dimension (Elastic) - at 1,536 dimensions that is roughly 6 KB per product. For an example catalog of 5 million products that is already around 30 GB, without replication and without graph overhead. Quantisation cuts this footprint considerably: scalar INT8 stores one byte per dimension, binary quantisation (BBQ) and product quantisation (PQ) go further.

MongoDB documents a memory reduction of 73% to 96% for quantised vectors while scalar quantisation preserves recall performance (MongoDB) - quantisation is no longer experimental but a production default. The right choice depends on recall requirements, candidate counts, and whether a rescoring stage can follow at higher precision.

Latency budget: 50-200ms for live search

Live-search UX demands response times below 200 ms - above that, waiting becomes perceptible. A realistic budget for the full pipeline looks like this:

These values are experience figures from projects and serve as orientation - catalog size, replication, filtering and network topology shift them case by case. For globally distributed shops, a look at edge caching strategies helps reduce search response times regionally.

Practical levers to hold the budget: keep the query embedding on a dedicated inference server with a warm model cache, cap num_candidates sensibly, restrict the cross-encoder to top-30 or top-50 and batch its inference. On the infrastructure side, HTTP/2 or HTTP/3, gRPC for internal hops and strict per-stage timeouts help - a slow reranker must not block search but fall back to the hybrid result. Monitoring p50/p95/p99 is mandatory, not optional.

Typical mistakes in semantic rollouts

• Dense-only instead of hybrid: the fastest route to 'search no longer finds my SKU'. Hybrid is the safe default.
• No evaluation suite: without offline metrics (nDCG@10, recall@50, MRR) and online A/B tests, no change is provable.
• Ignoring product data: an embedding is only as good as its input. Sparsely described products yield blurry vectors.
• No negative signals: click logs and purchases are valuable feedback - ignoring them throws away the most useful fine-tuning signal.
• Reranker always and everywhere: apply cross-encoders only to top-K, not to the full candidate list.
• Forgotten categories/filters: semantic hits must stay inside active filters and stock constraints.
• No fallback: if the vector service stalls briefly, BM25 must continue to serve - otherwise the search function disappears entirely.
• Model drift: language and catalog change. Without periodic re-indexing and re-evaluation, search ages silently.

A six-phase rollout roadmap

1. Measure the baseline: zero-result rate, click-through on top-5, search exit rate, conversion rate for searchers vs non-searchers. No baseline, no provable gains.
2. Clean the data: titles, categories, attributes, synonyms. Embeddings are only as precise as the product text - see PIM systems.
3. Choose a model and index: one-time embedding of the catalog, storage in an HNSW index, quantisation enabled. Define the re-indexing workflow.
4. Build the hybrid query: BM25 + kNN in parallel, RRF fusion, field-boost tuning. Include filter constraints.
5. Integrate the reranker: cross-encoder on top-50, measure latency, adjust model size. Offline evaluation against baseline.
6. A/B test and iterate: online rollout with traffic split, conversion measurement, fine-tuning on click and purchase signals. Then continuous monitoring.

Whether the effort paid off is shown by your own measurement: zero-result rate, search exit rate, conversion rate of search sessions and the revenue share coming through search. These four values belong on record before the rebuild and again in the same time window afterwards - only then can the effect be separated from seasonality and parallel campaigns. Personalisation moves revenue to a degree that varies by sector and execution maturity; what holds up in the end is the before-and-after comparison in your own shop.

Search as a product discovery engine

In 2026, semantic product search is no longer an experimental add-on but the infrastructural foundation of modern shops. The building blocks - embeddings, HNSW, hybrid fusion, reranking, quantisation - are production-ready, verifiably effective and integrable within reasonable latency budgets. Treating search as a mere filter facade gives up conversion and loses ground against shops that treat search as a primary discovery engine. For a strategic entry point, see the overview article on AI-powered product search and the piece on AI product recommendations - both describe complementary parts of the same discovery architecture.