X-bench: End-to-End Filtered Vector Search Benchmark

X-bench is a controllable and end-to-end benchmark suite for evaluating filtered vector search—a core capability of modern vector databases that combines high-dimensional similarity search with structured scalar filtering. It is designed to reflect realistic filtered search scenarios, enabling fair, scalable, and explainable performance comparison across diverse database systems.

X-bench provides a modular, auto-scalable benchmarking pipeline covering data generation, query synthesis, and workload execution. The benchmark leverages statistically grounded methods—such as Johnson–Lindenstrauss–based random projection and distribution-aware query generation—to preserve vector-scalar correlations while flexibly controlling data dimension, scale, filtering rate, and filter correlation. This design enables systematic stress testing of vector databases under both static and dynamic workloads, including concurrent queries and online updates.

A unified ranking-based metric (Vrank) aggregates six-phase performance (indexing, query, concurrency, and update) into a single comparable score, providing a holistic view of system efficiency and robustness.

X-bench is system-agnostic and can be easily adapted to a wide range of vector databases and indexing strategies. It encourages combined software–hardware optimization and fair comparison under standardized configurations, promoting deeper understanding of the trade-offs in filtered vector search and driving innovation in next-generation vector data systems.

With its controllable workloads, realistic data distributions, and unified metric, X-bench offers the first principled framework for ranking and understanding filtered vector search performance at scale.