# Techniques: A Review and Collection of Metrics and Benchmarks for Quantum Computers: definitions, methodologies and software

## What the paper contributes (one paragraph)
This paper (arXiv 2502.06717) reviews the literature on performance metrics for quantum computers and assembles a curated collection that forms a comprehensive benchmarking suite. The metrics are organised into categories covering architectural properties, qubit quality, gate quality, circuit quality, application-level quality, speed, and stability, and also include measures tailored to non-gate-based architectures such as quantum annealers, boson sampling devices, and neutral atom analogue simulators. For every metric the paper supplies a consistent definition, a step-by-step measurement methodology, the main assumptions and limitations, and references. It links each metric to an open-source software package (QCMet) that implements the methodology against an emulator or real hardware, enabling reproducible and transparent evaluation under a single consistent noise model. The aim is to enable fair, unambiguous, objective comparisons of quantum computers across manufacturers and platforms, and to seed standardisation.

## Techniques offered
- **benchmark suite** - a curated collection of metrics spanning architecture, quality, speed, and stability, and component-level to application-level benchmarking, for holistic comparison of quantum computers across platforms.
  - guarantee: none; supplies comparative characterisation of device performance rather than a formal bound or proof.
  - quote: "We review the literature and use it to identify a collection of metrics that forms a comprehensive benchmarking suite covering the relevant aspects of device performance such as speed, quality, and stability, as well as performs different levels of benchmarking: from component-level benchmarking to application-level benchmarking."
- **metric suite** - a consistent definition format across all metrics with a transparent description of methodology, assumptions, and limitations, so that the same metric is evaluated and reported the same way.
  - guarantee: none; standardised methodology and reporting for transparency and reproducibility, not a statistical or cryptographic assurance.
  - quote: "This article addresses these challenges by providing a review of metrics and benchmarks for quantum computers and 1) a comprehensive collection of benchmarks allowing holistic comparisons of quantum computers, 2) a consistent format of the definitions across all metrics including a transparent description of the methodology and of the main assumptions and limitations, and 3) a reproducible approach by linking the metrics to open-source software used to evaluate them."
- **reproducible benchmarking software** - an open-source package (QCMet) implementing each metric's methodology against an emulator or hardware backend under a single consistent noise model, exposing all parameters needed to reproduce a reported value.
  - guarantee: none; provides reproducibility and transparency of reported metric values, not a proof.
  - quote: "The linked software package implements the methodologies to evaluate the metrics and is designed to be used in conjunction with this document, both as tutorial to provide all information on the implementation details of the described methodology, as well as a practical tool to evaluate the collection of metrics on a quantum computing emulator or on a hardware platform."

## Where it could apply
- Target primitive(s): sampling, VQE, QAOA, QPE, Trotterised simulation (application-level metrics cover general well-studied tasks); also individual gates, circuits, and idle qubits at the component level.
- Target application group(s): platforms_tooling; error_correction (most metrics also apply to logical qubits); chemistry_materials, optimization, quantum_ml (via application-level and well-studied-task benchmarks).
- Code family / hardware assumptions: hardware-agnostic across gate-based platforms (superconducting, trapped ion, neutral atom, photonic, semiconductor); dedicated non-gate-based metrics for quantum annealers, boson sampling, and neutral atom analogue simulation. Metrics generally assume Markovian noise and no leakage outside the qubit subspace.

## Caveats
None; the paper is open access and the metric methodologies and linked open-source software (QCMet) are publicly available and suppliable to external vendor or application results.