A QANTIS reproducibility checklist for quantum hardware evidence.

A reproducible campaign begins with one sentence that can be falsified. For the latest QANTIS study, an appropriate primary claim is that a calibrated, shallow quantum evidence-estimation service can return sequential Tiger posteriors within a declared distance band on specified IBM Heron runs. “Quantum improves autonomy” is not reproducible because it does not name an observable, comparator, budget, or boundary.

Label every planned result as primary, supporting, or exploratory before execution. Primary rows test the central sequential claim. Supporting controls examine calibration, transfer, mitigation, longer horizons, or rare-event behavior. Exploratory runs probe scale or synthesis paths without inheriting the primary language.

The manifest should also state what will not be claimed: no total-runtime advantage without matched timing, no policy replacement when planning stays classical, and no production-scale conclusion from a small belief oracle. This negative scope prevents later metrics from drifting into stronger marketing language.

Quantum posterior quality cannot be reconstructed without the exact prior, observation likelihood, normalization rule, and reference posterior. Store them at every sequential step with enough precision to reproduce the reported Hellinger distance. If a posterior becomes the next prior, preserve both the unrounded internal value and the displayed value.

The decision comparator is separate. QANTIS uses an immediate Tiger reward rule to check whether the exact and hardware-derived posteriors choose the same action. Store the reward matrix, tie-breaking convention, threshold margin, selected actions, and value loss. Otherwise an “action agreement” number cannot be audited or tested under alternative costs.

Where a classical estimator can solve the same evidence problem, include it under matched statistical targets and compute cost. The first QANTIS paper demonstrates why: its tested Hungarian and GNN MTDA baselines reached 100% reported solution quality in under 0.1 milliseconds while the 11-variable quantum experiment returned about 64.1 ± 3.3% reported solution quality around 180 seconds. Baselines turn a quantum experiment into evidence instead of spectacle.

A source-level circuit describes intent; a transpiled circuit describes what the device received. A reproducibility bundle needs both, plus parameter values, qubit ordering, endianness, initial layout, routing method, optimization level, basis gates, compiler and SDK versions, transpiler seed, and any custom synthesis pass.

Resource fields should include logical qubits, physical qubits, total depth, two-qubit depth, one- and two-qubit gate counts, measurement count, and the actual ISA circuit serialization or a stable hash. For amplitude-amplification work, record the number and phase schedule of Grover or fixed-point stages and the oracle definition that marks accepted evidence.

This distinction is essential for QANTIS scaling claims. Its optimized four-state encoding and deeper UCGate or QSD stress paths can represent similar logical problem sizes while producing very different physical depth and accuracy. Without the compiled artefact, another team cannot reproduce the boundary or explain why mitigation helped one route and hurt another.

Record the backend name, processor generation, job identifier, submission timestamp, queue entry and exit, execution start and end, and calibration snapshot closest to execution. The snapshot should include available readout, single-qubit, and two-qubit error indicators for the physical qubits used, together with coherence values where the provider exposes them.

Preserve the mapping from logical to physical qubits and the provider-reported calibration identifier or retrieval time. A backend name alone is insufficient because two jobs on the same device can see different calibrations, routing, or queue conditions. Cross-backend transfer should be reported as a new context, not as a silent continuation of the original run.

QANTIS currently lacks a reliable matched end-to-end timing comparison for the sequential belief-update service. Future records should expose at least submit time, queue duration, device execution duration, and classical post-processing duration, with compilation and mitigation overhead separately identified. This supports both queue-inclusive and queue-exclusive analysis without mixing them.

A shot count is only one part of the resource budget. Record shots per circuit, circuits per estimator iteration, oracle calls, amplification depth, coarse and fine estimation shots, number of independent jobs, discarded shots, postselection rate, and any adaptive stopping rule. If one method allocates shots adaptively, publish the realized allocation at each sequential step.

The distinction explains the QANTIS FPAA result. The maximum Hellinger distance of 0.009 comes from 32,768 shots per step, while the 10,000-shot control is near 0.033. Guarded Grover uses an adaptive 8,000-to-16,000-shot range averaging around 10,000. Reproducing the table requires those per-step allocations, not one aggregate label.

Store raw bit-string counts before mitigation, after any readout correction, and after postselection as separate datasets. Record every clipping, renormalization, and boundary prior used by BIQAE. The final posterior should be derivable from immutable raw observations through versioned transformation steps.

For each step, store the incoming prior, observation, oracle parameters, circuit identifiers, raw estimator output, calibrated evidence, returned posterior, exact posterior, distance metrics, threshold margin, chosen action, and outgoing prior. A unique trajectory identifier should bind those records in order. Failed, retried, or excluded jobs stay in the chain with a reason code.

Report maximum, mean, and per-step posterior distance, not only the best row. Show confidence intervals or repeat distributions where possible. Decision reporting should include action agreement, immediate value loss, and distance to the nearest decision threshold. A zero value-loss result is informative only under the exact reward rule and trajectory that produced it.

Define an operational gate before deployment-like tests: acceptable posterior distance, minimum threshold margin, maximum compiled depth, calibration validity window, and a classical fallback path. If any gate fails, the service should abstain or use the classical update rather than returning an unqualified posterior.

A complete campaign release should contain a machine-readable manifest, environment lockfile, source and ISA circuits, parameter tables, raw counts, calibration snapshots, job identifiers, processing scripts, exact-reference code, derived metrics, figures, and a map from paper table rows to artefact paths. Checksums and a release tag make the bundle citable and immutable.

The QANTIS Community Edition repository is a useful public software surface, but it does not publicly bundle the self-service campaign artefact set an independent reviewer would need to reconstruct every reported row without private context. Public and private product modules can remain separated while the scientific evidence path is made complete through sanitized job data and reproducible derivations.

Negative results belong in the same bundle: deeper circuits where mitigation increased error, failed calibration windows, discarded jobs, and the 19-variable MTDA solution-quality decline. Recording them prevents survivorship bias and helps another team predict where not to spend hardware time. Both QANTIS papers are arXiv v1 preprints, so a release designed for independent replication is a direct path toward stronger review.