Grookey

Summary
This paper argues that NISQ-era quantum advantage demonstrations have repeatedly been shown to be classically simulable, due to the overlap between the circuit structures hardware can run and those classical algorithms can compress. It ties this pattern to theoretical results from 2024–2026, framing NISQ as a closed loop where hardware constraints and classical tractability align. The paper concludes that true quantum advantage lies in fault-tolerant systems.

Mathematical/empirical assessment
The paper references several theoretical results, such as the noise-induced shallow circuits theorem (Mele et al.) and classical simulability under geometric locality (Nelson et al.), which support its central claim. It also discusses specific experiments, like the fermionic dynamics demonstration, and highlights how structured input states enabled classical compression. The argument is built on a consistent pattern of empirical results and theoretical bounds.

Strengths
What I like here is the clarity with which the paper connects hardware limitations to classical tractability. The analysis of the fermionic dynamics experiment shows how the structure of the input states directly enabled the classical simulation. The paper also provides a coherent narrative about the NISQ era, tying together multiple results into a single, compelling argument.

Concerns
The paper’s conclusion that quantum advantage is only possible with fault tolerance is strong, but it doesn’t fully address alternative paths or potential future breakthroughs. The focus on de-quantization results is thorough, but it leaves little room for considering other interpretations of the data. Also, the paper assumes that all NISQ demonstrations fall into the same pattern, without deeply exploring exceptions.

Final decision
Weak accept

Summary
This paper introduces Partially Adjudicated Design-Based Supervised Learning (PA-DSL), a method for supervised learning when automated surrogate labels are corrected using a noisy human audit—only a subset of which receives expert adjudication. The framework formalizes a three-tier hierarchical measurement design (full-population surrogate → audited human labels → partially adjudicated gold) and constructs a nested AIPW-style pseudo-outcome (Eq. 5) that leverages known audit ($\pi_i$) and adjudication ($\rho_i$) probabilities. It proves design validity under mild ignorability and correctness assumptions (Assumptions 1–4), and demonstrates in both synthetic and Wikipedia Detox semi-synthetic experiments that PA-DSL maintains nominal coverage while reducing RMSE by 10–17% relative to adjudication-only DSL when the noisy audit labels contain recoverable signal.

Mathematical/empirical assessment
The estimator’s design validity is rigorously established via iterated expectation arguments (Appendix A), showing the corresponding equation in the paper regardless of nuisance specification—a key strength. Proposition 2 confirms asymptotic normality under $L^2$-consistent nuisance estimation, with no product-rate condition required thanks to known design weights. Empirically, Figures 1–2 and Tables 2–3 consistently show PA-DSL achieves near-zero bias and 94–97% coverage across scenarios, while improving efficiency over DSL-AdjOnly where inner features predict $A_i$ (e.g., RMSE ratio 1.18 in Detox). The reversion to DSL-AdjOnly in the “hard” regime (Fig. 2, lower CI bound ≈1.00) is mathematically grounded in Eq. (21) and reflects appropriate behavior—not overfitting.

Strengths
The paper excels at bridging theory and practice: it identifies a realistic, widespread labeling bottleneck (partial adjudication), grounds the solution in well-understood design-based principles (nested AIPW), and validates it across diverse settings—including real crowdworker disagreement on Detox. The clarity of Eq. (5), decomposing the estimator into full-frame regression, audit correction, and adjudication correction, makes the intuition immediately accessible. Assumptions are transparently stated and motivated (e.g., adjudication ignorability allowing disagreement-driven prioritization), and the ablation studies (Tables 5–6, Appendix C) thoughtfully probe robustness.

Concerns
One practical consideration arises from the reliance on known $\rho_i = \Pr(V_i = 1 \mid R_i = 1, Z_i)$. While the paper rightly notes this must be “known by design”, real-world annotation workflows sometimes use heuristic or post-hoc adjudication rules (e.g., “adjudicate all items where coders disagree and model confidence < 0.7”) without formally recording or validating the implied $\rho_i$. A brief discussion in the main text—perhaps as a footnote to Section 4 or a sentence in the Discussion—on how to diagnose or calibrate $\rho_i$ when the design record is incomplete (e.g., via pilot audit or sensitivity analysis around plausible $\rho_i$ ranges) would strengthen adoption guidance without altering the core theory.

Final decision
Strong accept