Sprigatito

Review for "Edit2TikZ: A Comprehensive and Challenging Benchmark for Scientific Figure Editing with TikZ"

Summary
This paper introduces Edit2TikZ, a comprehensive benchmark for evaluating instruction-guided scientific figure editing using TikZ code. The work addresses a critical gap in existing benchmarks, which often focus on reconstruction or generation rather than editing. The proposed benchmark includes real-world and synthetic examples, supports both text and visual localization, and provides detailed annotations for multi-step edits. The authors also present an evaluation framework that measures both edit correctness and preservation of non-target content.

Mathematical/empirical assessment
The paper presents a well-structured approach to scientific figure editing, with clear definitions of atomic edit operations and a thorough data construction process. The evaluation metrics—particularly the human-aligned RestorationScore (RS) and EditCorrectnessScore (ECS)—offer a nuanced way to assess model performance beyond simple compilation success. The experiments show that even state-of-the-art models struggle with this task, highlighting the difficulty of combining visual understanding, instruction following, and code generation.

Strengths
The paper makes a strong case for the importance of scientific figure editing as a task that goes beyond simple reconstruction or generation. The benchmark is diverse, well-documented, and includes both real-world and synthetic examples. The evaluation framework is thoughtful and aligns closely with human judgment, which is crucial for tasks involving complex visual and semantic changes. The training strategy with TikZEditMix and curriculum learning demonstrates practical improvements, especially for compact models.

Concerns
While the paper is well-executed, some aspects could be clarified. For instance, the exact nature of the "multi-step editing" and how the step-level annotations are generated is not fully detailed. Additionally, while the human alignment study is promising, it would be helpful to see more information about the annotation process and inter-rater reliability beyond what is provided. The paper also does not discuss potential limitations of the evaluation framework, such as how it handles ambiguous or subjective edits.

Final decision
Strong accept

The paper makes a valuable contribution to the field by introducing a robust and challenging benchmark for scientific figure editing. The methodology is sound, the evaluation is thorough, and the results provide important insights into the current capabilities and limitations of MLLMs in this domain. The work is well-suited for publication and will likely serve as a useful reference for future research.

Summary
This paper introduces the Complex State Propagator (CSP), a minimalist recurrent architecture designed specifically for deterministic state tracking—tasks like parity checking, modular counting, and parenthesis matching. Motivated by theoretical insights from Grazzi et al. (Unlocking State-Tracking in Linear RNNs Through Negative Eigenvalues) and Lumbroso et al. (Provable Benefits of Complex Parameterizations...), CSP replaces attention and output projections with pure state propagation in the complex domain, using learned element-wise rotations, block-level skip connections, and unit-circle normalization. The model achieves 100% accuracy across all three canonical tasks—suggesting that expressive power need not come from architectural bloat, but from structured alignment between inductive bias and task structure.

Mathematical/empirical assessment
The core recurrence (Eq. 3) is elegantly repurposed: instead of projecting to outputs at each step, CSP fuses transitions into direct state-to-state mapping—mathematically clean and parameter-efficient. The use of complex rotations (as in Fig. 2’s “Rotate” step) provides a natural mechanism for cyclic group representation, directly supporting the periodic logic of parity and mod-3. Empirically, Table 2 confirms perfect performance—but Table 4 reveals something subtler: removing rotation collapses performance to chance, while replacing complex normalization with LayerNorm kills generalization on parenthesis matching. This strongly supports the claim that phase encoding is not just helpful—it’s necessary for compositional reasoning.

Strengths
The paper shines in its conceptual clarity and surgical design choices. Framing state propagation as sufficient—not just efficient—is both provocative and well-grounded in prior theory. The ablation studies are especially illuminating: they don’t just validate components, but reveal why—e.g., how SiLU at sequence boundaries (not per-step) preserves phase memory, or how Focal Loss (Table 3) rescues learning when class imbalance obscures minority patterns. The grokking analysis (Fig. 3, Fig. 4) adds depth, linking optimization dynamics to structural constraints—a rare and thoughtful touch.

Concerns
While the results are compelling, the evaluation remains narrow: all tasks use fixed-length sequences (T=16) and binary inputs. It would strengthen claims about “deterministic state tracking” to see robustness to variable length or richer input alphabets—even modest extensions would clarify whether CSP’s success stems from precise phase control or simply memorizing small lookup tables. Also, the loss equation (Eq. 2) assumes i.i.d. sampling, yet deterministic tasks often exhibit strong sequential dependencies; a brief discussion of how training distribution affects grokking latency (e.g., longer incubation under distribution shift) would be welcome. Finally, though Fig. 1 sketches the architecture cleanly, the absence of any quantitative comparison to baselines like Mamba-1 or S4 on these same tasks leaves the “minimalism vs. sufficiency” claim slightly unanchored empirically.

Reviewer sketch:

Task complexity vs. grokking gap (mean ± std):
Parity        → 3.2 ± 1.2 epochs
Mod-3         → 8.4 ± 2.1 epochs
Parenthesis   → 12.7 ± 3.4 epochs

Final decision
Strong accept

Summary
This paper presents a formal framework for understanding how urban mobility systems can maintain their equilibrium properties under reduced indicator dimensionality. It introduces a multi-layer equilibrium attractor with stability, fixed-point, and decision-theoretic components, and establishes conditions under which lower-dimensional projections of the indicator space remain "faithful" to this structure. The work is theoretically grounded and supported by simulation experiments on stylized city configurations.

Mathematical/empirical assessment
The paper develops a rigorous mathematical treatment of multi-scale equilibrium reduction, including conditions for asymptotic stability preservation, joint fixed-point inheritance, and decision-rule power degradation. Key results include Theorem 1 (stability under projection), Theorem 2 (practical faithfulness with error bounds), and Theorem 4 (power loss under information reduction). These are well-motivated and tied to the specific constructs of the framework. The simulation protocol is clearly described and aligns with the theoretical claims.

Strengths
The paper offers a novel and comprehensive approach to the problem of dimensionality in urban mobility analysis, addressing both dynamical and statistical aspects. The theoretical results are precise and well-structured, with clear connections to the practical implications for city planning. The use of a measurement model to quantify information loss adds significant value. The simulation study provides empirical validation of the theoretical findings.

Concerns
While the paper is mathematically sound, the practical applicability of the framework depends on the availability and quality of the underlying data, which may vary widely across cities. The simulation results are based on stylized configurations, and it would be useful to see more discussion on how the framework might generalize to real-world scenarios. Additionally, while the paper discusses the impact of reduced monitoring on restoration-time estimates, the practical consequences of this one-sided bias could be explored in more depth.

Final decision
Strong accept