Machine learning strategies for near-term quantum algorithms

Abstract

The central claim of this thesis is simple but underappreciated: the primary bottleneck in near-term quantum computing is not a shortage of algorithms; it is a lack of systematic decision-making. Which problem instances should we test? Which circuit architectures are worth training? Which hyperparameters actually drive performance, and which are distractions? Today’s quantum hardware sits in an uncomfortable middle ground. It is expressive enough to explore nontrivial quantum dynamics, yet it is too noisy and shallow for the theoretical speedups that motivate the field. This thesis argues that applying machine learning to classical control decisions in hybrid quantum classical methods is one of the most tractable ways to make these methods genuinely useful in this regime.

This argument is developed through four research questions, each addressing a distinct decision-making layer and illustrating the thesis.

The first asks whether the decisions inside a hybrid heuristic can be learned rather than hand-coded. Chapter 4 analyzes Recursive QAOA (RQAOA), a promising yet brittle approach to combinatorial optimization, and identifies regimes in which its fixed-elimination strategy fails. It then replaces that strategy with a reinforcement learning policy, RL-RQAOA, that adapts to instance structure. The resulting method performs better on the constructed failure cases while remaining competitive elsewhere. The broader point is that learned classical control policies can materially improve hybrid quantum heuristics without increasing quantum depth, which is a significant practical advantage on constrained hardware. The second question evaluates whether the quantum optimization algorithm benchmarking instances we use are principled. Chapter 5 observes that much benchmarking in this space is driven by convenience. Researchers test on whatever graphs are readily available and then report aggregate performance. Instead, the chapter proposes a generative pipeline that couples a permutation-invariant graph autoencoder with evolutionary optimization to deliberately construct Max-Cut instances that strongly discriminate between a quantum-inspired solver, RQAOA, and a strong classical baseline, Goemans-Williamson, in both directions. By connecting solver behavior to structural graph features, the chapter turns benchmarking from a static choice into an interpretable experimental tool. The generated instances also expose failure modes that random graphs often fail to reveal. The third question asks what changes when realistic noise constraints are taken seriously during circuit design, not only during execution. Chapter 6 develops RLQAS, a reinforcement learning framework for quantum architecture search that incorporates noise assumptions informed by real IBM device characteristics throughout training, rather than treating noise as a post hoc correction. Paired with an efficient Pauli transfer matrix simulation, the framework makes large-scale noisy circuit search computationally feasible. The circuits it discovers are compact, trainable, and achieve competitive accuracy on ground-state energy tasks. This suggests that hardware-aware circuit design is tractable and can yield materially different choices than noise-agnostic search. The fourth question asks: even when the circuit architecture is fixed, how much of what we attribute to quantum model design is actually due to hyperparameter sensitivity? Chapter 7 extends functional ANOVA to quantum neural networks to quantify which hyperparameter choices drive performance variance. The finding is clear. Learning rate, circuit depth, and data encoding/reuploading dominate, while many architectural micro-choices contribute far less than the literature often implies. Rather than stopping at diagnosis, the chapter turns these importance rankings into data-driven priors that accelerate hyperparameter optimization. It converts a critique into a practical improvement. Taken together, these contributions form a coherent toolkit for developing and evaluating hybrid quantum–classical methods in the NISQ regime. While the work does not eliminate the fundamental constraints of shallow, noisy hardware, it clarifies what can be improved immediately and how to do so in a reproducible manner. The broader goal is to help shift near-term quantum research from isolated demonstrations toward an engineering discipline, in which algorithms, benchmarks, and model-selection practices can be compared fairly, stress-tested rigorously, and iterated systematically as hardware matures.