New breakthroughs fuel quantum AI's leap in performance
At some point, when training a machine learning model on a quantum computer using a large number of parameters, more is better.
A groundbreaking theoretical proof shows that the use of a technique called overparametrization can enhance the performance of quantum machine learning for tasks that challenge conventional computers.
"Theory of overparametrization in quantum neural networks."
Diego Garcia-Martin, a postdoctoral researcher at Los Alamos National Laboratory, said, "We believe that our results will be useful for using machine learning to learn properties of quantum data, such as classifying phases of different substances in quantum materials research, which is very difficult on classical computers. " He is one of the co-authors of a new paper on the technique published by the Los Alamos team in Nature Computational Science.
Machine learning or artificial intelligence typically involves training neural networks to process information (data) and learn how to solve specific tasks. In short, we can think of a neural network as a box with knobs or parameters that takes data as input and produces an output based on the configuration of the knobs.
Garcia-Martin explains, "During the training phase, the algorithm updates these parameters as it learns, trying to find the optimal settings. Once the optimal parameters are determined, the neural network should be able to extrapolate what it has learned from the training examples to new and previously unseen data points."
However, both classical and quantum AI face a common challenge when it comes to training parameters: algorithms can reach suboptimal configurations during training and stagnate.
The advent of quantum computers brings exciting possibilities for performing data processing tasks using these devices. Quantum neural network (QNN)-based models are one of the most attractive learning solutions because of their versatility, ease of implementation, and promise to outperform classical models.
Despite the great promise of quantum neural networks, the theoretical understanding of them is still in its infancy and some of the fundamental results obtained for classical neural networks have not yet been derived to quantum neural networks. One such result is the phenomenon of hyperparameterization, which occurs when the number of parameters in a model exceeds a critical value.
Hyperparameterization is a well-known concept in classical machine learning that allows adding more and more parameters to avoid stagnation.
Until now, little is known about the impact of hyperparameterization in quantum machine learning models.
In this new paper, the Los Alamos team develops a theoretical framework for predicting the critical number of parameters at which a quantum machine learning model becomes overparameterized. At a certain tipping point, increasing the parameters drives a leap in network performance and the model becomes easier to train.
Martin Larocca, first author and postdoctoral researcher at Los Alamos, explains, "By establishing a fundamental theory of quantum neural network hyperparameterization, our research paves the way for optimizing the training process and achieving higher performance in practical quantum applications."
By taking advantage of quantum mechanical aspects such as entanglement and superposition, quantum machine learning promises to achieve faster speeds or quantum advantages over machine learning on classical computers.
To illustrate the Los Alamos team's findings, Marco Cerezo, the paper's senior scientist and the lab's quantum theorist, describes a thought experiment: a training process in which a hiker searches for the tallest mountain in a dark landscape. The hiker can only move in a specific direction and assess their progress by measuring altitude using a limited global positioning system.
In this analogy, the number of parameters in the model corresponds to the direction in which the hiker can move, Cerezo says: "One parameter allows for forward and backward movement, two parameters allow for lateral movement, and so on. Unlike our hypothetical hiker's world, the data landscape may have more than three dimensions."
"If there are too few parameters, a hiker can't explore thoroughly and may mistake a hill for the highest peak or get stuck in a flat area where taking any step seems futile. However, as the number of parameters increases, the walker can move in more directions in higher dimensions. What initially appears to be a localized hill may become a high valley between peaks. With more parameters, the walker can avoid getting stuck and find the real peak or the solution to the problem."
Hyperparameterization in QNN
It is worth noting that the results of this experiment were derived in a noise-free environment and do not include the presence of quantum hardware noise. Since noise is an inherent part of currently available quantum hardware, the question arises: how does the theorem change when noise is taken into account? The team performed preliminary simulations, which showed that for smaller noise levels, a quasi-superparameterization mechanism exists; in this case, the above theorem holds and can still be used to understand the hyperparameterization of QNNs.
However, more work is needed to understand the scope and limitations of this quasi-hyperparameterization mechanism.
Reference links:
[1] https://scitechdaily.com/a-leap-in-performance-new-breakthrough-boosts-quantum-ai/#google_vignette
[2]https://www.nature.com/articles/s43588-023-00468-5
[3] https://www.nature.com/articles/s43588-023-00467-6
