Why deep learning god Bengio chose quantum computing

Last week, quantum computing company Multiverse Computing announced [1] the launch of a new partnership with Mila, the world's largest academic research center for deep learning, which will use quantum and quantum-inspired methods to advance artificial intelligence and machine learning (ML); the collaboration will cover multiple domains, with an initial focus on the biotechnology and pharmaceutical industries.

 

Multiverse's quantum experts will work with Mila's researchers to advance enhanced machine learning. multiverse uses tensor networks to support this work, and Mila has several researchers with expertise in the field: these networks use models based on quantum physics that can improve the speed and accuracy of training ML models.

 

01Mila, the world's largest deep learning center

 

Mila, the Montreal Institute for Learning Algorithms, was founded by Yoshua Bengio, a professor at the University of Montreal, one of the triumvirate of deep learning, and director of the Institute for the Valuation of Data (IVADO).Yoshua Bengio is a Canadian computer scientist who, in 2018, along with Geoffrey Hinton and Yann LeCun received the Turing Award, the "Nobel Prize of computing," for the conceptual foundations and engineering advances they laid over a 30-year period: significant advances due to the proliferation of powerful graphics processing unit (GPU) computers and access to massive data sets. In recent years, these discoveries have also laid the foundation for leaps and bounds in technologies such as computer vision, speech recognition, and machine translation.The specific research for which Bengio was awarded includes [2].

 

Probabilistic models of sequences: In the 1990s, Bengio combined neural networks with probabilistic models of sequences (e.g., Hidden Markov Models). These ideas were incorporated into the AT&T/NCR system for reading handwritten checks and are considered the pinnacle of neural network research in the 1990s, and modern deep learning speech recognition systems are extending these concepts.

 

High-dimensional word embeddings and attention: In 2000, Bengio wrote the landmark paper "A Neural Probabilistic Language Model", which introduced high-dimensional word embeddings as a representation of word meaning. Bengio's insights have had a huge and lasting impact on natural language processing tasks, including language translation, question and answer, and visual question and answer. His team also introduced an attention mechanism that has led to breakthroughs in machine translation and formed a key component of deep learning sequential processing.

 

Generative Adversarial Networks: Since 2010, Bengio's papers on generative deep learning, particularly the Generative Adversarial Network (GAN) developed with Ian Goodfellow, have spawned a revolution in computer vision and computer graphics: computers can actually create the original images that were once considered the hallmark of human intelligence for creativity.

 

Thanks to this, Bengio, Hinton and LeCun are sometimes referred to as the "godfathers of artificial intelligence" and the "godfathers of deep learning".

 

Yoshua Bengio

holds a bachelor's degree in electrical engineering, a master's degree in computer science and a doctorate in computer science from McGill University.

He has been appointed an Officer of the Order of Canada, a Fellow of the Royal Society of Canada, and a Marie Victor Award.

 

In October 2018, Yoshua Bengio was joined by quantum computing experts from IBM and MIT for a panel discussion; participants included Peter Shor, the man behind the most famous quantum algorithm. bengio said [3] that he is passionate about exploring new computer designs and asked the collaborative panelists questions about what quantum computers might be capable of . This established Bengio's connection to quantum computing.

 

Today, Mila brings together more than 1,000 researchers focused on machine learning and is known worldwide for its significant contributions to the field of deep learning, particularly in language modeling, machine translation, object recognition, and generative models. Today, Mila has an extensive network of enterprise partners across multiple verticals, including biology and pharmaceuticals, agri-tech and Industry 4.0.

 

 

Since its inception, Mila's applied research team has worked on more than 112 projects to help domestic and international companies accelerate the development and deployment of AI in their products and services. "The socially responsible development of AI is a fundamental part of Mila's mission." Mila said, "We will continue to drive social progress and the development of applications and projects that benefit society."

 

02Quantum Deep Learning: A Giant Leap Forward

 

At its core, quantum computing is about "solving classical problems through techniques that are computationally cheaper. Just as research in deep learning and quantum computing has developed in parallel in recent years, so many are now looking at the intersection of these two fields: quantum deep learning.

 

If one is completely new to quantum computing, an important introductory concept is the difference between classical computing (which we typically use for computational tasks) and quantum. On a classical computer, when executing a program, a compiler is used to convert the program's statements into operations on binary bits.

 

Unlike classical computers where the bits represent either 1 or 0 at all times, quantum bits can "superimpose" between these two states. A quantum bit "explores" one of its states only when it is being measured. "Superposition" is essential for quantum computing tasks: by superposition, quantum computers can perform tasks in parallel, without the need for a fully parallel architecture or GPU to handle parallel computing tasks. This is because if each superposition state corresponds to a different value, and if an operation is performed on the superposition state, the operation is performed on all states simultaneously.

An example of a superposed quantum state is as follows.

 

 

Where a and b refer to the probability magnitude, which gives the probability of projecting to a state once a measurement is performed. Superposition quantum states are created by using quantum logic gates.

 

So, what is entanglement again?

 

Just as superposition is an important principle in quantum physics, another key area to discuss is entanglement. Entanglement is the act of creating or causing interactions between two or more particles in some way, which means that the quantum states of these particles can no longer be described independently of each other, even when separated by a great distance. When particles are entangled, if one particle is measured, then the other particle with which it is entangled will immediately be measured in the opposite state (these particles have no local state).

 

As the understanding of quantum bits and entanglement evolves, Bell states can now be discussed. These are the maximum entangled states of quantum bits, which are.

 

 

Bell states are created using the following quantum circuits.

 

Bell state circuits. From: Perry's Temple of Quantum Computing

 

A Bell state circuit is shown here that accepts quantum bit input and applies Hadamard and CNOT gates to create an entangled Bell state.

 

Today, Bell states are used to develop a range of quantum computing applications. For example, Hegazy, Bahaa-Eldin and Dakroury proposed the theory that Bell states and ultra-dense coding can be used to achieve "unconditional security".

 

Having introduced quantum computing, we will now move on to discuss classical approaches to deep learning, in particular convolutional neural networks (CNNs) - CNNs have proven popular for tasks such as image classification because of their ability to build hierarchies of patterns, for example by first representing lines and then representing the edges of those edges of the lines. This allows CNNs to build on the information between layers and represent complex visual data.

 

CNNs also use pooling layers to reduce the size of the feature map, thus reducing the resources required for learning.

 

Convolutional Neural Networks (CNNs)

 

Returning to the topic of this paper, having defined classical CNNs, it is now possible to explore how quantum CNNs can take advantage of these traditional methods and extend them: a common approach to developing quantum neural networks is to develop a "hybrid" approach, introducing so-called "quantum convolutional layers " - a transformation based on random quantum circuits - as an additional component in a classical CNN.

 

Although quantum CNN approaches have been developed, one of the main challenges in this area is that the hardware required to implement the theoretical model is not yet available. In addition to this, there are challenges related to hybrid approaches, which introduce quantum convolutional layers as well as the classical computation of CNNs.

 

If we consider that one of the main benefits of quantum computing is the possibility of solving "classical puzzles through computationally cheaper techniques", then an important aspect of these solutions is "quantum acceleration": when exploring the benefits of quantum machine learning compared to When exploring the benefits of quantum machine learning, it is expected that quantum algorithms will have polynomial or even exponential speedup times compared to classical implementations. However, the "quantum acceleration" gain is limited for algorithms that require consistent decoding/encoding of classical data and measurements (e.g., QCNNs); there is limited information on how best to design protocols that encode/decode and require minimal measurements to benefit from "quantum acceleration". There is limited information on how best to design protocols for encoding/decoding and requiring the fewest measurements to benefit from "quantum acceleration.

 

More generally, entanglement has been shown to be an important property of quantum machine learning; QCNNs utilize strongly entangled circuits and can generate entangled states as their fully connected layer, thus allowing models to make predictions. Entanglement has been used elsewhere to assist deep learning models, such as entanglement to extract important features from images. Furthermore, it has been found that using entanglement in a dataset may mean that the model is able to learn from a smaller training dataset than previously expected, thus refining the so-called no free lunch theorem.

 

In summary, this paper provides a comparison of classical and quantum deep learning methods, as well as an overview of QCNNs that generate predictions using quantum layers, including strongly entangled circuits, and discusses the benefits and limitations of quantum deep learning, including the more general application of entanglement in machine learning.

 

The next steps in quantum deep learning, particularly QCNNs, can now be considered; in addition to this, we have seen advances in quantum hardware, with companies such as PsiQuantum aiming to develop quantum processors with millions of quantum bits. So, while we have seen the challenges associated with applying quantum neural networks, we can expect to see further developments in quantum deep learning as research continues at the "intersection" of deep learning and quantum computing.

 

Reference links:

[1]https://multiversecomputing.com/resources/Multiverse%20Computing%20and%20Mila%20Join%20Forces%20to%20Advance%20Artificial%20Intelligence%20with%20Quantum%20Computing

[2]https://www.acm.org/media-center/2019/march/turing-award-2018

[3]https://www.technologyreview.com/2018/10/03/139933/quantum-machine-learning-is-a-big-leap-away-at-least-for-now/

[4]https://towardsdatascience.com/quantum-deep-learning-a-quick-guide-to-quantum-convolutional-neural-networks-d65284e21fc4