DeepMind uses AI to simulate matter at the quantum scale for the first time Russian scientists NO!
In December 2021, DeepMind, an artificial intelligence company of Google, announced its first simulation of matter at the quantum scale using artificial intelligence, with related research published in the journal Science [1]. They proposed a neural network model DM21 to approximate the energy density functional component of density functional theory (DFT), which describes the quantum mechanical behavior of molecules. In February this year, DM21 was officially open-sourced.
In short, the DeepMind team showed how to use neural networks to describe electronic interactions in chemical systems more accurately than existing methods. The journal Nature even hailed it as one of the most valuable technologies in chemistry.
Now, eight Russian researchers have commented in the journal Science [2] that the ability of DeepMind AI to generalize the behavior of such systems does not follow from the published results and needs to be revisited.
01Density generalization theory (DFT)
Solving some of the major challenges of the 21st century, such as producing clean electricity or developing high-temperature superconductors, requires us to design new materials with specific properties. To do this on a computer, one needs to model electrons, the subatomic particles that control how atoms combine to form molecules and are also responsible for the electric currents in solids.
Understanding where electrons are located in a molecule can go a long way toward explaining its structure, properties and reactivity. Chemists use density flooding theory (DFT) methods to build accurate and computationally efficient models of molecules and materials.
DFT is actually an approximation of Schrödinger's equation. Nearly a century ago, Erwin Schrödinger proposed the Schrödinger equation for describing the behavior of quantum particles. However, applying this equation to electrons in molecules is challenging because all electrons repel each other. This seems to require keeping track of the probability of each electron's position - a very complex task even for a small number of electrons.
A major breakthrough came in the 1960s when two men, Pierre Hohenberg and Walter Kohn, realized that it was not necessary to track each electron individually. Instead, knowing the probability of any electron at each position (i.e., the electron density) was sufficient to calculate all interactions accurately. kohn won the Nobel Prize in chemistry after proving this, thus creating density generalized theory (DFT).
But DFT tools fail in a number of well-known situations. One is the prediction of how atoms share electrons; in one famous example, the DFT approach incorrectly predicted that chlorine atoms retain some of the electrons of sodium atoms even though chlorine and sodium atoms are infinitely far apart. Such an error occurred because the DFT equation is only an approximation of physical reality. And researchers from the DeepMind machine learning project say their neural network eliminates some of the electron error and makes more accurate predictions than traditional DFT methods.
DeepMind says they have solved two long-standing problems with traditional generalizations.
The first is out-of-domain error: In DFT calculations, generalized functions determine the charge density of a molecule by finding the electron configuration that minimizes its energy. Therefore, errors in the generalized function may lead to errors in the calculated electron density. Most existing density generalization approximations prefer to unrealistically spread the electron density over several atoms or molecules rather than correctly locating it around a single molecule or atom, as shown below.
Then there is spin symmetry breaking: When describing chemical bond breaking, existing general functions tend to unrealistically favor a fundamental symmetry breaking configuration called spin symmetry. Since symmetry plays a crucial role in our understanding of physics and chemistry, this artificial symmetry breaking reveals a major flaw in existing generalized functions.
Left: The conventional generalized function (B3LYP) predicts that the charge is applied to two adjacent molecules. Right: The learned generalized function (DM21) correctly locates the charge on one molecule.
DeepMind says that these long-standing challenges are related to the behavior of generalized functions when presenting systems with "fractional electronic features". The problem of off-domain and spin symmetry breaking was solved by using neural networks to represent generalized functions and tuning their training data sets to capture the fractional electron behavior of precise generalized functions. "Our generalized functions have also shown high accuracy in extensive large-scale benchmark tests, demonstrating that this data-driven approach can capture aspects of hitherto elusive exact generalized functions."
02Is DeepMind's method just memorizing answers?
To demonstrate its superiority, the authors tested DM21 on a set of stretched dimers (called BBB sets), for example, two hydrogen atoms far apart with a total of one electron.
DeepMind concluded that the DM21 generalization showed excellent performance on the BBB test set, far outperforming all the classical DFT generalizations tested and DM21m (which is the same as DM21, except that there is no fractional electron system in the training set).
But here's the problem, say the eight Russian researchers, although it looks like DM21 has understood the physics behind the fractional electron system, a closer look shows that all the dimers in the BBB set become very similar to the system in the training set. In fact, due to the localized nature of the electroweak interactions, the interactions between the atoms are only strong at short distances; outside of short distances, the two atoms essentially behave as if they were not interacting, as shown below.
The Russian researchers indicated that, in their opinion, the improvement in the performance of DM21 on the BBB test dataset relative to DM21m may be caused by a more prosaic reason: the unexpected overlap between the training and test datasets.
The relationship between the BBB test system and the fractionally charged atoms from the training set.
In some ways, neural networks are like humans: they prefer to get the right answer by finding mistakes and then the other way around," explains Michael Medvedev of the Zelinsky Institute of Organic Chemistry of the Russian Academy of Sciences. Thus, it is not so difficult to train a neural network to prove that it has learned the laws of physics, rather than to remember the correct answers. Testing a neural network on a system seen during training is similar to examining an elementary school student who saw his teacher solve a task five minutes earlier."
According to the researchers, "The BBB test set is not an appropriate test set because it does not test DM21's understanding of fractional electronic systems: DM21 can be easily acquired by memory. And a thorough analysis of the other four evidences of DM21's handling of such systems does not lead to a decisive conclusion: only its good accuracy on the SIE4x4 set may be reliable, but even the presence of a clear trend of error growth with distance suggests that DM21 is not completely free of problems with fractional electronic systems."
If the Russian researchers are correct, this would mean that DeepMind did not actually predict quantum mechanics by training the neural network.
In response, DeepMind responded [3] that DM21 does not achieve performance improvements by memorizing data, and "we disagree with their analysis and believe that the points raised are either incorrect or irrelevant to the main conclusions of the paper and the overall quality assessment of DM21."
No further rebuttal from the Russian team has been seen as of PhotonBox's press time.
Reference links：
[1]https://www.science.org/doi/10.1126/science.abj6511
[2]https://www.science.org/doi/10.1126/science.abq3385
[3]https://thenextweb.com/news/deepmind-feuds-russian-scientists-over-quantum-ai-research
