Photonic computers are on their way to commercialization
For decades, Moore's Law has been dominant in the semiconductor industry, with the number of transistors in each microchip doubling about every two years. But in terms of the energy consumption of these chips, Moore's Law has begun to break down. Previously, the energy consumption per logic operation declined rapidly with each new generation of chips. Now, the trend is flat, or only slightly down, resulting in much lower performance gains.
This energy issue has become even more pressing due to the rapid growth of artificial intelligence, which is now used in Internet searches, language translation, image recognition and self-driving cars. Technology companies such as Google have devised ways to make digital AI computing more efficient.
But some experts believe the need for processing will become so urgent that an entirely new technology will be needed: photonic computing.
The U.S. company Ayar Labs is developing and marketing the TeraPHY optical I/O "chip" wafer.
Challenging Transistors
Rather than encoding and manipulating data by setting currents to zeros and ones, photonic computers rely on the physical, analog properties of light. The data is typically represented as a change in the amplitude or phase of the laser beam, and is processed as the beam passes through a series of properly activated optical elements. This is all done at the speed of light, consuming no more energy than is required to power the laser source: both processing time and energy requirements are independent of the size of the input.
Until recently, work on photonic computers was mainly limited to discrete laboratory experiments. But integrated photonics has changed this, making it easier to scale up this technology and opening the door to commercial products. The result has been the creation of about a dozen startups that raise the prospect that, after decades of frustration, optical technology may finally challenge the mighty transistor.
Between them, these companies have raised hundreds of millions of dollars, hired some of the brightest minds in photonics and computing, and forged partnerships with leading multinational companies. While some of the companies may not ultimately survive, Harish Bhaskaran, founder of Salience Labs, an Oxford, England, spin-off, says each is trying to address a slightly different part of a potentially huge market. "It's a huge space and there's enough room for many companies," he says.
Salience Labs, a spin-off company from Harish Bhaskaran's lab at Oxford University in the United Kingdom, is working to commercialize hybrid electronic-photonic chips for artificial intelligence applications.
Securing data with light
When scientists began working on optical computing in the 1960s, their goal was to build devices dedicated to fairly specific tasks. In particular, they wanted to take advantage of the fact that the image created by a coherent beam passing through a lens is a Fourier transform of the input field. The optical correlator uses two spatial light modulators and two lenses to compare the Fourier transform of the input signal with a predetermined optical filter: this in principle enables target identification and signal processing, for example for synthetic aperture radar.
Many companies are trying to commercialize this technology. Some companies have gone out of business, while others are growing.
In 2001, engineer Nick New founded Cambridge Correlators in the UK to sell optical correlators for pattern recognition. Since then, however, he has founded a new company called Optalysys, which makes products using silicon photonics and electronics. These programmable chips convert digital information into the optical domain and back again for a very secure but demanding type of data encryption, such as full homomorphic encryption (FHE).
FHE allows sensitive data to remain encrypted while being processed, so that even the processing hardware cannot access them. It involves adding random noise to the encryption process, but relies on multiplying by a high-order polynomial to control the noise. These complex calculations require about a million times more operations than unencrypted processing, which makes them untenable on standard digital computers.
However, these calculations can be greatly simplified if the data is first Fourier transformed and then re-transformed by a second transformation.
Optalysys performs the Fourier transform on the input data stream by encoding the data in a tiny laser beam through a diffraction grating etched into a piece of silicon. Joseph Wilson, the company's director of applications, says this arrangement is far better suited to converting electrical signals into optical signals and vice versa than bulk optics techniques. "Optics gives you two paths, either you go very big or very fast. We have now chosen the latter."
Founded in 2013 and based in Leeds, the company has raised about £10 million (about $11.5 million) to date and is currently testing its first chips in the lab, said company director Nick New. After integrating the necessary electronics over the next 18 months, he predicted the devices should be about 500 times faster than the best pure electronic chips. He added that if all goes well, the product should appear on the market in 2025.
Enable chip, described by its manufacturer Optalysys as "a complete solution for FHE".
A more versatile photonic processor: neural networks
While Optalysys has brought a modern twist to the accepted approach to natural computing using light, other companies have tried to make more versatile photonic processors. In the 1970s and 1980s, scientists began working on the optical equivalent of transistors. But it soon became apparent how demanding this would be, given the lack of photon interactions and therefore the inability to make (nonlinear) switches. As a result, in recent years, many researchers who founded photonic computing companies have turned to a specific but in-demand calculation, and one that is difficult to perform with digital electronics: matrix multiplication, which is the basis for artificial neural networks.
Neural networks are able to recognize certain patterns in data thanks to the weighted connections between their many artificial neurons, which are usually arranged in layers. The output of any neuron is determined by the sum of the weighted inputs of the neurons of the previous layer and a nonlinear activation function: this arrangement is repeated throughout the network and allows the last layer to indicate the presence or absence of a specific pattern at the input.
The sum of the weighted inputs is represented mathematically as a multiplication between a vector (input values) and a matrix (weights). For a digital processor, this corresponds to a long sequence of multiplications and additions, involving the transfer of data back and forth along the line between the memory and the processor in electronic form. This takes time and generates heat: specialized AI chips, such as Google's Tensor Processing Unit TPU, save both time and energy by reducing many of the operations that occur in a general-purpose CPU and avoiding repeated data fetches from memory. Ultimately, however, all digital electronic chips are limited by their von Neumann architecture.
Abu Sebastian, an electronics engineer at IBM in Zurich, Switzerland, highlights this limitation by comparing such chips to a particularly effective neural network: the one in the human brain. He noted that the brain consumes on average only about 20 watts and an estimated 1 femtojoule (10-15 joules) per synaptic operation. In contrast, their continuous data movement means that a digital chip based on the von Neumann architecture could consume hundreds of femtojoules per operation, "even though the processor consumes almost no energy."
Integrated optics: Mach-Zendel interferometer
Optical processors can cut the energy required for matrix multiplication because the non-interaction of photons makes them well suited to this linear algebra. Until recently, however, they were too bulky to compete with electronics.
Now, as Bhaskaran points out, they can also be fabricated as chips, using a CMOS process similar to that used to make conventional integrated circuits - a fabrication offered by a number of foundries around the world. The whole manufacturing scenario has changed," he says. It means that if there's a market for it, you can make it."
One of the first groups to study integrated photonics for optical computing was the group of Dirk Englund and Marin Soljačić at the Massachusetts Institute of Technology (MIT) in the United States. The group explored a number of different schemes, including one proposed by then-postdoc Ryan Hamerly, which used an optical homodyne detector to multiply the intensities of laser pulses and a capacitor to accumulate the electrically encoded results of successive multiplications. This allows for low-energy matrix multiplication, but to be practical it may require a large number of lenses as well as chip-based components.
The proposal that has attracted the most attention involves the so-called Mach-Zehnder interferometers (MZI). These devices consist of two 50:50 beam splitters and a pair of phase shifters incorporated into a waveguide that can rotate input vectors encoded as laser pulses, and can also scale these vectors if one half of their two input and output channels are blocked. Since any matrix can be cast as a combination of two rotations and a rescaling, a suitable MZI network can perform matrix multiplication of the incoming optical pulses.
The scheme was used to implement a two-layer neural network that can recognize different vowels, as reported in a 2017 Nature Photonics paper. This work later led to the spin-off of two separate companies, both with offices in Boston, called Lightmatter and Lightelligence, but while both companies went on to develop chips that utilized MZI in some way, neither stuck to the original design.
"The World's Leading Photonic Computing Company": Lightelligence
Lightelligence was founded in 2017, four years after releasing a chip called PACE that contains about 10,000 photonic devices and a microelectronic chip that provides control and memory. Co-founder and CTO Huaiyu Meng explained that the company had to break up the original single MZI network because it was too difficult to maintain a very delicate phase state in a coherent network. If any of the interferometers failed, he said, it would be "very difficult to isolate the impact.
Yichen Shen (left), co-founder and CEO of Lightelligence, and Nicholas Harris (right), co-founder and CEO of Lightmatter.
Huaiyu Meng said that despite this limitation, PACE can solve a "maximum cut problem" that scales exponentially and is more than 100 times faster than state-of-the-art GPUs: the company plans to launch a more powerful chip for image recognition and object detection around the end of 2022. In fact, after securing about $200 million in investment and hiring about 200 employees, the company's founder and CEO, Yichen Shen, recently declared Lightelligence to be "the world's leading photonic computing company.
Nicholas Harris, founder and CEO of Lightmatter, begs to differ. He is skeptical of Lightelligence's claim to have solved the maximum cut problem, arguing that to be scalable is "a bit like claiming we can do cold fusion." He wouldn't reveal any technical details of his own company's new general-purpose AI chip, dubbed Envise-beta, except to say that several MZIs in the device don't actually do any processing. But he did say that the chip contains millions of optical components.
Lightmatter was founded in 2017 and has hired at least 100 employees to date: including Richard Ho, who led Google's development of the tensor processing unit, according to Harris, who added that the company has also raised more than $115 million in funding from top venture capital firms, with "more funding on the way " He claims that the company's technology is the only photonic technology that is beginning to match the capabilities of the leading digital chips, and "I don't know anyone else who comes close to doing that."
Lightelligence's PACE, a photonic chip launched in 2021.
More than just memory
While the initial work underpinning the two MIT startups relies on the phase of light, companies elsewhere are basing their photonic computing technology on other fluctuating properties. For example, Salience, spun off from Oxford University, was founded to build neural networks that exploit changes in the intensity of light. To do this, it uses phase change materials (PCM) - in this case, phase change refers not to light but to the property of the material, which has a higher transmittance when it is in its amorphous state than when it is in its crystalline state.
Drawing inspiration from how the brain works, Bhaskaran and colleagues at Oxford University, along with Wolfram Pernice and colleagues at the University of Münster in Germany, initially combined PCM with wavelength division multiplexing to create a set of excitation neurons that they reported in 2019 were trained to recognize letters in the alphabet. But that's not the scheme Bhaskaran will develop at Salience, which he built the following year after hearing about primitive electronics from IBM's Sebastian in 2017 to commercialize a photonic version of stored computing.
Sebastian explains that the key idea of his research is to avoid data movement by performing computation in memory. This memory, which is based on charge or resistance in electronics, can represent weights in a neural network so that the memory remains fixed as data flows through it, enabling matrix multiplication. This has the advantage of saving processing time and effort, with reduced accuracy compared to numerical computation.
Bhaskaran, Sebastian, Pernice and colleagues first demonstrated photonic memory computing using a single integrated optical device in 2018. Two years later, they used a directional coupler to transfer the multiplexed output of an on-chip optical frequency comb to a single PCM block (the weights of a neural network) while performing 64 matrix multiplications. Given the potential speed of photon modulation, the researchers say this "tensor kernel" could perform 2 trillion operations per second.
According to Bhaskaran, the work could lead to the first prototype tensor core by the end of 2023, followed by a commercial device a few years later. But he said it remains to be seen whether their small device can maintain "extraordinary performance" on a larger scale. Given the different scales of the two technologies, with light wavelengths of hundreds of nanometers and transistors in microchips often only a few tens of nanometers long, they still need to demonstrate how their processors interface with the electronic devices that set their weights. "This is a huge research problem that we are solving."
Photonics + Electronics: The Connection Problem
Almost all developers of photonic computers face the thorny problem of connecting photonics to electronics. Cognifiber, an Israeli company, is one company that has taken a novel approach to this problem. The solution completely abandons silicon photonics in favor of glass in the form of multicore optical fibers. The idea is to use interference between cores to simulate the interactions between neurons and synapses in a multilayer neural network.
Coupling between cores in multicore fibers can lead to crosstalk, which usually needs to be avoided in telecommunication applications. However, Zeev Zalevsky of Bar-Ilan University in Israel and Eyal Cohen, CEO of Cognifiber, use this coupling to their advantage by transmitting signals at 1550 nm between cores representing neurons, synapses or connections between the two.
The logical (left) and physical (right) layout of the Cognifiber scheme.
Their scheme envisions up to hundreds of thousands of cores arranged in a specific cross-sectional geometry. The cores representing the input neurons form a ring closest to the periphery of the fiber, while the inner neurons and their synapses gradually approach the center of the fiber. Some of the cores are erbium-doped, carrying 980 nm pump light that allows controlling the weighted summation of the input and the activation function of the neuron (through the nonlinear region of the erbium gain function).
As Zalevsky explains, the scheme does not rely on coherent light and is therefore not affected by small fluctuations in parameters such as polarization or temperature. Another huge benefit, he says, is the amount of time and money that has been spent on developing fiber-optic communications, resulting in a smooth and fast interface between photonic and electronic data. But he also noted that their system faces a major challenge: how to connect a large number of cores to their respective fibers to bring in light from the outside.
Cognifier was founded in 2018 and is still in the relatively early stages. The researchers showed in principle how to connect dozens of neurons and demonstrated how a single neuron works in its entirety.Zalevsky estimates that it would take two to three years to build a working prototype with hundreds of processing neurons. But he is optimistic that the substantial engineering and financial effort will lead to commercial success for the company.
Abandoning optical, electrical interconnections
Given the difficulty of connecting photonic processors to the world of electronics, some companies have decided it's best not to try. That's the approach taken by California-based Ayar Labs, which was founded in 2015 to commercialize optical interconnects. Mark Wade, the company's co-founder and chief technology officer, said he and his colleagues concluded that the limited speed and high energy consumption of data transfers between processors would likely be a bigger bottleneck than the performance of the processors themselves.
With about $195 million in funding and about 100 employees, Wade said Ayar Labs is now preparing two optical I/O products for mass production: an optical "small chip" that converts electrical signals into optical pulses at a rate of 1 terabyte per second, and a fiber-optic-connected laser to deliver the light. . The idea is to connect components on a given board, between boards in the same rack, or between racks over distances of tens of centimeters or longer, rather than enhancing communication within a single chip.
Full production by the end of 2026, just sayin'?
