China Develops Optical System for 100x Faster AI Inference
- tech360.tv
- 3 hours ago
- 3 min read
Researchers in China have developed an optical interconnect system designed to accelerate distributed artificial intelligence inference. This prototype reportedly achieved over a 100-fold increase in inference speed. The system required approximately one-ninth of the computational power typically used by commercial GPU-based systems for similar tasks.
The system, created by researchers at Peking University, links multiple computing chips through an on-chip all-optical network. This differs from traditional electrical connections. The design aims to minimise delays and enhance data flow between chips, addressing a growing impediment in scaling artificial intelligence workloads.
Central to this platform is a 400 gigabits per second silicon photonic transceiver. This component converts electrical signals into optical signals and then back again. It operates alongside a bespoke 16 by 16 optical switch chip, which routes data between computing nodes. So, this arrangement establishes a scalable communication network with an overall switching bandwidth reaching 6.4 terabits per second.
The researchers indicate that this design reorients the focus. Instead of merely adding more computing hardware, the emphasis is now on improving how chips communicate. This methodology enables multiple processors to cooperate more efficiently during artificial intelligence inference processes.
A notable characteristic of the optical switch involves its total signal loss, measured at less than 5 decibels, which includes coupling loss. According to the team, this permits high-speed, error-free data transmission without the need for external optical gain compensation. And the switch maintains error-free performance across several communication pathways. It also supports a spectral response exceeding 100 nanometres, rendering it suitable for future bandwidth expansion through wavelength-division multiplexing.
To demonstrate the architecture's capabilities, the researchers implemented a five-layer convolutional neural network. This network was used for an image denoising task. Each layer of the network was assigned to a separate computing unit, with the optical switch connecting these processors into a pipeline configuration.
The system avoided the practice of repeatedly storing intermediate data in memory before transmitting it to the subsequent processor. Instead, feature maps were conveyed directly through the optical network. This approach reduced delays associated with memory transfers. But it also ensured the computing units operated continuously without interruption.
When compared against a standard commercial GPU performing the same image-denoising operation, the optical system delivered markedly faster inference. The researchers reported a speed improvement exceeding a hundred times. This was achieved while consuming only approximately one-ninth of the computational resources.
The researchers propose that this work presents an alternative method to enhance artificial intelligence performance as model complexity continues to increase. They wrote that "specific objectives can be realised under limited computational resources when algorithms, processor micro-architectures and chip-level interconnections are co-designed." So, this co-design approach could offer efficiency gains.
The authors further stated that this fabric "can also alleviate unsustainable energy usage in data centres and optimise latency or consumption in edge-computing scenarios." This suggests potential benefits for both large-scale data operations and smaller, localised computing needs.
The team believes that advancements in co-packaged optics, along with silicon photonic transceivers and faster artificial intelligence chip interfaces, could help establish on-chip optical supernodes. And such supernodes might form a practical basis for future distributed computing systems. These systems would then provide the necessary bandwidth, low latency, and energy efficiency to support next-generation artificial intelligence workloads without sole reliance on larger, increasingly power-intensive processor clusters.
A new optical interconnect system increases AI inference speed over 100-fold.
The system uses approximately one-ninth of the compute power of commercial GPUs.
Researchers from Peking University developed the on-chip all-optical network.
The design focuses on improving communication between chips rather than solely adding hardware.
This approach aims to reduce energy consumption in data centres and optimise performance in edge computing.
Source: Peking University Research (China's optical chip network delivers 100x faster computing with one-ninth the compute.pdf) Source: interestingengineering
