Oppo Joins Chip Design Race By Designing Its Own Neural Processing Unit

Oppo has entered the Chip design race by designing its own Neural Processing Unit (NPU) called the MariSilicon X. Unlike the Apple Neural Engine (ANE) and Google's Tensor, the MariSilicon X main function is on improving its phones' imaging system. Hot on the heels of Xiaomi's Surge C1 and Vivo's V1 own self designed Image Signal Processor (ISP), Oppo ups the ante by integrating its self-designed ISP into its new imaging NPU. We spoke exclusively to Jun Luo, Imaging Director of Oppo to find out what the MariSilicon X is all about.

Jun Luo, Imaging Director - Oppo

Why has Oppo decided to build its own NPU?

OPPO approaches innovation with the aim of achieving breakthroughs not only through a single piece of technology but rather, an entire integrated system, aligned with our mission – “Technology for mankind, kindness for the world.”

In the case of the imaging system, OPPO’s previous improvements to elements such as camera lenses, sensors, and algorithms have each had an impact. However, OPPO took one step further to fully integrate the different elements so that their influence could be further enhanced. As OPPO’s first self-designed NPU, MariSilicon X is a testament to OPPO’s commitment to pushing the boundaries of image technology by being an important step towards fully integrating our image processing pipeline to deliver truly professional performance for consumers.

How is the MariSilicon X different from other ISP or NPU?

MariSilicon X is a cutting-edge Imaging NPU built on 6nm process technology that integrates OPPO’s self-designed NPU, self-designed ISP, and high-speed multi-tier memory architecture to provide advanced computing and high-power efficiency to meet OPPO’s specific needs in image processing. All key elements of MariSilicon X were built from scratch entirely by OPPO without the use of third-party IP, including the NPU’s architecture, its circuit design, and even its physical appearance. The final result is a cutting-edge Imaging NPU, which raises the bar for NPUs with its exceptional speed and power efficiency. By directly working with original image data, MariSilicon X can obtain images with an 8dB signal-to-noise ratio and is able to preserve 30% more detail compared to the conventional image pipeline, resulting in a significant increase in image quality across the board. On top of this, the NPU also delivers real-time previews for night-time and HDR videos, giving users a true “what you see is what you get” experience that offers more potential for creativity.

How does the NPU help to create better images?

Processing images at the pixel level in real-time is a crucial requirement for high-performance algorithms, and in OPPO’s experience this can only be achieved when the computing power dedicated to image processing exceeds 10 TOPS (trillion operations per second). OPPO has designed MariSilicon X’s NPU to deliver a huge amount of processing speed — 18 TOPS to be precise. This allows MariSilicon X to offer an incredible power efficiency of 11.6 TOPS per watt. For instance, when running AI noise reduction, MariSilicon X performs noise reduction on 4K video at 40fps. That’s up to 20-times faster than what was previously possible on the OPPO Find X3 Pro, and with MariSilicon X, 50% less power is used in the process. Thanks to these super high processing speeds, MariSilicon X enables high-resolution 4K video to be captured even at night by performing complex brightness enhancements to each individual frame in real-time.

Can the MariSilicon X also improve the phone's video capabilities?

In order to process RAW data in real-time, MariSilicon X includes OPPO’s in-house ISP, the first processing unit that can work with image data once it is captured by the sensor to process data at a granular level. OPPO has based the ISP’s unique design on future trends we see in online video and user-generated video content. The ISP is able to conduct Ultra High HDR fusion on multiple HDR frames directly in the RAW domain, enabling 4K Ultra HDR video to be captured with an incredible 20bit, 120db (20 stops) dynamic range as maximum spec. Such video offers a 1 million to 1 contrast ratio between the brightest and darkest parts of the image, even rivalling what can be seen by the human eye.

Compared with the Find X3 Pro which offers a maximum dynamic range of 18bit, or 108db, that equates to a fourfold increase in usable dynamic range with MariSilicon X.

The OPPO Find X5 Pro will be the first to come equipped with the MariSilicon X, what can users expect?

As the first smartphone to be equipped with the MariSilicon X, the OPPO Find X5 Pro is set to revolutionise videography with 4K Ultra Night Video powered by the imaging NPU. It runs an advanced AI noise reduction (AINR) algorithm designed by the OPPO Research Institute, which means that it can detect and reduce noise in each frame, pixel by pixel while preserving finer detail, skin tone and colour accuracy. The result is a four times improvement in perceived night video resolution, less grain and superior colour reproduction, making 4K Ultra Night Video possible, with each frame as clean as a still photo.

Want to know how the Oppo Find X5 Pro fairs? Check out our review below.