![\"microchip](\"https:\/\/ieeecs-media.computer.org\/wp-media\/2022\/07\/06214911\/Pixel-Works.jpg\"){kind=tracking}
Motion estimation (ME) and motion compensation (MC) have been extensively used for video frame interpolation systems since the turn of the century. However, typical solutions are power-hungry and introduce interpolation errors. Pixelworks, a video alchemist skunkworks in San Jose, California, has been quietly developing a new back-end mobile processor that works for both movies and now video games. The result is an incredibly brilliant image, at super high resolutions and high frame rates, but with minimal latency that can actually improve battery life as well\u2014it sounds almost as if they have broken the law of physics. But no, says their CEO, we just know those laws very well and how to work within them.<\/p>\n
By using an expert-trained video motion estimation and compensation algorithm in silicon for video frame interpolation, the company has pushed the bounds limiting performance and quality. Through a novel adaptive compensation scheme trained by the company\u2019s picture-quality experts (that the company doesn\u2019t want to talk too much about), Pixelworks has integrated motion processing\u2014things they\u2019ve done for the film and TV industry in software\u2014into a mobile gaming solution that is now possible in a smartphone.<\/p>\n This secret sauce, the expert training by their best engineers and picture-quality people, is differentiable such that the flow and estimation can be optimized jointly in real time, and very fast, with minimal latency. In addition to the smart MEMC, the design can be seamlessly adapted to several other video enhancement tasks such as super-resolution, denoising, and deblocking. Quantitative and qualitative evaluations and demonstrations have shown the X7 post processor outperforms state-of-the-art video frame interpolation and enhancement algorithms on a wide range of datasets. Compared to existing methods, Pixelworks\u2019 magic is computationally efficient and able to generate more visually appealing results.<\/p>\n One segment that appreciates this is gaming, and as most people know, gaming has become a big business in units and dollars worldwide. It is especially big in China, where it is estimated there are 670 million gamers\u201420 percent of the world\u2019s estimated 3.3 billion gamers. Most of those gamers play on a smartphone. Chinese gamers have adopted Genshin Impact from miHoYo as a benchmark. One of the elements in the game is weather. Weather is an exploration mechanic. While exploring the world, the weather can shift. Depending on the new weather conditions, special properties will be applied to the environment, to the characters, and to opponents.<\/p>\n \u00a0<\/p>\n \u00a0<\/p>\n Want More Tech News? Subscribe to ComputingEdge<\/i> Newsletter Today!<\/strong><\/p>\n \u00a0<\/p>\n \u00a0<\/p>\n Games like Genshin Impact and PC games like Cyberpunk 2077 that have been ported from the PC to smartphones challenge the phone\u2019s performance vs. battery and heat operating parameters. Big games demand everything the phone\u2019s processor has, and in the process, runs down the battery while heating up the phone.<\/p>\n Pixelworks saw that as an opportunity to apply its magic and introduced the X7 chip architecture, post processors that exploited some of the techniques the company used in the film and TV industry. The X7 chips created a new distributed computing\/rendering chip market segment in mobile\u2014a bit like what Nvidia and AMD have done in the PC and console segments. Chinese phone manufacturers have referred to X7 as stand-alone display solution, a render accelerator, or a dual-chip gaming architecture.<\/p>\n MEMC and resolution scaling (RS) are very demanding on any platform and especially in a mobile device with limited cooling and power supply. The phone must deliver a high frame rate, with high color accuracy, high pixel resolution, no artifact or aliasing, and low latency\u2014and accomplish all that while using very little power.<\/p>\n Pixelworks\u2019 post processor has proven to be just what the doctor ordered: a unique technology for motion compensation and resolution scaling that off-loads the phone\u2019s SoC. For gaming specifically, by taking care of the motion vectors and working with the game engine developers, the company\u2019s X7 processor uses metadata, which guarantees the best gaming experiences.<\/p>\n As a result, three out of four of the largest Chinese phone OEMs have been shipping their top phones with Pixelworks\u2019 chips. The leading game engine providers (e.g., Unity) have been partnering with Pixelworks, the big game studios in China (ByteDance, Tencent, miHoYo, etc.) have engaged with Pixelworks, and mobile SoC processor makers such as MTK and Qualcomm are working with Pixelworks.<\/p>\n The X7 is a tiny chip and draws so little power that the smartphone hardly knows it\u2019s running. Some smartphone suppliers in China have already added it to their phones for the game-crazy customers they have, and if one can satisfy that price-performance market segment, then they must have something special.<\/p>\n This is a classic story of a smallish (~$70 million in revenue) company quietly working in close collaboration with some big-name and colossally demanding customers and perfecting a solution that is scalable, efficient, and economical.<\/p>\n Pixelworks didn\u2019t just stumble into this market. Founded in 1997, the company developed video processors for professional projectors. The video engineers and scientists at the company had a collective range of experience unmatched in companies many times their size. But, they were technogeeks, not salesmen. You almost had to know someone who knew someone to find out about them. Their customers weren\u2019t eager to reveal much about them, either\u2014Pixelworks was their secret sauce and how they differentiated themselves in a cutthroat take-no-prisoners business.<\/p>\n The mythology of the company is the CEO or CTO (the story varies depending on who is telling it) was chatting at a conference with a CTO or president of a phone company who said, \u201cIt\u2019s my dream that someday we will have smartphones that you can watch a movie on, and it will be as good as being in the theater.\u201d The Pixelworks guy went home thinking about that, and the next day he told his team, \u201cI\u2019ve got an idea.\u201d<\/p>\n A few years later, when visiting the lab (if you could get in), one could see a big 4K screen showing video and games as if it was driven by an enthusiast-level PC, but it wasn\u2019t\u2014it was driven by a two-year-old smartphone, and not even a top-of-the-line smartphone.<\/p>\n The term skunkworks isn\u2019t used lightly here. Pixelworks has managed to beat out the top SoC builders in Silicon Valley and San Diego in video processing. In one sense, that\u2019s not too surprising\u2014video is all they do. Today\u2019s smartphones are a marvel of technology, but they can\u2019t be the best at everything they do. Product cycles and R&D budgets mitigate how much they can invest in time and dollars. Pixelworks is a specialty company\u2014a pixel polisher. They can also polish pixels coming from the sensor but acknowledge that the big SoC builders had done a really good job with their integrated ISPs\u2014you climb one mountain at a time, the CEO told me.<\/p>\n The tantalizing end of this story is not really an end but the future. Pixelworks has now proven their bespoke approach to video processing\u2014pixel polishing works; in fact, it works almost better than they thought it would. And it\u2019s scalable. It will ride Moore\u2019s law and just get better with each generation. Smartphones have (or will have) never looked so good.<\/p>\n<\/p><\/div>\n![\"HDR](\"https:\/\/ieeecs-media.computer.org\/wp-media\/2022\/07\/06215322\/Pixel-Magic-1-300x300.png\"){kind=tracking}
\n
\n![\"application](\"https:\/\/ieeecs-media.computer.org\/wp-media\/2022\/07\/06215440\/Pixel-Magic-2-300x120.png\"){kind=tracking}