Sony has been hard at work on several projects, and the popular company has revealed fascinating details about one of them: a new blend between an image sensor and an AI logic chip which form an integrated system. The potential applications for such technology are quite impressive as the ties between imaging and coding continue to grow.
The concept is not as complicated as it seems. It requires a classic CMOS sensor that can be found in most smartphones or cameras and place it on a logic chip that can run a machine learning model that extracts relevant information from captured pixels.
Advanced processing
A major boon I represented by the fact that a photo can be processed heavily before it is sent to a different location, like the storage space of the device or the cloud. Image sensors are often accompanied by processors that convert images into specific file formats, but they are limited to a small number of tasks that can be performed fast.
Sony’s chip would be able to perform tasks that are more complicated. For example, if one takes a photo of a central object with other objects in the background, the chip could scan the image and identify specific objects, highlighting them on an individual basis.
It could also perform some editing tasks, with one example being the ability to crop unknown objects from the picture or to keep only objects that have been highlighted by the user in advance.
Great benefits
Such a system could discard a significant amount of unnecessary data before it reaches the main processor or the storage space. This would reduce the amount of power that is consumed in the long run while also improving overall security.
Cameras located in public places could blur some parts of the image in advance, while smart home devices could acknowledge individuals without the need to save or send image data. Previous research has shown that chips with integrated AI processes work fast and are also cost-effective.
