In future, you don’t have to compose and focus before taking a picture. Just click the shutter on your camera and all 3D information of the scene will be recorded. You can then choose your subject, depth of field and perspective afterwards. This is the beauty of light field photography. But a light field camera needs an array of precise tiny lenses that is extremely costly to make. Scientists have tried finding ways to make the technology more accessible commercially. In fact, Dr Lihua Li, Department of Industrial and Systems Engineering, and her research team achieved several technical breakthroughs in the past few years to bring the cost down. This year the team took it one step further and developed a light field sensor, to provide camera and device manufacturers with a low-cost micro-lens array that can replace the glass cover on CCD or CMOS, making light field video possible for distance measurement, object detection, and quality control in production lines.
Light field imaging
To map a 3D space, a light field camera takes tens of thousands of images of the same scene from slightly different angles at the same time via a micro-lens array. In the past, micro-lens arrays were made with complicated processes such as photolithography or ion etching that call for expensive equipment and stringently controlled environment. Therefore, in 2016, Dr Li’s team developed the Compound Eye, a micro-lens array that is injection-moulded with optical plastic, to reduce the cost. In 2018, they developed a micro-embossing technology that locally heats up optical glass surface and moulds it into an ultra-high-precision micro-lens array. “The light field sensor we developed this year is a natural continuation of those two projects, combining a moulded Compound Eye with a CCD or CMOS sensor. With our inventions in the past few years, we have substantially brought the cost down,” explained Dr Li.
B2B solution for camera and device manufacturers
The sensor is intended to be a B2B solution for camera or portable device manufacturers as the micro-lens array can replace the cover glass for sensors and can be tailor-made according to the sensor’s specification. A powerful software then analyses the images produced on the sensor to generate full 3D information of the scene. The team even developed a deep learning algorithm to detect a target object within the range of 10 cm and 50 m from the lens, and measure its distance therefrom. To top it off, this sensor, depending on manufacturer’s specification, can be used to capture light field video, shooting as much as 25 frames per second. In other words, besides mapping the 3D space, it also incorporates the time element to make it 4D, especially useful for scenes that keep changing.
Applications
The sensor is suitable for all uses that require depth and distance measurements. For instance, a 3D scanner drone can capture light field video as it flies around a place. The topography and dimensions of all structure within the drone’s range can be measured instantly. With the sensor in surveillance camera, the police can reveal the height and body shape of any suspect. When connected to a microscope, this sensor helps semiconductor chip manufacturers conduct quality control easily. Light field video of chips on a conveyor belt can be used to check the thickness of the etched or deposited circuits, or whether the semiconductor is stacked according to the standard. For jewellery manufacturers, the sensor is a convenience tool to check the evenness and thickness of metal prongs in gemstone settings. For biomedical uses, the sensor can be fitted to an endoscope or a microscope for monitoring and measuring live body tissues.
In November 2019, the object detection and distance measurement sensor based on light field imaging won a gold medal at the 71st International Trade Fair for Ideas, Inventions & New Products (iENA) in Nuremberg, Germany.
