No icon

Spatial and Angular Resolution Enhancement of Light Fields Using Convolutional Neural Networks

Spatial and Angular Resolution Enhancement of Light Fields Using Convolutional Neural Networks

Abstract:

Light field imaging extends the traditional photography by capturing both spatial and angular distribution of light, which enables new capabilities, including post-capture refocusing, post-capture aperture control, and depth estimation from a single shot. Micro-lens array (MLA) based light field cameras offer a cost-effective approach to capture light field. A major drawback of MLA based light field cameras is low spatial resolution, which is due to the fact that a single image sensor is shared to capture both spatial and angular information. In this paper, we present a learning based light field enhancement approach. Both spatial and angular resolution of captured light field is enhanced using convolutional neural networks. The proposed method is tested with real light field data captured with a Lytro light field camera, clearly demonstrating spatial and angular resolution improvement.

Existing System:

Light field imaging extends the traditional photography by capturing both spatial and angular distribution of light, which enables new capabilities, including post-capture refocusing, post-capture aperture control, and depth estimation from a single shot. Micro-lens array (MLA) based light field cameras offer a cost-effective approach to capture light field. A major drawback of MLA based light field cameras is low spatial resolution, which is due to the fact that a single image sensor is shared to capture both spatial and angular information. In this paper, we present a learning based light field enhancement approach. Both spatial and angular resolution of captured light field is enhanced using convolutional neural networks. The proposed method is tested with real light field data captured with a Lytro light field camera, clearly demonstrating spatial and angular resolution improvement.

 

 

Proposed System:

In this paper, we present a convolutional neural network based light field super-resolution method. The method has two sub-networks; one is trained to increase the angular resolution, that is, to synthesize novel viewpoints (sub-aperture images); and the other is trained to increase the spatial resolution of each sub-aperture image. We show that the proposed method provides significant increase in image quality, visually as well as quantitatively (in terms of peak signal-to-noise ratio and structural similarity index), and improves depth estimation accuracy.

Comment As:

Comment (0)