No icon

Moir´e Photo Restoration Using Multiresolution Convolutional Neural Networks

Moir´e Photo Restoration Using Multiresolution Convolutional Neural Networks

Abstract:

Digital cameras and mobile phones enable us to conveniently record precious moments. While digital image quality is constantly being improved, taking high-quality photos of digital screens still remains challenging because the  photos are often contaminated with moir´e patterns, a result of the interference between the pixel grids of the camera sensor and the device screen. Moir´e patterns can severely damage the visual quality of photos. However, few studies have aimed to solve this problem. In this paper, we introduce a novel multiresolution fully convolutional network for automatically removing moir´e patterns from photos. Since a moir´e pattern spans over a wide range of frequencies, our proposed network performs a nonlinear multiresolution analysis of the input image before computing how to cancel moir´e artefacts within every frequency band. We also create a large-scale benchmark dataset with 100; 000+ image pairs for investigating and evaluating moir´e pattern removal algorithms. Our network achieves state-of-the-art performance on this dataset in comparison to existing learning architectures for image restoration problems.

Existing System:

The photo of a screen is the result of the interference between the pixel grids of the camera sensor and the device screen. It can appear as stripes, ripples, or curves of intensity and colour diversifications superimposed onto the photo. The moir´e pattern can vary dramatically due to a slight change in shooting distance or camera orientation.

This moir´e artefact severely damages the visual quality of the photo. There is a large demand for post-processing techniques capable of removing such artefacts. In this paper, we call images of digital screens taken with digital devices moir´e photos. It is particularly challenging to remove moir´e patterns in photos, which are mixed with original image signals across a wide range in both spatial and frequency domains. A moir´e pattern typically covers an entire image.

 

Proposed System:

We introduce a novel multiresolution fully convolutional neural network for automatically removing moir´e patterns from photos. Since a moir´e pattern spans over a wide range of frequencies, to make the problem more tractable, our network first converts an input image into multiple feature maps at various different resolutions, which include different levels of details. Each feature map is then fed into a stack of cascaded convolutional layers that maintain the same input and output resolutions. These layers are responsible for the core task of canceling the moir´e effect associated with a specific frequency band. The computed components at different resolutions are finally  psampled to the input resolution and fused together as the final output image.

We present a novel and highly effective learning architecture for restoring images contaminated with moir´e patterns.

We also create the first large-scale benchmark dataset for moir´e pattern removal. This dataset contains 100; 000+ image pairs, and will be publicly released for research and evaluation.

Comment As:

Comment (0)