Digital Depth Refocusing and Aperture Adjustment with Light Field Data

Shafin Haque


Overview

Lightfield cameras capture a grid of images over a plane orthogonal to the optical axis, capturing a 4D representation of an image, including both the light intensty, and their angles. This additional information allows for various post capture affects such as depth estimation, refocusing, and persepctive shifts. The lightfield camera data is from the Stanford Light Field Archive


Digital Refocusing

Following Ng et al.'s approach on plenoptic cameras, digital refocusing can be done by properly shifting and averaging the sub-aperture images based on their offset from the center of the grid. Choosing how close or far the image should be focused is done by setting an alpha value to scale the shifting.

alpha = -1.1

alpha = -1.1

alpha = -0.5

alpha = 1.0

alpha = 1.9

Aperature Adjustment

Post capture aperature adjustment is also possible with the plenoptic camera by choosing how many images from the grid you use to average. Averaging over a larger set of images on the grid mimics a camera with a much larger aperature, while averaging less images closer to the center of the grid mimics a smaller aperature. This effect can change various parts of the image, including the depth of field. The images are in order of max distance from the center of the grid.

Summary

Plenoptic cameras allow for lightfield photography which capture not only the intensity of light at each point, but also the direction of the light rays. This 4D information allows for powerful digital post-capture processing affects through the shifting and averaging of the sub-aperture images. In this project, it is shown how to computationally refocus the image at various points, as well as mimic the effect of changing the aperture of the camera.