Visual Mechanisms for Perceiving Materials
Unmasking Neural Computations for Material Perception
We effortlessly perceive such surface properties as glossiness, lightness, and translucency. Yet we still don’t know how the brain estimates surface properties. The image of a natural surface results from complex optical processes, and the reconstruction of surface properties from a single image may also involve highly complex neural computations. However, we found that the brain utilizes simple image statistics to estimate several surface properties. For example, humans can estimate the glossiness and lightness of surfaces using the skewness of image luminance histograms. Material perception seems to be based on low-level visual mechanisms that require shallower neural computations than previously thought.
Future Applications
Our findings will provide a novel framework for visual displays, image compression, and computer graphics with rich appearances. The fast computation of simple image statistics will also enable real-time 3D CG rendering and high-speed image retrieval.
World is Full of Materials
Every surface in our natural environments has its own material property: foods, clothing, skin, antiques. In daily life, we effortlessly see such things. We may also enjoy, evaluate, and create surface materials in arts and crafts. How can we perceive material properties from images so immediately?
Visual Cues of Materials in the Image
In physics, natural surfaces involve highly complex interactions among the surface's optical properties, its 3D structure, and its illumination. This complexity may lead to a notion that material perception should be based on deep visual processing in the brain. However, we found that the visual system utilizes simple features in the 2D image to estimate material properties. This suggests that material perception is based on shallow, low-level visual processing.
Glossiness, Lightness, and Histogram
For example, the perceived gloss of natural surfaces is well predicted by skewness in the luminance (intensity) histogram of the image. Images of glossy materials tend to have positive skewness, and the visual system utilizes it to estimate the gloss. We can skew the image histogram of a less glossy material to make it glossier. Skewness is a simple statistic that can be computed by neural networks in the retina or in the low-level visual cortex.
Changing Materials by Image Processing
We are exploring key image features for translucency, metallic luster, and human skin. We found several techniques to alter the apparent materials by an extremely simple image manipulation. For example, reversing the contrast of the shading pattern, but not of the highlight, alters an opaque surface into translucent one.