An object’s appearance depends on both the observer’s viewpoint and the illumination. The question arises: What is the set of images of an object under all lighting conditions and viewpoints? In this talk, we focus on enlightenment, including multiple, extended light sources and shadowing. It will be shown that the set of n-pixel images of an object of any shape and with a general reflectance function, seen under all possible illumination conditions forms a convex cone in \(R^n\). If the object is convex with a Lambertian reflectance function and is illuminated by an arbitrary number of point light sources at infinity, this illumination cone is polyhedral, and its dimension equals the number of distinct surface normals on the object. It can be constructed from three or more images, and recognition from a single image can be cast as determining set membership.
This leads us to another question: What objects are indistinguishable from only their images? It will be shown that if two objects differ by a “generalized bas-relief transformation,” then their illumination cones are identical. For each image of a surface produced by a light source, there exists an identical image of the bas-relief produced by a transformed light source. This equality holds for both shaded and shadowed regions. Since antiquity, artists have been aware of this ambiguity when creating flattened bas-relief sculptures which give an exaggerated perception of depth. For a moving observer, the set of all possible motion fields for a bas-relief is identical to the set produced by a full relief. Thus, neither small observer motions nor changes of illumination can resolve the bas-relief ambiguity.
These results are consistent with recent psychophysical studies and have lead to new methods for shape reconstruction from shadows. They have been used for face recognition in a system which is trained with a small number of well-illuminated images, yet which can perform recognition under illumination extremes where faces are heavily shadowed. When compared to competitive techniques, our method had an order of magnitude fewer errors.