We present an algorithm for separating the shading and reflectance images of photographed natural scenes. The algorithm exploits the constraint that in natural scenes chromatic and luminance variations that are co-aligned mainly arise from changes in surface reflectance, whereas near-pure luminance variations mainly arise from shading and shadows. The novel aspect of the algorithm is the initial separation of the image into luminance and chromatic image planes that correspond to the luminance, red–green, and blue–yellow channels of the primate visual system. The red–green and blue–yellow image planes are analysed to provide a map of the changes in surface reflectance, which is then used to separate the reflectance from shading changes in both the luminance and chromatic image planes. The final reflectance image is obtained by reconstructing the chromatic and luminance-reflectance-change maps, while the shading image is obtained by subtracting the reconstructed luminance-reflectance image from the original luminance image. A number of image examples are included to illustrate the successes and limitations of the algorithm.
Get full access to this article
View all access options for this article.
References
1.
ArendL E, 1994“Surface colors, illumination, and surface geometry: Intrinsic-image models of human color perception”, in Lightness, Brightness and Transparency Ed. GilchristA L, (Hillsdale, NJ: Lawrence Erlbaum Associates) pp 159–213.
2.
BarrowH GTenenbaumJ, 1978“Recovering intrinsic scene characteristics from images”, in Computer Vision Systems Eds HansonA RRisemanE M, (Orlando, FL: Academic Press) pp 3–26.
3.
BergströmS S, 1994“Color constancy: Arguments for a vector model for the perception of illumination, color, and depth”, in Lightness, Brightness and Transparency Ed. GilchristA L, (Hillsdale, NJ: Lawrence Erlbaum Associates) pp 257–286.
4.
BlakesleeBMcCourtM, 1999“A multiscale spatial filtering account of the White effect, simultaneous brightness contrast and grating induction”Vision Research394361–4377.
5.
BlakesleeBMcCourtM, 2004“A unified theory of brightness contrast and assimilation incorporating oriented multiscale filtering and contrast normalization”Vision Research in press.
6.
CannyJ, 1986“A computational approach to edge detection”IEEE Transactions on Pattern Analysis and Machine Intelligence8679–689.
7.
CavanaghP, 1991“Vision at equiluminance” in Limits of Vision Eds KulikowskiJ JMurrayI JWalshV, volume 5 of Vision and Visual Dysfunction Ed. Cronly-DillonJ, (Boca Raton, FL: CRC Press) pp 234–250.
8.
CavanaghPLeclercY, 1989“Shape from shadows”Journal of Experimental Psychology: Human Perception and Performance153–27.
9.
ColeG RHineTMcIlhaggaW, 1993“Detection mechanisms in L-, M-, and S-cone contrast space”Journal of the Optical Society of America A1038–51.
10.
DakinS CBexP J, 2004“Natural image statistics explain brightness filling-in”Proceedings of the Royal Society of London, Series B, in press.
11.
DerringtonA MKrauskopfJLennieP, 1984“Chromatic mechanisms in lateral geniculate nucleus of macaque”Journal of Physiology357241–265.
12.
De ValoisR L, 1965“Analysis and coding of colour vision in the primate visual system”Cold Spring Harbor Symposia on Quantitative Biology30567–579.
13.
FineIMacLeodD I ABoyntonG M, 2003“Surface segmentation based on the luminance and colour statistics of natural scenes”Journal of the Optical Society of America A201283–1291.
14.
FinlaysonG DHordleyS DDrewM S, 2002a“Removing shadows from images”European Conference on Computer Vision, Copenhagen, ECCV′02 Proceedings part 4, pp 823–836.
15.
FinlaysonG DHordleyS DDrewM S, 2002b“Removing shadows from images using retinex” IS&T/SID Tenth Colour Imagining Conference pp 73–79.
16.
GeversTStockmanH M G, 2000“Classifying colour transitions into shadow-geometry, illumination highlight or material edges”IEEE International Conference on Image Processing (ICIP), Vancouver, Canada, 2000. Proceedings part I, pp 521–525.
17.
GilchristA L, 1979“The perception of surface blacks and whites”Scientific American240(3) 112–123.
18.
GilchristA LDelmanSJacobsenA, 1983“The classification and integration of edges as critical to the perception of reflectance and illumination”Perception & Psychophysics33425–436.
19.
GilchristAKossyfidisCBonatoFAgostiniTCataliottiJLiXSpeharBAnnanVEconomouE, 1999“An anchoring theory of lightness perception”Psychological Review106795–834.
20.
GrossbergSTodorovicD, 1988“Neural dynamics of 1-D and 2-D brightness perception: A unified model of classical and recent phenomena”Perception & Psychophysics43241–277.
21.
JainRKasturiRSchunckB G, 1995Machine VisionComputer Science Series (McGraw-Hill International Editions).
22.
KingdomF A A, 2003“Colour brings relief to human vision”Nature Neuroscience6641–644.
23.
KingdomF A ABeauceCHunterL, 2004“Colour vision brings clarity to shadows”Perception33 in press.
KingdomFMouldenB, 1988“Border effects on brightness: A review of findings, models and issues”Spatial Vision3225–262.
26.
KnillD CKerstenDMamassianP, 1996“Implications of a Bayesian formulation of visual information for processing for psychophysics”, in Perception as Bayesian Inference Eds KnillD CRichardsW, (Cambridge: Cambridge University Press) pp 239–286.
27.
LandE HMcCannJ J, 1971“Lightness and retinex theory”Journal of the Optical Society of America611–11.
28.
LehkyS RSejnowskiT J, 1988“Network model of shape-from-shading: Neural function arises from both receptive and projective fields”Nature333452–454.
29.
MacLeodD I ABoyntonR M, 1979“Chromaticity diagram showing cone excitation stimuli by stimuli of equal luminance”Journal of the Optical Society of America691183–1186.
30.
MullenK TKingdomF A A, 1991“Colour contrast in form perception”, in The Perception of Colour Ed. GourasP, volume 6 of Vision and Visual Dysfunction Ed. Cronly-DillonJ, (London: Macmillan) pp 198–217.
31.
NorlanderCKoenderinkJ J, 1983“Spatial and temporal discrimination ellipsoids in color space”Journal of the Optical Society of America731533–1543.
32.
PárragaC ABrelstafGTrościankoTMooreheadI R, 1998“Color and luminance information in natural scenes”Journal of the Optical Society of America A15563–569.
33.
PárragaC ATrościankoTTolhurstD J, 2002“Spatiochromatic properties of natural images and human vision”Current Biology12483–487.
34.
PessoaLThompsonENoëA, 1998“Finding out about filling in: A guide to perceptual completion for visual science and the philosophy of perception”Behavioral and Brain Sciences21723–802.
35.
RamachandranV S, 1988“Perception of shape from shading”Nature331163–166.
36.
RubinJ MRichardsW A, 1982“Color vision and image intensities: When are changes material?”Biological Cybernetics45215–226.
37.
SankeralliM JMullenK T, 1997“Postreceptoral chromatic detection mechanisms revealed by noise masking in three-dimensional cone contrast space”Journal of the Optical Society of America A14906–915.
38.
SmithV CPokornyJ, 1975“Spectral sensitivity of the foveal cone photopigments between 400 and 700 nm”Vision Research15161–171.
39.
StromeyerC FIIIColeG RKronauerR E, 1985“Second-site adaptation in the red–green chromatic pathways”Vision Research25219–237.
40.
SunJPeronaR, 1997“Shading and stereo in early perception of shape and reflectance”Perception26519–529.
41.
TappenM FFreemanW TAdelsonE H, 2003“Recovering intrinsic images from a single image”, in Advances in Neural Information Processing Systems15 [Neural Information Processing Systems, NIPS 2002, Vancouver, BC, Canada] Eds BeckerSThrunSObermayerK, (Cambridge, MA: MIT Press) pp 1343–1350 [also available online at http://books.nips.cc/papers/files/nips15/VS13.pdf].
42.
WandellB A, 1995, chapter 9 in Foundations of Vision (Sunderland, MA: Sinauer).
43.
WeissY, 2001“Deriving intrinsic images from image sequences”Proceedings of the 8 th ICCV Vancouver, 9–12 July 15, volume 2, pp 68–76, http://www.cs.huji.ac.il/∼yweiss/.
