Abstract
Recently, a method for removing shadows from colour images was developed (Finlayson et al. in IEEE Trans. Pattern Anal. Mach. Intell. 28:59–68, 2006) that relies upon finding a special direction in a 2D chromaticity feature space. This “invariant direction” is that for which particular colour features, when projected into 1D, produce a greyscale image which is approximately invariant to intensity and colour of scene illumination. Thus shadows, which are in essence a particular type of lighting, are greatly attenuated. The main approach to finding this special angle is a camera calibration: a colour target is imaged under many different lights, and the direction that best makes colour patch images equal across illuminants is the invariant direction. Here, we take a different approach. In this work, instead of a camera calibration we aim at finding the invariant direction from evidence in the colour image itself. Specifically, we recognize that producing a 1D projection in the correct invariant direction will result in a 1D distribution of pixel values that have smaller entropy than projecting in the wrong direction. The reason is that the correct projection results in a probability distribution spike, for pixels all the same except differing by the lighting that produced their observed RGB values and therefore lying along a line with orientation equal to the invariant direction. Hence we seek that projection which produces a type of intrinsic, independent of lighting reflectance-information only image by minimizing entropy, and from there go on to remove shadows as previously. To be able to develop an effective description of the entropy-minimization task, we go over to the quadratic entropy, rather than Shannon’s definition. Replacing the observed pixels with a kernel density probability distribution, the quadratic entropy can be written as a very simple formulation, and can be evaluated using the efficient Fast Gauss Transform. The entropy, written in this embodiment, has the advantage that it is more insensitive to quantization than is the usual definition. The resulting algorithm is quite reliable, and the shadow removal step produces good shadow-free colour image results whenever strong shadow edges are present in the image. In most cases studied, entropy has a strong minimum for the invariant direction, revealing a new property of image formation.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
References
Barrow, H., & Tenenbaum, J. (1978). Recovering intrinsic scene characteristics from images. In A. Hanson & E. Riseman (Eds.), Computer vision systems (pp. 3–26). New York: Academic Press.
Baxter, B., & Roussos, G. (2002). A new error estimate of the fast Gauss transform. SIAM Journal on Scientific Computing, 24(1), 257–259.
Beatson, R., & Greengard, L. (1997). A short course on fast multipole methods. In M. Ainsworth, J. Levesley, W. Light, & M. Marletta (Eds.), Wavelets, multilevel methods and elliptic PDEs. Oxford: Oxford University Press.
Bergner, S., Drew, M. S., & Möller (2009). A tool to create illuminant and reflectance spectra for light-driven graphics and visualization. ACM Transactions on Graphics, 28(5), 1–11.
Cho, J.-H., Kwon, T.-G., Jang, D.-G., & Hwang, C.-S. (2005). Moving cast shadow detection and removal for visual traffic surveillance. In: Australian conference on artificial intelligence (pp. 746–755).
CIE, (1995). Method of measuring and specifying colour rendering properties of light sources. Publication 13.3, ISBN 978-3900734572. http://www.cie.co.at/publ/abst/13-3-95.html.
Comaniciu, D., & Meer, P. (2002). Mean shift: a robust approach toward feature space analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24, 603–619.
Daly, S. (1992). The visible difference predictor: an algorithm for the assessment of image fidelity. In: A. Rogowitz and Klein (Eds.): Proceedings of SPIE: Vol. 1666. Human vision, visual processing, and digital display III (pp. 2–15).
Drew, M., Chen, C., Hordley, S., & Finlayson, G. (2002). Sensor transforms for invariant image enhancement. In: Tenth color imaging conference: color, science, systems and applications (pp. 325–329).
Drew, M., Finlayson, G., & Hordley, S. (2003). Recovery of chromaticity image free from shadows via illumination invariance. In: IEEE workshop on color and photometric methods in computer vision, ICCV’03 (pp. 32–39).
Drew, M., Salahuddin, M., & Fathi, A. (2007). A standardized workflow for illumination-invariant image extraction. In: 15th color imaging conference: color, science, systems and applications.
Elgammal, A., Duraiswami, R., & Davis, L. S. (2003). Efficient kernel density estimation using the fast Gauss transform with applications to color modeling and tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(11), 1499–1504.
Finlayson, G., & Drew, M. (2001). 4-sensor camera calibration for image representation invariant to shading, shadows, lighting, and specularities. In: ICCV’01: international conference on computer vision (pp. II: 473–480).
Finlayson, G., & Hordley, S. (2001). Colour constancy at a pixel. Journal of the Optical Society of America A, 18(2), 253–264.
Finlayson, G., Drew, M., & Funt, B. (1994). Spectral sharpening: sensor transformations for improved color constancy. Journal of the Optical Society of America A, 11(5), 1553–1563.
Finlayson, G., Hordley, S., & Drew, M. (2002). Removing shadows from images. In Lecture Notes in Computer Science : Vol. 2353. ECCV 2002: European conference on computer vision (pp. 823–836). Berlin: Springer.
Finlayson, G., Drew, M., & Lu, C. (2004). Intrinsic images by entropy minimization. In Lecture Notes in Computer Science : Vol. 3023. ECCV 2004: European conference on computer vision (pp. 582–595). Berlin: Springer.
Finlayson, G., Hordley, S., Lu, C., & Drew, M. (2006). On the removal of shadows from images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28, 59–68.
Finlayson, G., Fredembach, C., & Drew, M. S. (2007). Detecting illumination in images. In: ICCV’07: international conference on computer vision.
Greengard, L., & Strain, J. (1991). The fast Gauss transform. SIAM Journal on Scientific and Statistical Computing, 12(1), 79–94.
Hsu, E., Mertens, T., Paris, S., Avidan, S., & Durand, F. (2008). Light mixture estimation for spatially varying white balance. ACM Transactions on Graphics, 27(3), 1–7.
Jiang, H., & Drew, M. (2003) Shadow-resistant tracking in video. In: ICME’03: international conference on multimedia and expo (Vol. III, pp. 77–80).
Jiang, H., & Drew, M. (2007). Shadow resistant tracking using inertia constraints. Pattern Recognition, 40, 1929–1945.
Land, E., & McCann, J. (1971). Lightness and retinex theory. Journal of the Optical Society of America, 61, 1–11.
Li, Z.-N., & Drew, M. (2004). Fundamentals of multimedia. New York: Prentice-Hall.
Liu, Z., Huang, K., Tan, T., & Wang, L. (2006). Cast shadow removal with GMM for surface reflectance component. In: ICPR06 (pp. 727–730).
Martel-Brisson, N., & Zaccarin, A. (2007). Learning and removing cast shadows through a multidistribution approach. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29, 1133–1146.
Nadimi, S., & Bhanu, B. (2004). Physical models for moving shadow and object detection in video. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26, 1079–1087.
Parzen, E. (1962). On estimation of a probability density function and mode. Annals of Mathematical Statistics, 33, 1065–1076.
Prati, A., Mikic, I., Trivedi, M., & Cucchiara, R. (2003). Detecting moving shadows: algorithms and evaluation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25, 918–923.
Ramanath, R., Snyder, W., Yoo, Y., & Drew, M. S. (2005). Color image processing pipeline in digital still cameras. IEEE Signal Processing, 22(1), 34–43.
Renyi, A. (1987). A diary on information theory. New York: Wiley.
Scott, D. (1992). Multivariate density estimation: theory, practice and visualization. New York: Wiley and Kegan Paul.
Stauder, J., Mech, R., & Ostermann, J. (1999). Detection of moving cast shadows for object segmentation. IEEE Transactions on Multimedia, 1, 65–76.
Tappen, M., Freeman, W., & Adelson, E. (2003). Recovering intrinsic images from a single image. In: Advances in neural information processing systems 15.
Tappen, M., Freeman, W., & Adelson, E. (2005). Recovering intrinsic images from a single image. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27, 1459–1472.
Vrhel, M., Gershon, R., & Iwan, L. (1994). Measurement and analysis of object reflectance spectra. Color Research and Application, 19, 4–9.
Weiss, Y. (2001). Deriving intrinsic images from image sequences. In: ICCV01 (Vol. II, pp. 68–75.
Weyrich, T., Matusik, W., Pfister, H., Bickel, B., Donner, C., Tu, C., McAndless, J., Lee, J., Ngan, A., Wann Jensen, H., & Gross, M. (2006). Analysis of human faces using a measurement-based skin reflectance model. ACM Transactions on Graphics, 25, 1013–1024.
Wyszecki, G., & Stiles, W. (1982). Color science: concepts and methods, quantitative data and formulas (2nd ed.). New York: Wiley,
Xu, D., & Principe, J. (1998). Learning from examples with quadratic mutual information. In: Neural networks for signal processing (pp. 155–164).
Yang, C., Duraiswami, R., Gumerov, N., & Davis, L. (2003). Improved fast Gauss transform and efficient kernel density estimation. In: International conference on computer vision (pp. 464–471).
Author information
Authors and Affiliations
Corresponding author
Additional information
G.D. Finlayson’s work was supported by the Leverhulme Trust.
M.S. Drew’s work was supported by the Natural Sciences and Engineering Research Council of Canada.
Rights and permissions
About this article
Cite this article
Finlayson, G.D., Drew, M.S. & Lu, C. Entropy Minimization for Shadow Removal. Int J Comput Vis 85, 35–57 (2009). https://doi.org/10.1007/s11263-009-0243-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11263-009-0243-z