Abstract
Mosaicing is connecting two or more images and making a new wide area image with no visible seam-lines. Several algorithms have been proposed to construct mosaics from image sequence where the camera motion is more or less complex. Most of these methods are based either on the interest points matching or on theoretical corner models. This paper describes a fully automated image-mosaicing method based on the regions and the Harris points primitives. Indeed, in order to limit the search window of potential homologous points, for each point of interest, regions segmentation and matching steps are being performed. This enables us to improve the reliability and the robustness of the Harris points matching process by estimating the camera motion. The main originality of the proposed system resides in the preliminary manipulation of regions matching, thus making it possible to estimate the rotation, the translation and the scale factor between two successive images of the input sequence. This estimation allows an initial alignment of the images along with the framing of the interest points search window, and therefore reducing considerably the complexity of the interest points matching algorithm. Then, the resolution of a minimization problem, altogether considering the couples of matched-points, permits us to perform the homography. In order to improve the mosaic continuity around junctions, radiometric corrections are applied. The validity of the herewith described method is illustrated by being tested on several sequences of complex and challenging images captured from real-world indoor and outdoor scenes. These simulations proved the validity of the proposed method against camera motions, illumination variations, acquirement conditions, moving objects and image noise. To determine the importance of the regions matching stage in motion estimation, as well as for the framing of the search window associated to a point of interest, we compared the matching points results of this described method with those produced using the zero-mean normalized cross correlation score (without regions matching). We made this comparison in the case of a simple motion (without the presence of a rotation around optical axis and/or a scale factor), in the case of a rotation and in the general case of an homothety. For justifying the effectiveness of this method, we proposed an objective assessment by defining a reconstruction error.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Barhoumi, W., Zagrouba, E.: Segmentation en régions guidée par l’appariement pour un couple stéréoscopique non calibré. In: Proceedings of the 3eme Workshop Traitements et Analyse d’Information: Méthodes et Applications, Hammamet, Tunisia, pp. 287–292 (2003)
Baumberg, A.: Reliable feature matching across widely separated views. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, South Carolina, pp. 774–781 (2000)
Bhosle, U., Chaudhuri, S., DuttaRoy, S.: Fast method for image mosaicing using geometric hashing. IETE J. Res. Spec. Issue Visual Media Process. 48(4), 317–324 (2002)
Brown, M., Lowe, D.: Invariant features from interest point groups. In: Proceedings of the 13th British Machine Vision Conference, Cardiff, pp. 253–262 (2002)
Brown, M., Lowe, D.G.: Recognising panoramas. In: Proceedings of the IEEE International Conference on Computer Vision, Nice, pp. 1218–1225 (2003)
Brown, M.: Multi-image matching using invariant features. PhD thesis, The University of British Columbia (2005)
Burt, P.J., Adelson, E.H.: A multiresolution spline with application to image mosaics. ACM Trans. Graph. 2(4), 217–236 (1983)
Can, A., Stewart, C.V., Roysam, B.: Robust hierarchical algorithm for constructing mosaic from images of the curved human retina. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Colorado, pp. 286–292 (1999)
Choe, T.E., Cohen, I., Lee, M., Medioni, G.: Optimal global mosaic generation from retinal images. In: Proceedings of 18th International Conference on Pattern Recognition, Hong Kong, pp. 681–684 (2006)
Colombo, C., DelBimbo, A., Pernici, F.: Image mosaicing from uncalibrated views of a surface of revolution. In: Proceedings of British Machine Vision Conference, London (2004)
Davis, J.: Mosaics of scenes with moving objects. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Santa Barbara (1998)
Dellaert, F., Thrun, S., Thorpe, C.: Mosaicing a large number of widely dispersed, noisy, and distorted images: a bayesian approach. Technical Report, CMU-RI-TR-9934, Robotics Institute, Carnegie Mellon University, USA (1999)
Deriche, R., Blaszka, T.: Recovering and characterizing image features using an efficient model based approach. In: Proceedings of the Conference on Computer Vision and Pattern Recognition, New York, pp. 530–535 (1993)
Deriche, R.: Recursively implementing the Gaussian and its derivatives. Technical Report, RR 2422 INRIA, France (1993)
Douze, M., Charvillat, V., Thiesse, B.: Mosaïques d’images par approximations successives. In: Proceedings of Journées Francophones des Jeunes Chercheurs en Analyse d’Images et Perception Visuelle (ORASIS’01), Cahors, pp 97–102 (2001)
Duplaquet, M.L.: Building large image mosaics with invisible seam-lines. In: Proceedings of the SPIE Congres AeroSence, Orlando, pp. 369–377 (1998)
Feldman, D., Zomet, A.: Generating mosaics with minimum distortions. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshop, Washington DC, pp. 163–170 (2004)
Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Comm. ACM 24, 381–395 (1981)
Fraundorfer, F., Bischof, H.: Evaluation of local detectors on non-planar scenes. In: Proceedings of the 28th Workshop of the Austrian Association for Pattern Recognition, Hagenberg, pp. 125–132 (2004)
Fusiello, A., Aprile, M., Marzotto, R., Murino, V.: Mosaic of a video shot with multiple moving objects. In: Proceedings of the International Conference on Image Processing, Barcelona, pp. 307–310 (2003)
Gong, Y., Proietti, G., LaRose., D.: A robust image mosaicing technique capable of creating integrated panoramas. In: Proceedings of the IEEE International Conference on Information Visualization, London, pp. 24–29 (1999)
Gracias, X., Negahdaripour, S.: Underwater mosaic creation using video sequences from different altitudes. In: Proceedings of OCEANS’05 MTS/IEEE Conference, Boston, pp. 1295–1300 (2005)
Harris, C., Stephens, A.: Combined corner and edge detector. In: Proceedings of the 4th Alvey Vision Conference, Manchester, pp. 147–151 (1988)
Heikkila, M., Pietikainen, M.: An image mosaicing module for wide-area surveillance. In: Proceedings of the Third ACM International Workshop on Video Surveillance & Sensor Networks, Singapore, pp. 11–18 (2005)
Huang, X., Sun, Y., Metaxas, D., Sauer, F., Xu, C.: Hybrid image registration based on configural matching of scale-invariant salient region features. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshop, Washington DC, pp. 167–177 (2004)
Irani, M., Hsu, S., Anandan, P.: Video compression using mosaic representations. Elsevier-Sig. Process. Image Commun. 7, 529–552 (1995)
Jethwa, M., Zisserman, A., Fitzgibbon, A.W.: Real-Time panoramic mosaics and augmented reality. In: Proceedings of the Britsh Machine Vision Conference, London, pp. 852–862 (1998)
Jianchao, Y., Chern, C.T.: The practice of automatic satellite image registration. Asian J. Geoinf. 3(4), 11–18 (2003)
Kanade, T., Rander, P.W., Narayaman, P.J.: Virtualized reality: constructing virtual worlds from real scenes. IEEE Trans. Multimedia 4(1), 34–47 (1997)
Richmond, K., Rock, S.M.: An operational real-time large-scale visual mosaicking and navigation system. In: Proceedings of OCEANS’06 MTS/IEEE Conference, Boston, pp. 1–6 (2006)
Krotkov, E., Hebert, M., Simmons, R.: Stereo perception and dead reckoning for a prototype lunar rover. Auton. Robots 2(4), 313–331 (1995)
Lhuillier, M., Quan, L.: Edge-constrained joint view triangulation for image interpolation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, South Carolina, pp. 218–224 (2000)
Lowe, D.: Object recognition from local scale- invariant features. In: Proceedings of the International Conference on Computer Vision, Corfu, pp. 1150–1157 (1999)
Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)
Mallick, S.P.: Feature based image mosaicing. Technical Report, CSE 252C University of California, San Diego (2002)
Mann, S., Picard, R.: Virtual bellows: Constructing high quality stills from video. In: First IEEE International Conference on Image Processing, Austin, pp. 363–367 (1994)
Marzotto, R., Fusiello, A., Murino, V.: High resolution video mosaicing with global alignment. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Washington, pp. 692–698 (2004)
Matas, J., Burianek, J., Kittler, J.: Object recognition using the invariant pixel-set signature. In: Proceedings of the British Machine Vision Conference, London, pp. 606–615 (2000)
Matas, J., Chum, O., Urban, M., Pajdla, T.: Robust wide baseline stereo from maximally stable extremal regions. In: Proceedings of the British Machine Vision Conference, Cardiff, pp. 384–393 (2002)
McLauchlan, P.F., Jaenicke, A.: Image mosaicing using sequential bundle adjustment. In: Proceedings of the British Machine Vision Conference, Bristol, pp. 616–625 (2000)
McMillan, L., Bishop, G.: Plenoptic modeling: an image-based rendering system. In: Proceedings of the 22nd Annual Conference on Computer Graphics (SIGGRAPH’95), Los Angeles, pp. 39–46 (1995)
Mehnert, A., Jackway, P.: An improved seeded region growing algorithm. Pattern Recogn. Lett. 18, 1065–1071 (1997)
Merino, L., Wiklund, J., Caballero, F., Moe, A., De Dios, J.R.M., Forssen, P.E., Nordberg, K., Ollero, A.: Vision-based multi-UAV position estimation: Localization based on blob features for exploration missions. IEEE Robot. Autom. Magaz. 13(3), 53–62 (2006)
Meunier, L., Borgmann, M.: High-resolution panoramas using image mosaicing. Final Project, EE368 (Digital Image Processing), Stanford University, USA (2000)
Mikolajczyk, K., Schmid, C.: A performance evaluation of local descriptors. In: Proceedings of the Interational Conference on Computer Vision and Pattern Recognition (2003)
Mikolajczyk, K., Schmid, C.: Comparaison of affine-invariant local detectors and descriptors. In: Proceedings of the 12th European Signal Processing Conference, Vienna (2004)
Mikolajczyk, K., Tuytelaars, T., Schmid, C., Zisserman, A., Matas, J., Schaffalitzky, F., Kadir, T., Van Gool, L.: A comparaison of affine region detectors. Int. J. Comput. Vis. 65(1–2), 43–72 (2006)
NagaoM. Matsuyama, T.: Edge preserving smoothing. Comput. Graph. Image Process. 9, 394–407 (1979)
Nicolas, H.: New methods for dynamic mosaicing. IEEE Trans. Image Process. 10(8), 1239–1251 (2001)
Peleg, S., Rousso, B., Rav-Acha, A., Zomet, A.: Mosaicing on adaptive manifolds. IEEE Trans. Pattern Anal. Mach. Intell. 22(10), 1144–1154 (2000)
Peleg, S., Herman, J.: Panoramic mosaics by manifold projection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, San Juan, pp. 338–343 (1997)
Remondino, F.: Detectors and descriptors for photogrammetric applications. In: Proceedings of the Symposium on Photogrammetric Computer Vision, Bonn, pp. 49–54 (2006)
Robinson, J.A.: A simplex-based projective transform estimator. In: Proceedings of the International Conference on Visual Information Engineering, Guildford, pp. 290–293 (2003)
Rousso, B., Peleg, S., Finci, I., Rav-Acha, A.: Universal mosaicing using pipe projection. In: Proceedings of the Sixth IEEE International Conference on Computer Vision, Bombay, pp. 945–952 (1998)
Sawhney, H.S., Kumar, R., Gendel, G., Bergen, J., Dixon, D., Paragano, V.: VideoBrushTM: experiences with consumer video mosaicing. In: Proceedings of the Workshop on Applications of Computer Vision, Los Alamitos, pp. 56–62 (1998)
Schechner, Y.Y., Nayar, S.K.: Generalized mosaicing. In: Proceedings of the IEEE International Conference on Computer Vision, vol. 1, pp. 17–24, Vancouver (2001)
Schmid, C., Mohr, R., Bauckhage, C.: Evaluation of interest point detectors. Int. J. Comput. Vis. 37(2), 151–172 (2000)
Schmid, C., Mohr, R.: Local grayvalue invariants for image retrieval. IEEE Trans. Pattern Anal. Mach. Intell. 19(5), 530–534 (1997)
Shum, H.Y., Szeliski, R.: Systems and experiment paper: construction of panoramic mosaics with global and local alignment. Int. J. Comput. Vis. 36(2), 101–130 (2000)
Shum, H., Szeliski, R.: Panoramic image mosaics. Technical Report MSR-TR-97-23, Microsoft Research, USA (1997)
Singh, R., Vatsa, M., Ross, A., Noore, A.: Performance enhancement of 2D face recognition via mosaicing. In: Proceedings of the Fourth IEEE Workshop on Automatic Identification Advanced Technologies, MorganTown, pp. 63–68 (2005)
Smith, P., Sinclair, D., Cipolla, R., Wood, K.: Effective corner matching. In: Proceedings of the 9th British Machine Vision Conference, London, pp. 545–556 (1998)
Sun, C.: Dealing with dense rows in the solution of sparse linear least squares problems. Technical Report CTC95TR227, Advanced Computing Research Institute, Cornell University, Ithaca (1995)
Szeliski, R., Kang, S.B.: Direct methods for visual scene reconstruction. In: IEEE Workshop on Representation of Visual Scenes, Cambridge, pp. 26–33 (1995)
Tell, D., Carlsson, S.: Combining appearance and topology for wide baseline matching. In: Proceedings of the 7th European Conference on Computer Vision, Copenhagen, pp. 68–81 (2002)
Teyssier, P.: Reconstruction panoramique. Technical Report, Ecole Polytechnique, France (1999)
Trucco, E., Doull, A., Odone, F., Fusiello, A., Lane, D.: Dynamic video mosaicing and augmented reality for subsea inspection and monitoring. In: Oceonology International 2000, Bringhton (2000)
Unnikrishnan, R., Kelly, A.: Mosaicing large cyclic environments for visual navigation in autonomous vehicles. In: Proceedings of the IEEE International Conference on Robotics and Automation, Washington DC, pp. 4299–4306 (2002)
Van Gool, L., Tuytelaars, T., Turina, A.: Local features for image retrieval. In: State-of-the-Art in Content-Based Image and Video Retrieval, pp. 21–41. Kluwer, Dordercht (2001)
Wang, J., Quan, L., Sun, J., Tang, X., Shum, H.Y.: Picture Collage. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, New York, pp. 347–354 (2006)
Winter, M., Bischof, H., Fraundorfer F.: Maximally stable corner clusters: a novel distinguished region detector and descriptor. In: Proceedings of the Austrian Cognitive Vision Workshop, pp. 59–66 (2005)
You, X., Fang, B., Tang, Y.Y.: Mosaicing the retinal fundus images: a robust registration technique based approach. Lect. Notes Comput. Sci. Adv. Natural Comput. 3612, 663–667 (2005)
Zagrouba, E., Belhassen, S.: Construction Robuste de Mosaiques Généralisées. In: Proceedings of the International Conference on Image and Signal Processing (ICISP’2003), Agadir, pp. 519–526 (2003)
Zagrouba, E.: 3D facets construction for stereovision. In: International Conference on Artificial Intelligence pp. 116–126. Springer, Rome (1997)
Zuliani, M., Kenney, C., Manjunath, B.S.: A mathematical comparison of point detectors. In: Proceedings of the IEEE Image and Video Registration Workshop (IVR), Washington DC (2004)
Zitova, B., Flusser, J., Kautsky, J., Peters, G.: Feature point detection in multiframe images. In: Proceedings of the Czech Pattern Recognition Workshop, Czech Pattern Recognition Society, Perslak, pp. 117–122 (2000)
Zitova, B., Flusser, J.: Image registration methods: a survey. Image Vis. Comput. 21, 977–1000 (2003)
Zoghlami, I., Faugeras, O., Deriche, R.: Using geometric corners to build a 2D mosaic from a set of image. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Puerto Rico, pp. 420–425 (1997)
Zomet, A., Peleg, S., Arora, C.: Rectified mosaicing: mosaics without the curl. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, South Carolina, pp. 2459–2465 (2000)
Zomet, A., Peleg, S.: Applying super-resolution to panoramic mosaics. In: Proceedings of the IEEE Workshop on Applications of Computer Vision, Princeton, pp. 286–287 (1998)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Zagrouba, E., Barhoumi, W. & Amri, S. An efficient image-mosaicing method based on multifeature matching. Machine Vision and Applications 20, 139–162 (2009). https://doi.org/10.1007/s00138-007-0114-y
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00138-007-0114-y