Abstract
Stack Filters are a class of non-linear filter typically used for noise suppression. Advantages of Stack Filters are their generality and the existence of efficient optimization algorithms under mean absolute error (Wendt et al. in IEEE Trans. Acoust. Speech Signal Process. 34:898–910, 1986). In this paper we describe our recent efforts to use the class of Stack Filters for classification problems. This leads to a novel class of continuous domain classifiers which we call Ordered Hypothesis Machines (OHM). We develop convex optimization based learning algorithms for Ordered Hypothesis Machines and highlight their relationship to Support Vector Machines and Nearest Neighbor classifiers. We report on the performance on synthetic and real-world datasets including an application to change detection in remote sensing imagery. We conclude that OHM provides a novel way to reduce the number of exemplars used in Nearest Neighbor classifiers and achieves competitive performance to the more computationally expensive K-Nearest Neighbor method.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
References
Arce, G.: A general weighted median filter structure admitting negative weights. In: Proc. 11th Int. Joint Conf. on Artifical Intelligence, vol. 46, pp. 3195–3205 (1998)
Barner, K.: C-stack filters. In: ICASSP-91, International Conference on Acoustics, Speech, and Signal Processing, vol. 3, pp. 2005–2008 (1991). doi:10.1109/ICASSP.1991.150796
Blanchard, G., Schafer, C., Rozenholc, Y., Muller, K.R.: Optimal dyadic decision trees. Mach. Learn. 66(2–3), 209–241 (2007)
Bradski, G.: The OpenCV Library. Dr. Dobb’s J. Softw. Tools (2000)
Breiman, L.: Random forests. Mach. Learn. 45, 5–32 (2001). doi:10.1023/A:1010933404324
Breiman, L., Friedman, J., Olshen, R., Stone, C.: Classification and Regression Trees. Wadsworth, Belmont (1984)
Chang, C.C., Lin, C.J.: LIBSVM: a library for support vector machines (2001). Software available at http://www.csie.ntu.edu.tw/cjlin/libsvm
Fitch, J., Coyle, E., Gallagher, N.: Median filtering by threshold decomposition. IEEE Trans. Acoust. Speech Signal Process. ASSP-32(6), 1183–1189 (1984)
Han, C.C.: A supervised classification scheme using positive boolean function. In: 16th Int. Conf. on Pattern Recognition, vol. 2, pp. 100–103 (2002)
Hastie, T., Tibshirani, R.: Discriminant adaptive nearest neighbor classification. IEEE Trans. Pattern Anal. Mach. Intell. 18, 607–616 (1996). doi:10.1109/34.506411. URL http://portal.acm.org/citation.cfm?id=232678.232681
Jet Propulsion Laboratory (JPL), N.A., (NASA), S.A.: Airborne visible/infrared imaging spectrometer (aviris). http://aviris.jpl.nasa.gov/
Kaburlasos, V., Petridis, V.: Fuzzy lattice neurocomputing (fln) models. Neural Netw. 13(10), 1145–1170 (2000). doi:10.1016/S0893-6080(00)00074-5. URL http://www.sciencedirect.com/science/article/B6T08-41XM6GH-K/2/8961210cefa265a6d93967722c577d29
Kim, Y.T., Arce, G.: Permutation filter lattices: a general order-statistic filtering framework. IEEE Trans. Signal Process. 42(9), 2227–2241 (1994). doi:10.1109/78.317846
Lin, J., Coyle, E.J.: Minimum mean absolute error estimation over the class of generalized stack filters. IEEE Trans. Acoust. Speech Signal Process. 38, 663–678 (1990)
Lin, J., Sellke, T., Coyle, E.: Adaptive stack filtering under the mean absolute error criterion. In: Porter, W., Kak, S. (eds.) Advances in Communications and Signal Processing. Lecture Notes in Control and Information Sciences, vol. 129, pp. 263–276. Springer, Berlin (1989). doi:10.1007/BFb0042738
Muselli, M.: Approximation properties of positive boolean functions. In: Apolloni, B., Marinaro, M., Nicosia, G., Tagliaferri, R. (eds.) Neural Nets. 16th Italian Workshop on Neural Nets, WIRN 2005 and International Workshop on Natural and Artificial Immune Systems, NAIS 2005. Revised Selected Papers. Lecture Notes in Computer Science, vol. 3931. Springer, Berlin (2006)
Muselli, M.: Switching Neural Networks: A New Connectionist Model for Classification. Lecture Notes in Computer Science, vol. 3931, pp. 23–30 (2006)
Paredes, J.L., Arce, G.R.: Optimization of stack filters based on mirrored threshold decomposition. IEEE Trans. Signal Process. 49, 1179–1188 (2001)
Platt, J.C.: Fast training of support vector machines using sequential minimal optimization. In: Advances in Kernel Methods: Support Vector Learning, pp. 185–208 (1999)
Porter, R., Eads, D., Hush, D., Theiler, J.: Weighted order statistic classifiers with large rank-oder margin. In: Proc. 20th Int. Conf. on Machine Learning (2003)
Porter, R., Hush, D., Zimmer, B.: Error minimizing algorithms for nearest neighbor classifiers. In: Proceedings of the SPIE (2011)
Porter, R.B., Zimmer, G.B., Hush, D.: Stack filter classifiers. In: Wilkinson, M.F., Roerdink, J. (eds.) ISMM 2009, 9th International Symposium on Mathematical Morphology and Its Applications to Signal and Image Processing. Lecture Notes in Computer Science, vol. 5720, pp. 282–294. Springer, Berlin (2009)
Ritter, G.X., Sussner, P.: An introduction to morphological neural networks. In: 13th Int. Conf. on Pattern Recognition, vol. 4, pp. 709–717 (1996)
Sussner, P.: Morphological perceptron learning. In: International Symposium on Intelligent Systems and Semiotics, pp. 477–482 (1998)
Sussner, P., Esmi, E.L.: Morphological perceptrons with competitive learning: Lattice-theoretical framework and constructive learning algorithm. Inf. Sci. 181(10), 1929–1950 (2011)
Theiler, J.: Quantitative comparison of quadratic covariance-based anomalous change detector. Appl. Opt. 47, F12–F26 (2008)
Theiler, J., Perkins, S.: Proposed framework for anomalous change detection. In: ICML Workshop on Machine Learning Algorithms for Surveillance and Event Detection, pp. 7–14 (2006)
Tumer, K., Ghosh, J.: Linear and order statistics combiners for pattern classification. In: Combining Artificial Neural Nets, pp. 127–162 (1999)
Valle, M.E., Sussner, P.: A general framework for fuzzy morphological associative memories. Fuzzy Sets Syst. 159, 747–768 (2008). doi:10.1016/j.fss.2007.10.010. URL http://portal.acm.org/citation.cfm?id=1344840.1344937
Wang, J., Neskovic, P., Cooper, L.N.: Improving nearest neighbor rule with a simple adaptive distance rule. Pattern Recognit. Lett. 28, 207–213 (2006)
Wendt, P., Coyle, E., Gallagher, N.: Stack filters. IEEE Trans. Acoust. Speech Signal Process. 34, 898–910 (1986)
Wilson, D., Martinez, T.: Reduction techniques for instance-based learning algorithms. Mach. Learn. 38, 257–286 (2000)
Yang, P., Maragos, P.: Min-max classifiers: Learnability, design and application. Pattern Recognit. 28, 879–899 (1995)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Zimmer, G.B., Hush, D. & Porter, R. Ordered Hypothesis Machines. J Math Imaging Vis 43, 121–134 (2012). https://doi.org/10.1007/s10851-011-0293-z
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10851-011-0293-z