Evaluation of Optimal Feature Transformation Using Particle Swarm Optimization

Abstract

Feature reduction is one of the essential steps for machine learning applications. It reduces redundancy in the feature set, which reduces the computational cost in the learning phase. The success of the reduction stage depends on the size of the feature selected and the separability of the transformed matrix. In most of the work, the feature transformation matrix is determined mathematically and most of the time depends on eigenvectors and eigenvalues. If the feature space is high, it is difficult to calculate the eigen matrix in smaller devices. Hence, this work proposes a method to generate an optimum transformation matrix using heuristic optimization approach that leads to better classification accuracy with less feature size. This study uses Bhattacharyya distance as an objective function to evaluate the separability of the transformed feature matrix. Moreover, to select a proper subset of features, a penalty parameter is added with respect to number of features in transformation matrix. This work proposes a modified version of particle swarm optimization that can satisfy the objective. The study shows the ability to learn a transformation matrix with competitive results in classification task. The proposed method is evaluated on various freely available public datasets, namely Fisher’s IRIS, Wine, Wisconsin Breast Cancer, Ionosphere, Sonar, MNIST, NIT-R Bangla Numeral, and ISI-K Bangla Numeral datasets.