Analyzing and Augmenting the Linear Classification Models

Abstract

Statistical learning theory offers an architecture needed for analysing the problem of inference, which includes, gaining knowledge, predictions, decisions or constructing models from a set of data. It is studied in a statistical architecture that is there are assumptions of statistical nature of the underlying phenomena. For predictive analysis, Linear Models are considered. These models tell about the relation between the target and the predictors using a straight line. Each linear model algorithm encodes specific knowledge, and works best when this assumption is satisfied by the problem to which it is applied. To generalize logistic regression to several classes, one possibility is to proceed in the way described previously for multi-response linear regression by performing logistic regression independently for each class. Unfortunately, the resulting probability estimates will not sum to one. In order to obtain proper probabilities, it is essential to combine the individual models for each class. This produces a joint optimization problem. A simple way is address multiclass problems also known as pair-wise classification. In this study, a classifier is derived for every pair of classes using only the instances from these two classes. The output on an unknown test example which is based on the class which receives maximum votes. This method has produced accurate results in terms of classification error. It is further used to produce probability estimates by applying a method called pair-wise coupling, which calibrates the individual probability estimates from the different classifiers.