A Hybrid Whale Genetic Algorithm for Feature Selection in Biomedical Dataset

1. Introduction

Classification is a process of segregating samples into their respective labels based on the information provided by their features. Some features in data can be relevant or irrelevant. Irrelevant and redundant features do not contribute to the classification task and reduce the performanceby increasing the search space(Miao & Niu, 2016). This is referredto as the curse of dimensionality problem. The problem can be tackled with the help of feature selection techniques. Feature selection helps to identify and select the features which contribute greatly to the classification process. It is different from dimensionality reduction and feature extraction. All the methods reduce the number of features in a dataset, but each onehas a different technique.Dimensionality reduction combines different features to form new attributes, feature extraction forms new features along the direction where maximum variation is present and feature selection just includes or excludes features present without any changes.

An efficient feature selection algorithm determines the subset of features that help in the precise determination of the class. The aim of feature selection is three-fold: improve the performance of classifiers, reducing computational complexity, and getting better insights from the data. Feature selection techniques help to select those features which will boost classification accuracy requiring fewer data. Fewer attributes are preferred because it reduces the complexity of the model and is easier to learn. Feature selection techniques are broadly divided into three categories: Filter, Wrapper, and Embedded.The filter approach(Bosin et al., 2007) applies mathematical measures and calculates a score for each feature. A rank is assigned to the features based on this score. This approach is independent of any supervised learning algorithm and hence achieves more generality than any other approach. They are called filters since they filter the features before training. It is also computationally less expensive and therefore good for high-dimensional spaces.The wrapper approach(Nnamoko et al., 2014) is based on a greedy search algorithm. It evaluates all possible combinations of the features andis based on a particular supervised learning algorithm. It selects a random subset of features and trains the algorithm with those features. The scoring function of the wrapper approach is the accuracy of the algorithm itself. Since it evaluates the feature subset using a specific learning algorithm, it does not possess generality like the filter method. Due to its high computational complexity, it can prove to be an obstruction in high-dimensional datasets. Filters are faster but less effective than wrappers because they do not take into consideration the fact that different learning algorithms might provide good results with different feature subsets.The embedded approach(Fu et al., 2009) uses a part of the learning algorithm to generate subsets and has less generality than a filter. An example of an embedded technique is a decision tree.At each step of the tree, a feature is selected recursively.Ridge with L2 penalty and LASSO with L1 penalty are also examples of theembedded approach.