Evaluation of Information Gain Feature Selection on Support Vector Machines Performance in Craniometric Sex Classification

Abstract

Sex classification is a fundamental stage in forensic anthropology because it provides the basis for constructing an individual's biological profile during the identification process. Although Support Vector Machine (SVM) has demonstrated high performance for craniometric sex classification, previous studies have primarily focused on classification performance using the complete feature set, with limited evaluation of the trade-off between feature reduction and predictive performance. Therefore, this study aims to evaluate the influence of Information Gain feature selection on SVM performance, identify the optimal combination of Information Gain threshold and SVM kernel configuration, and analyse the trade-off between feature reduction and classification performance. The proposed approach was evaluated using the William W. Howells Craniometric Dataset consisting of 2,524 skull samples. After removing identification attributes, 82 predictor features were used for classification. The research process included data preprocessing, label transformation, Z-score normalization, Information Gain feature selection using threshold values of 0.01, 0.05, and 0.1, followed by classification using SVM with Linear, Radial Basis Function (RBF), Polynomial, and Sigmoid kernels. Model performance was evaluated using 10-fold cross-validation based on Accuracy, Precision, Recall, and F1-score. The baseline SVM model without feature selection achieved an accuracy of 89.59% using 82 features. Experimental results show that Information Gain effectively reduced feature dimensionality while maintaining competitive classification performance. The best feature-selection result was obtained using an Information Gain threshold of 0.01, which retained 68 features. The optimal configuration used the RBF kernel with C = 1 and gamma='auto', achieving an Accuracy of 89.42%, Precision of 90.73%, Recall of 89.69%, and F1-score of 90.17%. Compared with the baseline model, the Information Gain-based model reduced the number of features by 17.07% while producing an accuracy difference of only 0.17 percentage points. These findings indicate that Information Gain primarily contributes to feature reduction and model simplification rather than improving classification accuracy. The results demonstrate that a moderate reduction in craniometric features can maintain classification performance close to the baseline model and highlight the trade-off between feature reduction and predictive performance in computer-assisted forensic sex classification.