Deep adaptive feature enrichment

Abstract

Features play an important role in the performance of machine learning and classification applications. Usually, separability of classes by using raw or original features are so low, and it is necessary to use complex classifiers with high computational costs or use enrichment modules to increase distinctiveness of features. In this paper, a deep feature enrichment method is proposed to increase the distinguishing power of features using an adaptive neural network-based structure. Proposed method adaptively uses linear/non-linear activation functions for coding, and the dimension of the coding space adaptively adjusted to be lower, the same, or higher than the original feature space. Then the best neural network structure (number of layers and neurons per layers) and the optimum weights for the proposed neural structure are optimized using an evolutionary optimization algorithm. Optimized modules can map/code raw input features into an enriched feature space that can increase the separability of the data points among classes. In fact, our obtained enriched features can adapt themselves to the nature of the training data and they can improve the generalization power also the performance of conventional classifiers. Experimental results on popular UCI datasets such as Glass, Liver, Iris, Wine, Breast cancer and seeds show increase of significant correct recognition rates (11.63% for Glass, 4.35% for Liver, 13.34% for Iris, 27.78% for Wine, 0.72% for Breast cancer and 11.9% for seeds) and also improvement of more than 1.5% of verification rate and 2% of Identification rate for the Yale face database.