Evolutionary Training of Binary Neural Networks by Differential Evolution - Okada, Hidehiko (International Journal of Scientific Research in Computer Science and Engineering, 2022)
A problem with deep neural networks is that the memory size for recording a trained model becomes large. A solution to this problem is to make the parameter values binary. A challenge for the binary neural networks is that they cannot be trained by the ordinary gradient-based optimization methods. The author previously applied Evolution Strategy (ES) and Genetic Algorithm (GA) to the training of binary neural networks and evaluates its ability. In this paper, the author applies Differential Evolution, another instance of evolutionary algorithms, and compares DE with ES and GA. The experimental results with a classification task revealed that DE could also optimize binary weights well so that the trained model accurately classified both trained and untrained data. Classification accuracies for training data were significantly better by DE than those by ES and GA, which revealed better ability of DE in training binary neural networks.