Boosting the training of neural networks through hybrid metaheuristics

Authors

Mohammed Azmi Al-Betar, Ajman UniversityFollow
Mohammed A. Awadallah, Al-Aqsa University
Iyad Abu Doush, American University of Kuwait
Osama Ahmad Alomari, University of Sharjah
Ammar Kamal Abasi, Mohamed Bin Zayed University of Artificial IntelligenceFollow
Sharif Naser Makhadmeh, Ajman University
Zaid Abdi Alkareem Alyasseri, University of Kufa, Information Technology Research and Development CentreFollow

Document Type

Article

Publication Title

Cluster Computing

Abstract

In this paper, the learning process of multilayer perceptron (MLP) neural network is boosted using hybrid metaheuristic optimization algorithms. Normally, the learning process in MLP requires suitable settings of its weight and bias parameters. In the original version of MLP, the gradient descent algorithm is used as a learner in MLP which suffers from two chronic problems: local minima and slow convergence. In this paper, six versions of memetic algorithms (MAs) are proposed to replace gradient descent learning mechanism of MLP where adaptive β-hill climbing (AβHC) as a local search algorithm is hybridized with six population-based metaheuristics which are hybrid flower pollination algorithm, hybrid salp swarm algorithm, hybrid crow search algorithm, hybrid grey wolf optimization (HGWO), hybrid particle swarm optimization, and hybrid JAYA algorithm. This is to show the effect of the proposed MA versions on the performance of MLP. To evaluate the proposed MA versions for MLP, 15 classification benchmark problems with different size and complexity are used. The AβHC algorithm is invoked in the improvement loop of any MA version with a probability of Br parameter, which is investigated to monitor its effect on the behavior of the proposed MA versions. The Br setting which obtains the most promising results is then used to set the hybrid MA. The results show that the proposed MA versions excel the original algorithms. Moreover, HGWO outperforms all other MA versions in almost all the datasets. In a nutshell, MAs are a good choice for training MLP to produce results with high accuracy.

DOI

10.1007/s10586-022-03708-x

Publication Date

8-26-2022

Keywords

Hybrid metaheuristics, Multilayer perceptron neural network, Optimization, β-hill climbing

Comments

IR Deposit conditions:

OA version (pathway b) Accepted version

12 months embargo

Published source must be acknowledged

Must link to publisher version with DOI

Post-prints are subject to Springer Nature re-use terms

Recommended Citation

M.A. Al-Betar, et al., "Boosting the training of neural networks through hybrid metaheuristics", <1>Cluster Computing, Aug. 2022, doi:10.1007/s10586-022-03708-x