Date of Graduation
5-2017
Document Type
Thesis
Degree Name
Master of Science in Computer Engineering (MSCmpE)
Degree Level
Graduate
Department
Computer Science & Computer Engineering
Advisor/Mentor
Michael S. Gashler
Committee Member
Susan Gauch
Second Committee Member
Wing Ning Li
Keywords
Applied sciences, Machine learning, Neural networks, Weight averaging
Abstract
We investigate a method, weighted average model fusion, that enables neural networks to learn from the experiences of other networks, as well as from their own experiences. This method is inspired by the the Social natural of humans, which has been shown to be one of the biggest factors in the development of our cognitive abilities. Modern machine learning has focuses predominantly on learning from direct training, and has largely ignored learning through Social engagement with peers, neural networks will the same topology. In order to explore learning through engagement with peers, we have created a way for neural networks to teach each other. Our method allows neural networks to exchange knowledge by combining their weights. It calculates a pairwise weighted average of the weights of two neural networks, and then replaces the existing weight with the new value. We find that weighted average model fusion successfully enables neural networks to learn from the experiences of their peers and combine it with the knowledge that is gained from its own
individual experiences. Additionally, we explore the effects that several meta-parameters have on model fusion to provide deeper insights into how the behaves in a variety of scenarios.
Citation
Smith, J. R. (2017). Investigation of How Neural Networks Learn From the Experiences of Peers Through Periodic Weight Averaging. Graduate Theses and Dissertations Retrieved from https://scholarworks.uark.edu/etd/1877