Date of Graduation
Master of Science in Computer Engineering (MSCmpE)
Computer Science & Computer Engineering
Michael S. Gashler
Second Committee Member
Wing Ning Li
Applied sciences; Machine learning; Neural networks; Weight averaging
We investigate a method, weighted average model fusion, that enables neural networks to learn from the experiences of other networks, as well as from their own experiences. This method is inspired by the the Social natural of humans, which has been shown to be one of the biggest factors in the development of our cognitive abilities. Modern machine learning has focuses predominantly on learning from direct training, and has largely ignored learning through Social engagement with peers, neural networks will the same topology. In order to explore learning through engagement with peers, we have created a way for neural networks to teach each other. Our method allows neural networks to exchange knowledge by combining their weights. It calculates a pairwise weighted average of the weights of two neural networks, and then replaces the existing weight with the new value. We find that weighted average model fusion successfully enables neural networks to learn from the experiences of their peers and combine it with the knowledge that is gained from its own
individual experiences. Additionally, we explore the effects that several meta-parameters have on model fusion to provide deeper insights into how the behaves in a variety of scenarios.
Smith, Josh Reeves, "Investigation of How Neural Networks Learn From the Experiences of Peers Through Periodic Weight Averaging" (2017). Theses and Dissertations. 1877.