Learning from Mistakes - A Framework for Neural Architecture Search

Authors

Bhanu Garg, University of California, San Diego, United States
Li Zhang, Zhejiang University, China
Pradyumna Sridhara, University of California, San Diego, United States
Ramtin Hosseini, University of California, San Diego, United States
Eric P. Xing, Carnegie Mellon University, United States & Mohamed bin Zayed University of Artificial IntelligenceFollow
Pengtao Xie, University of California, San Diego, United States

Document Type

Article

Publication Title

arXiv

Abstract

Learning from one's mistakes is an effective human learning technique where the learners focus more on the topics where mistakes were made, so as to deepen their understanding. In this paper, we investigate if this human learning strategy can be applied in machine learning. We propose a novel machine learning method called Learning From Mistakes (LFM), wherein the learner improves its ability to learn by focusing more on the mistakes during revision. We formulate LFM as a three-stage optimization problem: 1) learner learns; 2) learner re-learns focusing on the mistakes, and; 3) learner validates its learning. We develop an efficient algorithm to solve the LFM problem. We apply the LFM framework to neural architecture search on CIFAR-10, CIFAR-100, and Imagenet. Experimental results strongly demonstrate the effectiveness of our model. © 2021, CC BY.

DOI

10.48550/arXiv.2111.06353

Publication Date

11-11-2021

Keywords

Human learning, Learn+, Learning strategy, Learning techniques, Machine learning methods, Neural architectures, Optimization problems, Learning systems, Artificial Intelligence (cs.AI), Machine Learning (cs.LG)

Comments

Preprint: arXiv

Archived with thanks to arXiv

Preprint License: CC by 4.0

Uploaded 20 May 2022

Recommended Citation

B. Garg, L. Zhang, P. Sridhara, R. Hosseini, E. Xing, and P. Xie, "Learning from Mistakes - A Framework for Neural Architecture Search", arXiv, Nov 2021, doi: 10.48550/arXiv.2111.06353