Document Type

Conference Proceeding

Publication Title

Proceedings of Machine Learning Research

Abstract

This paper focuses on continual meta-learning, where few-shot tasks are heterogeneous and sequentially available. Recent works use a mixture model for meta-knowledge to deal with the heterogeneity. However, these methods suffer from parameter inefficiency caused by two reasons: (1) the underlying assumption of mutual exclusiveness among mixture components hinders sharing meta-knowledge across heterogeneous tasks. (2) they only allow increasing mixture components and cannot adaptively filter out redundant components. In this paper, we propose an Adaptive Compositional Continual Meta-Learning (ACML) algorithm, which employs a compositional premise to associate a task with a subset of mixture components, allowing meta-knowledge sharing among heterogeneous tasks. Moreover, to adaptively adjust the number of mixture components, we propose a component sparsification method based on evidential theory to filter out redundant components. Experimental results show ACML outperforms strong baselines, showing the effectiveness of our compositional meta-knowledge, and confirming that ACML can adaptively learn meta-knowledge.

First Page

37358

Last Page

37378

Publication Date

7-2023

Keywords

Evidential theory, Knowledge-sharing, Learn+, Meta-knowledge, Metalearning, Mixture components, Mixture modeling, Sparsification

Comments

Open Access version from PMLR

Uploaded on June 20, 2024

Download

Share

COinS