image: The figure highlights the core goal of SLeM, which is to construct a learning methodology mapping from task space to learning space. It also showcases three SLeM approaches for achieving this mapping in the “SLeM Framework & Approaches” cell. The lower cell depicts a series of SLeM algorithms and related applications developed based on the SLeM framework.
Credit: ©Science China Press
As a fundamental technology of artificial intelligence, existing machine learning (ML) methods often rely on extensive human intervention and manually presetting, like manually collecting, selecting, and annotating data, manually constructing the fundamental architecture of deep neural networks, and determining the algorithm types and their hyperparameters of the optimization algorithms, etc. These limitations limit the ability of ML to effectively deal with complex data and varying multi-tasks environments in the real world.
To address these challenges existed in current ML, the research team from Xi'an Jiaotong University has developed a novel approach called “simulating learning methodology” (SLeM briefly). The core concept of SLeM is to simulate and extract the “learning methodology” of ML traditionally set by humans, transforming it into an automated learning process. Essentially, the SLeM framework represents a “ML for ML” paradigm, where ML tools are employed to design and optimize ML’s fundamental components.
The author team has developed a series of algorithms for ML automation based on the SLeM framework, demonstrating its effectiveness in enhancing the adaptive learning capabilities of existing ML methods. “Recently, many AutoML methods have been proposed to achieve ML automation. However, most existing AutoML methods are heuristic in nature, making it difficult to establish a solid theoretical foundation. In contrast, the SLeM framework offers a unified mathematical formulation for ML automation and provides theoretical insights into the task transfer generalization capabilities of SLeM,” says Prof. Zongben Xu, the lead author of this paper and an academician of the Chinese Academy of Sciences.
The development of advanced large language models (LLMs) has become a cornerstone of artificial intelligence, significantly expanding the capacity to solve a wide range of applications and tasks. However, the underlying theoretical evidence for LLMs’ remarkable task generalization abilities has not been fully addressed by the ML community. The novel SLeM approach provides a promising perspective and tool for advancing the study and understanding of task generalization capabilities in large language models (LLMs).
See the article:
Simulating Learning Methodology (SLeM): An Approach to Machine Learning Automation
https://doi.org/10.1093/nsr/nwae277
Journal
National Science Review