526互联

Proj CDeepFuzz Paper Reading: AutoML: A survey of the state-of-the-art

发布时间 2023-08-31 16:17:25作者: 雪溯

Abstract

Github: https://github.com/marsggbo/automl_a_survey_of_state_of_the_art
本文：

  1. intro AutoML methods: data preparation, feature engineering, hyperparameter optimization, neural architecture search(NAS)
  2. 在CIFAR-10和ImageNet数据集上研究代表性NAS算法：one/two-stage NAS, one-shot NAS, joint hyperparameter, architecture optimization, resource-aware NAS
  3. 讨论open problem

1. Intro

2. Data Preparation

2.1 Data Collection

2.1.1 Data Searching

2.1.2 Data Synthesis

2.2 Data Cleaning

2.3 Data Augmentation

3. Feature Engineering

3.1 Feature Selection

3.2 Feature Construction

3.3 Feature Extraction

4. Model Generation

4.1 Search Space

4.1.1 Entire-structured Search Space

4.1.2 Cell-based Search Space

4.1.3 Hierarchical Search Space

4.1.4 Morphism-based Search Space

4.2 Architecture Optimization

4.2.1 Evolutionary Algo

4.2.2 Reinforcement Learning

4.2.3 Gradient Descent

4.2.4 Surrogate Model-based Optimization

4.2.6 Hybrid Optimization Method

4.3 Hyperparameter Optimization

4.3.2 Bayesian Optimization

4.3.3 Gradient-based Optimization

5 Model Evaluation

5.1 Low fidelity

5.2 Weight Sharing

5.3 Surrogate

5.4 Early Stopping

6. NAS Discussion

6.1 NAS Performance Comparision

6.1.1 Kendall Tau Metric

6.1.2 NAS-Bench Dataset

6.2 One-stage vs. Two-stage

6.3 One-shot/Weight-sharing

6.4 Joint Hyperparameter and Architecture Optimization

6.5 Resource-aware NAS

7 Open Problems and Future Directions

7.1 Flexible Search Space

7.2 Exploring More Areas

7.3 Interpretability

7.4 Reproducibility

7.5 Robustness

7.6 Joint Hypermeter and Architecture Optimization

7.7 Complete AutoML Pipeline

7.8 Lifelong Learning

7.8.1 Learn New Data

7.8.2 Remember Old Knowledge

8 Conclusions