6 Auto-sklearn: Efficient and Robust Automated Machine Learning 117
and a set of meta-features, i.e., characteristics of the dataset that can be computed
efficiently and that help to determine which algorithm to use on a new dataset.
This meta-learning approach is complementary to Bayesian optimization for
optimizing an ML framework. Meta-learning can quickly suggest some instan-
tiations of the ML framework that are likely to perform quite well, but it is
unable to provide fine-grained information on performance. In contrast, Bayesian
optimization is slow to start for hyperparameter spaces as large as those of
entire ML frameworks, but can fine-tune performance over time. We exploit this
complementarity by selecting k configurations based on meta-learning and use their
result to seed Bayesian optimization. This approach of warmstarting optimization
by meta-learning has already been successfully applied before [21, 22, 38], but
never to an optimization problem as complex as that of searching the space of
instantiations of a full-fledged ML framework. Likewise, learning across datasets
has also been applied in collaborative Bayesian optimization methods [4, 45]; while
these approaches are promising, they are so far limited to very few meta-features and
cannot yet cope with the high-dimensional partially discrete configuration spaces
faced in AutoML.
More precisely, our meta-learning approach works as follows. In an offline phase,
for each machine learning dataset in a dataset repository (in our case 140 datasets
from the OpenML [43] repository), we evaluated a set of meta-features (described
below) and used Bayesian optimization to determine and store an instantiation of
the given ML framework with strong empirical performance for that dataset. (In
detail, we ran SMAC [27] for 24 h with 10-fold cross-validation on two thirds of
the data and stored the resulting ML framework instantiation which exhibited best
performance on the remaining third). Then, given a new dataset D, we compute its
meta-features, rank all datasets by their L
1
distance to D in meta-feature space and
select the stored ML framework instantiations for the k = 25 nearest datasets for
evaluation before starting Bayesian optimization with their results.
To characterize datasets, we implemented a total of 38 meta-features from the
literature, including simple, information-theoretic and statistical meta-features [29,
33], such as statistics about the number of data points, features, and classes, as
well as data skewness, and the entropy of the targets. All meta-features are listed in
Table 1 of the original publication’s supplementary material [20]. Notably, we had
to exclude the prominent and effective category of landmarking meta-features [37]
(which measure the performance of simple base learners), because they were
computationally too expensive to be helpful in the online evaluation phase. We note
that this meta-learning approach draws its power from the availability of a repository
of datasets; due to recent initiatives, such as OpenML [43], we expect the number
of available datasets to grow ever larger over time, increasing the importance of
meta-learning.
剩余21页未读，继续阅读
P("Struggler")?
- 粉丝: 1122
- 资源: 4
我的内容管理
收起
- 我的资源
快来上传第一个资源
我的收益 登录查看自己的收益
我的积分
登录查看自己的积分
我的C币
登录后查看C币余额
我的收藏 
我的下载 
下载帮助
会员权益专享
最新资源
- 京瓷TASKalfa系列维修手册：安全与操作指南
- 小波变换在视频压缩中的应用
- Microsoft OfficeXP详解：WordXP、ExcelXP和PowerPointXP
- 雀巢在线媒介投放策划：门户网站与广告效果分析
- 用友NC-V56供应链功能升级详解(84页)
- 计算机病毒与防御策略探索
- 企业网NAT技术实践：2022年部署互联网出口策略
- 软件测试面试必备：概念、原则与常见问题解析
- 2022年Windows IIS服务器内外网配置详解与Serv-U FTP服务器安装
- 中国联通：企业级ICT转型与创新实践
- C#图形图像编程深入解析：GDI+与多媒体应用
- Xilinx AXI Interconnect v2.1用户指南
- DIY编程电缆全攻略：接口类型与自制指南
- 电脑维护与硬盘数据恢复指南
- 计算机网络技术专业剖析：人才培养与改革
- 量化多因子指数增强策略：微观视角的实证分析
资源上传下载、课程学习等过程中有任何疑问或建议，欢迎提出宝贵意见哦~我们会及时处理！
点击此处反馈
