Scientific machine learning benchmarks

J Thiyagalingam, M Shankar, G Fox, T Hey - Nature Reviews Physics, 2022 - nature.com
… such a benchmarking approach and identifying metrics for the application of machine learning
… Here, we introduce the concept of machine learning benchmarks for science and review …

Machine learning benchmarks and random forest regression

MR Segal - 2004 - escholarship.org
… with regard prediction error on a suite of benchmark datasets. As the base constituents of
the … of those benchmarking datasets extracted from the UCI machine learning repository for …

Survey and benchmarking of machine learning accelerators

A Reuther, P Michaleas, M Jones… - 2019 IEEE high …, 2019 - ieeexplore.ieee.org
… We then surveyed and analyzed machine learning processors categorized into six regions
… Finally, we presented benchmarking results for two low power machine learning accelerator …

[HTML][HTML] PMLB: a large benchmark suite for machine learning evaluation and comparison

RS Olson, W La Cava, P Orzechowski, RJ Urbanowicz… - BioData mining, 2017 - Springer
… number of established machine learning methods to the entire benchmark suite and analyze
… existing benchmarks lack the diversity to properly benchmark machine learning algorithms, …

Accounting for variance in machine learning benchmarks

X Bouthillier, P Delaunay, M Bronzi… - … Machine Learning …, 2021 - proceedings.mlsys.org
… Our findings outline recommendations to improve reliability of machine learning benchmarks:
1) randomize as many sources of variations as possible in the performance estimation; 2) …

[HTML][HTML] MoleculeNet: a benchmark for molecular machine learning

Z Wu, B Ramsundar, EN Feinberg, J Gomes… - Chemical …, 2018 - pubs.rsc.org
… a series of benchmark results of implemented machine learning algorithms … machine learning
models included as part of MoleculeNet. The results section will analyze the benchmarking

Benchmark and survey of automated machine learning frameworks

MA Zöller, MF Huber - Journal of artificial intelligence research, 2021 - jair.org
machine learning applications automatically without extensive knowledge of statistics and
machine learning. … current AutoML methods and a benchmark of popular AutoML frameworks …

Open graph benchmark: Datasets for machine learning on graphs

W Hu, M Fey, M Zitnik, Y Dong, H Ren… - Advances in neural …, 2020 - proceedings.neurips.cc
benchmark datasets to facilitate scalable, robust, and reproducible graph machine learning
(… In addition to building the datasets, we also perform extensive benchmark experiments for …

Pervasive label errors in test sets destabilize machine learning benchmarks

CG Northcutt, A Athalye, J Mueller - arXiv preprint arXiv:2103.14749, 2021 - arxiv.org
… in these benchmarks’ test data, we find that relative rankings of models in benchmarks
are … However, we find that these benchmark results are unstable: higher-capacity models (like …

A case study on machine learning for synthesizing benchmarks

A Goens, A Brauckmann, S Ertel, C Cummins… - … on Machine Learning …, 2019 - dl.acm.org
… are natural candidates for synthetic benchmark generation. In … of machine learning in the
prominent CLgen benchmark generator. We re-evaluate CLgen by comparing the benchmarks