[HTML][HTML] A BenchCouncil view on benchmarking emerging and future computing

J Zhan - BenchCouncil Transactions on Benchmarks, Standards …, 2022 - Elsevier
The measurable properties of the artifacts or objects in the computer, management, or
finance disciplines are extrinsic, not inherent—dependent on their problem definitions and …

AIBench training: Balanced industry-standard AI training benchmarking

F Tang, W Gao, J Zhan, C Lan, X Wen… - … Analysis of Systems …, 2021 - ieeexplore.ieee.org
Earlier-stage evaluations of a new AI architecture/system need affordable AI benchmarks.
Only using a few AI component benchmarks like MLPerf alone in the other stages may lead …

Scenario-based AI benchmark evaluation of distributed cloud/edge computing systems

T Hao, K Hwang, J Zhan, Y Li… - IEEE Transactions on …, 2022 - ieeexplore.ieee.org
Distributed cloud/edge (DCE) platform has become popular in recent years. This paper
proposes a new AI benchmark suite for assessing the performance of DCE platforms in …

Understanding hot interconnects with an extensive benchmark survey

Y Li, H Qi, G Lu, F Jin, Y Guo, X Lu - BenchCouncil Transactions on …, 2022 - Elsevier
Understanding the designs and performance characterizations of hot interconnects on
modern data center and high-performance computing (HPC) clusters is a fruitful research …

Hpc ai500 v2. 0: The methodology, tools, and metrics for benchmarking hpc ai systems

Z Jiang, W Gao, F Tang, L Wang… - 2021 IEEE …, 2021 - ieeexplore.ieee.org
Recent years witness a trend of applying large-scale distributed deep learning algorithms
(HPC AI) in both business and scientific computing areas, whose goal is to speed up the …

[HTML][HTML] DPUBench: An application-driven scalable benchmark suite for comprehensive DPU evaluation

Z Wang, C Wang, L Wang - BenchCouncil Transactions on Benchmarks …, 2023 - Elsevier
With the development of data centers, network bandwidth has rapidly increased, reaching
hundreds of Gbps. However, the network I/O processing performance of CPU improvement …

[HTML][HTML] MetaverseBench: Instantiating and benchmarking metaverse challenges

H Ye, L Wang - BenchCouncil Transactions on Benchmarks, Standards …, 2023 - Elsevier
The rapid evolution of the metaverse has led to the emergence of numerous metaverse
technologies and productions. From a computer systems perspective, the metaverse system …

[HTML][HTML] Open-source computer systems initiative: The motivation, essence, challenges, and methodology

J Zhan - BenchCouncil Transactions on Benchmarks, Standards …, 2022 - Elsevier
The global community faces many pressing and uncertain challenges like pandemics and
global climate change. Information technology (IT) infrastructure has become the enabler to …

[HTML][HTML] Training, testing and benchmarking medical AI models using clinical aibench

Y Huang, X Miao, R Zhang, L Ma, W Liu… - BenchCouncil …, 2022 - Elsevier
AI technology has been used in many clinical research fields, but most AI technologies are
difficult to land in real-world clinical settings. In most current clinical AI research settings, the …

[HTML][HTML] High fusion computers: The IoTs, edges, data centers, and humans-in-the-loop as a computer

W Gao, L Wang, M Chen, J Xiong, C Luo… - BenchCouncil …, 2022 - Elsevier
Emerging and future applications rely heavily upon systems consisting of Internet of Things
(IoT), edges, data centers, and humans-in-the-loop. Significantly different from warehouse …