Swarmrob: A toolkit for reproducibility and sharing of experimental artifacts in robotics research

A Pörtner, M Hoffmann, M König - arXiv preprint arXiv:1801.04199, 2018 - arxiv.org
Due to the complexity of robotics, the reproducibility of results and experiments is one of the
fundamental problems in robotics research. While the problem has been identified by the …

Swarmrob: A docker-based toolkit for reproducibility and sharing of experimental artifacts in robotics research

A Pörtner, M Hoffmann, S Zug… - 2018 IEEE International …, 2018 - ieeexplore.ieee.org
Due to the high complexity, the reproducibility of experiments is one of the fundamental
problems of robotics research. Although the problem has been identified by the community …

Towards automated system and experiment reproduction in robotics

F Lier, M Hanheide, L Natale, S Schulz… - 2016 IEEE/RSJ …, 2016 - ieeexplore.ieee.org
Even though research on autonomous robots and human-robot interaction accomplished
great progress in recent years, and reusable soft-and hardware components are available …

[PDF][PDF] Can we reproduce it? toward the implementation of good experimental methodology in interdisciplinary robotics research

F Lier, P Lücking, J De Leeuw… - ICRA 2017 workshop …, 2017 - researchgate.net
The insufficient level of reproducibility of published experimental results has been identified
as a core issue in the field of robotics in recent years. Why is that? First of all, robotics …

Robot runner: a tool for automatically executing experiments on robotics software

S Swanborn, I Malavolta - 2021 IEEE/ACM 43rd International …, 2021 - ieeexplore.ieee.org
Software is becoming the core aspect in robotics development and it is growing in terms of
complexity and size. However, roboticists and researchers are struggling in ensuring and …

Robobench: Towards sustainable robotics system benchmarking

J Weisz, Y Huang, F Lier… - … on Robotics and …, 2016 - ieeexplore.ieee.org
We present RoboBench, a novel platform for sharing robot full-system simulations for
benchmarking. The creation of this platform and benchmark suite is motivated by a need for …

Run to the Source: The Effective Reproducibility of Robotics Code Repositories

E Cervera - IEEE Robotics & Automation Magazine, 2023 - ieeexplore.ieee.org
In recent years the robotics community has actively embraced the open paradigm, and
research articles are commonly enriched with the inclusion of a source code repository of …

Towards Using Multiple Iterated, Reproduced, and Replicated Experiments with Robots (MIRRER) for Evaluation and Benchmarking

A Norton, B Flynn - arXiv preprint arXiv:2408.04736, 2024 - arxiv.org
The robotics research field lacks formalized definitions and frameworks for evaluating
advanced capabilities including generalizability (the ability for robots to perform tasks under …

Integrated benchmarking and design for reproducible and accessible evaluation of robotic agents

J Tani, AF Daniele, G Bernasconi… - 2020 IEEE/RSJ …, 2020 - ieeexplore.ieee.org
As robotics matures and increases in complexity, it is more necessary than ever that robot
autonomy research be reproducible. Compared to other sciences, there are specific …

A Framework for Remotely Sharing Experimental Environments

SC Livingston - 2024 IEEE Intelligent Vehicles Symposium (IV), 2024 - ieeexplore.ieee.org
Reproducibility is fundamental to robotics research, but access to appropriate hardware is a
major limiting factor of reproducing experiments. This limitation usually arises from custom …