Learning from the failure of autonomous and intelligent systems: Accidents, safety, and sociotechnical sources of risk

C Macrae - Risk analysis, 2022 - Wiley Online Library
Efforts to develop autonomous and intelligent systems (AIS) have exploded across a range
of settings in recent years, from self‐driving cars to medical diagnostic chatbots. These have …

System safety and artificial intelligence

R Dobbe - Proceedings of the 2022 ACM Conference on Fairness …, 2022 - dl.acm.org
This article formulates seven lessons for preventing harm in artificial intelligence (AI)
systems based on insights from the field of system safety for software-based automation in …

Understanding and avoiding ai failures: A practical guide

R Williams, R Yampolskiy - Philosophies, 2021 - mdpi.com
As AI technologies increase in capability and ubiquity, AI accidents are becoming more
common. Based on normal accident theory, high reliability theory, and open systems theory …

[HTML][HTML] Learning lessons for automated vehicle design: Using systems thinking to analyse and compare automation-related accidents across transport domains

GJM Read, A O'Brien, NA Stanton, PM Salmon - Safety science, 2022 - Elsevier
Automation is already on our roads, and advanced automated vehicles are anticipated to
become a common feature in road transport systems in the near future. While this represents …

Kayotee: A fault injection-based system to assess the safety and reliability of autonomous vehicles to faults and errors

S Jha, T Tsai, S Hari, M Sullivan, Z Kalbarczyk… - arXiv preprint arXiv …, 2019 - arxiv.org
Fully autonomous vehicles (AVs), ie, AVs with autonomy level 5, are expected to dominate
road transportation in the near-future and contribute trillions of dollars to the global …

Identifying effective risk management frameworks for complex socio-technical systems

B Luther, I Gunawan, N Nguyen - Safety science, 2023 - Elsevier
The advent of truly complex systems inside the last two decades has invalidated the
approach of now-antiquated risk management frameworks. Antiquated frameworks remain …

Autonomous vehicle accident data analysis: California OL 316 reports: 2015–2020

RL McCarthy - ASCE-ASME Journal of Risk and …, 2022 - asmedigitalcollection.asme.org
Abstract Six years (2015–2020) of autonomous vehicle (AV Level 3) crash data from
California's (CA) OL 316 collision reports of AV crashes while in the autonomous mode (AM) …

Model-based analysis of socio-technical risk

NG Leveson - 2004 - dspace.mit.edu
Traditional approaches to hazard analysis and safety-related risk management are based
on an accident model that focuses on failure events in static engineering designs and linear …

Guidance on the safety assurance of autonomous systems in complex environments (SACE)

R Hawkins, M Osborne, M Parsons… - arXiv preprint arXiv …, 2022 - arxiv.org
Autonomous systems (AS) are systems that have the capability to take decisions free from
direct human control. AS are increasingly being considered for adoption for applications …

Managing extreme AI risks amid rapid progress

Y Bengio, G Hinton, A Yao, D Song, P Abbeel, T Darrell… - Science, 2024 - science.org
Artificial intelligence (AI) is progressing rapidly, and companies are shifting their focus to
developing generalist AI systems that can autonomously act and pursue goals. Increases in …