The public defence of the thesis takes place on Friday November 15th, 2024 at 09:15.

Thesis title: Critical Scenario Identification for Testing of Realistic Autonomous Driving Systems

Author: Qunying Song, Department of Computer Science, Lund University

Faculty opponent: Professor Dietmar Pfahl, University of Tartu, Estonia

Examination Committee:

• Professor Martin Törngren, Royal Institute of Technology
• Professor Christian Berger, University of Gothenburg
• Doctor Efi Papatheocharous, RISE
• Deputy: Senior Lecturer Aliaksei Laureshyn, Lund University

Session chair: Professor Björn Regnell, Lund University

Supervisors:

• Professor Per Runeson, Lund University
• Associate Professor Emelie Engström, Lund University

Location: E:1406, E-building, Klas Anshelms väg 10 / John Ericssons väg 2, Lund, Sweden

Zoom link: https://lu-se.zoom.us/j/61663216784

For download: Follow this link to download the pdf: https://portal.research.lu.se/sv/publications/critical-scenario-identification-for-testing-of-realistic-autonom

Abstract

Background: Testing is imperative to validate the functionalities and safety of autonomous driving systems. Simulated scenario-based testing is commonly adopted for autonomous driving systems, which aims to construct various driving scenarios and validate the autonomous driving systems in simulation. Nevertheless, identifying relevant test scenarios, especially critical ones that expose hazards or risks of harm to autonomous vehicles remains an open challenge.

Objective: We focus on critical scenario identification for testing of realistic autonomous driving systems in this thesis. Specifically, our objective is to establish an effective approach for identifying critical scenarios for testing of industrial autonomous driving systems. Also, we aim to explore and improve current practices of testing autonomous driving systems with critical scenarios in industry.

Methodology: We follow the design science research paradigm and perform two iterations of the design research cycle in this thesis. The first iteration focuses on the first objective of the thesis to design an approach for critical scenario identification. In the second iteration, we explore industry practices of using critical scenarios for testing of autonomous driving systems, and propose a preliminary solution to evaluate the realism of such scenarios and improve their validity.

Contributions: The thesis comprises five studies and our contributions are summarized into four folds. (1) Taxonomies of concepts, challenges, practices, techniques, and approaches for testing autonomous systems. (2) A workbench with interconnected tools and a workflow for end-to-end critical scenario identification, which is demonstrated on two industrial autonomous driving systems. (3) Taxonomies of industry practices, initiatives, and challenges for identifying and using critical scenarios for testing autonomous driving systems. (4) A methodology to evaluate the realism of synthetic critical scenarios and a proof of concept of it on two autonomous vehicle collision scenario sets.

Conclusion: We explore the industry practices of and design an effective approach for critical scenario identification as well as a methodology to evaluate the realism of the resulting scenarios. The current practices of critical scenario identification are limited and hindered by several challenges such as cost, tool support, and scenario realism. Therefore, we recommend the industry and academia combine existing approaches, collaborate, and continuously learn to improve their practical use for testing of autonomous driving systems.