IWASS will hold a special session at ESREL 2026
We look forward to your participation!
ESREL is the key annual event for meetings and knowledge exchange in the area of risk assessment, risk management, and optimization of the performance of socio-technological systems in Europe, and among the most important internationally.
Join us at the Braga 2026 conference!
Special Paper Session SS-18: Safety, Reliability, and Security of Autonomous Systems
Tuesday June 16, 11:00-11:45 AM - Meeting Room 5 - Level 1
For more details, please visit: ESREL Venue Information
Motivation: Autonomous systems are increasingly being deployed in safety-critical domains such as transportation, manufacturing, and agriculture. Their potential to operate with minimal human intervention promises transformative benefits, including improved efficiency, scalability, and adaptability. At the same time, these systems must function in uncertain and dynamic environments, often make real-time decisions, and may be required to interact and collaborate with humans or other agents. These are some of the characteristics that introduce unique challenges for risk assessment, safety assurance, and reliability engineering of autonomous systems. Traditional methods frequently fall short when applied to autonomous systems; for example, difficulties arise in modeling emergent behavior in learning-enabled systems, validating performance across diverse and dynamic operating conditions, or ensuring that a system can reliably fall back to a safe state. Importantly, such challenges are not limited to a single sector. Across transportation, maritime, aerospace, agriculture, and industrial automation, such common challenges emerge.
Objective: This special session focuses on the shared challenges of risk assessment, safety assurance, and reliability engineering across autonomous system applications and disciplines. It aims to bring together experts from academia, industry, and government to exchange perspectives, share methods for risk modeling, reliability engineering, and assurance frameworks, and identify techniques that can advance the successful deployment of autonomous systems. Key Topics of Discussion include, but are not limited to:
Risk assessment and modeling methods
Safety assurance frameworks and certification approaches
Reliability engineering, robustness, and fault tolerance
Risk and safety monitoring and safe fallback strategies
Domain case studies, lessons learned, and best practices
Ethical, legal, and regulatory frameworks for autonomy
Please contact:
Joachim Grimstad (joachim.grimstad@ias.uni-stuttgart.de),
Camila Correa-Jullian (ccorreaj@ucla.edu), or
Paul Lee (paul.lee@ntnu.no)
for more information.
Session Organizers
Thieme, Christoph A., christoph.thieme@ntnu.no, SINTEF
Ramos, Marilia, marilia@risksciences.ucla.edu, University of California Los Angeles
Utne, Ingrid B., ingrid.b.utne@ntnu.no, Norwegian University of Science and Technology
Mosleh, Ali, mosleh@ucla.edu, University of California Los Angeles
Morozov, Andrey, andrey.morozov@ias.uni-stuttgart.de, University of Stuttgart
Paper Presentations
RISK-AWARE OBSTACLE AVOIDANCE ALGORITHM FOR REAL-TIME APPLICATIONS
Authors: Kaya, Ozan; Gezer, Emir Cem; Skjetne, Roger; Utne, Ingrid Bouwer;
Department of Marine Technology, Norwegian University of Science and Technology
Abstract: Robust navigation in changing maritime environments requires autonomous systems capable of perceiving, reasoning, and acting under uncertainty. This study introduces a hybrid risk-aware navigation architecture that combines probabilistic environmental modeling with smooth trajectory optimization for autonomous surface vessels. The system constructs probabilistic risk maps that capture both obstacle proximity and the behavior of dynamic objects. A risk-biased Rapidly Exploring Random Tree (RRT) planner leverages these maps to generate collision-free paths, which are subsequently refined using B-spline algorithms to ensure trajectory continuity. Three distinct RRT* rewiring modes are implemented based on the cost function: minimizing the path length, minimizing risk, and optimizing a combination of the path length and total risk. The framework is evaluated in experimental scenarios containing both static and dynamic obstacles. The results demonstrate the system's ability to navigate safely, maintain smooth trajectories, and dynamically adapt to changing environmental risks. Compared with conventional LIDAR or vision-only navigation approaches, the proposed method shows improvements in operational safety and autonomy, establishing it as a promising solution for risk-aware autonomous vehicle missions in uncertain and dynamic environments. Future work will focus on applying the proposed risk-aware obstacle avoidance algorithm for real-time autonomous surface vessel operations.
Keywords: Obstacle Avoidance, Path Planning, Risk Modelling, Marine Autonomous Systems
MISSION PLANNING FOR AUTONOMOUS SYSTEMS INTEGRATING SOCIO-TECHNICALECOLOGICAL RISKS
Authors: Utne, Ingrid Bouwer; Johansen, Tor Arne; Johnsen, Geir; Ludvigsen, Martin; Pizarro, Oscar; Ødegård, Øyvind;
Department of Marine Technology, Norwegian University of Science and Technology (NTNU); Department of Engineering Cybernetics, NTNU; Department of Biology, NTNU; NTNU University Museum
Abstract: The ocean remains one of the most under-sampled environments on Earth due to its vastness and opacity to remote sensing. Traditional ship-based and in-situ observation methods are costly, infrequent, and heavily reliant on human intervention. Hence, there is a critical mismatch between societal demands for ocean knowledge and our current observational capabilities. To bridge this gap, there is an urgent need for robotic organizations composed of heterogeneous sensor platforms capable of real-time, adaptive, and intelligent data collection—operating safely, resiliently, and efficiently without direct human presence. The individual advancements in robotic platforms, sensors, and remote sensing assets, such as satellites, are promising. Achieving true autonomy and high-quality observations in the unstructured ocean environment, however, require integrated mission capabilities that transcend the limitations of single systems. Such integration increases mission complexity and introduces trade-offs between operational risks and data quality. Key challenges include enabling robotic organizations to collaborate effectively and achieving extreme cooperation and resilience across systems-of-systems, essential for long-endurance missions in remote and harsh ocean environments. This necessitates embedding domainspecific knowledge about mission risks related to marine ecosystems, the human legacy, and ocean infrastructure integrity, at unprecedented spatial and temporal scales. This knowledge must be integrated into autonomous reasoning, artificial intelligence (AI), and decision-making processes, which is a major unresolved research challenge. The objective of this paper is to explore relevant socio-technical-ecological risks associated with deploying robotic organizations in the ocean. The paper proposes a methodological approach towards mission planning for risk-aware autonomy. The approach emphasizes the need for real-time risk models operating at the edge and in the cloud, i.e., supervisory risk control, capable of interacting with autonomous systems—despite the constraints of limited underwater communications. These developments are decisive for achieving scalable, intelligent, and resilient autonomous robotic organizations and operations in the ocean.
Keywords: Mission planning, Supervisory risk control, Artificial intelligence, Risk assessment, Autonomy
USING LARGE LANGUAGE MODELS (LLMS) TO COMPLEMENT EXPERT-BASED HAZARDIDENTIFICATION UNDER LIMITED OPERATIONAL DATA FOR REMOTELY PILOTED SHIPS
Authors: Yeganeh Khaksar, Reza; Khorasani, Mahsa; Farokhi, Raheleh; A. Valdez Banda, Osiris;
Department of Energy and Mechanical Engineering, Marine and Arctic Technology, Research Group on Safe and Efficient Marine and Ship Systems, Aalto University, Espoo, Finland; Kotka Maritime Research Centre, Kotka, Finland
Abstract: Remotely piloted ships are still under development, and so, real operational experience and real-world data from their utilization are not yet available. This limited real-world evidence makes it challenging to comprehensively explore potential hazards during the early stages of design. However, a substantial amount of collected data already exists through previous expert intuition, questionnaires and workshops with maritime pilots. Recently, synthetic data generation using artificial intelligence (AI) generative models has emerged to deal with problem of real-world data scarcity in various studies. This study investigates the potential of Large Language Models (LLMs) as a source of synthetic data to complement existing expert-based hazard identification processes. In this feasibility study, structured prompts were designed to extract possible hazards related to communication, remote control, and human–machine interaction. The AIgenerated hazard lists were then qualitatively compared with the hazards identified through expert workshops to highlight differences and potential synergies. Preliminary observations suggest that LLM-generated outputs can complement existing human-derived insights by expanding the diversity of identified hazard scenarios. Nevertheless, careful consideration is needed when interpreting and integrating AI-generated content to ensure consistency, occurrence likelihood, feasibility, and reliability. The study aims to motivate discussion on how generative AI can responsibly contribute to systematic hazard identification in the development of remote pilotage services.
Keywords: Remote Pilotage, Large Language Models, Hazard Identification, Expert Review, Maritime Design, AI Application
MANAGING CHANGE IN MARITIME AI: A SEMANTIC AND MODULAR FRAMEWORK FORADAPTIVE MARITIME SYSTEMS
Authors: Knutsen, Knut Erik; Karandikar, Nikita; Wang, Shuai;
DNV
Abstract: Artificial intelligence (AI) is transforming maritime operations with increasingly autonomous functions. These systems must adapt continuously to changing environmental conditions, new technologies, diverse fleets, and evolving regulations to maintain reliability, safety, and compliance—critical factors in the maritime industry. This paper presents a semantic and modular framework designed to manage such change through interconnected layers. The base layer employs a maritime knowledge graph based on ISO 19848 and Vessel Information Structure (VIS) to integrate operational data from sensors and navigation systems. Above this, a layer of modular contracts wraps AI models and system modules, allowing easy updates and verification as conditions evolve. Modular contracts represent formal agreements specifying requirements, acceptance criteria, and interfaces for modules, such as an autonomous navigation system. These contracts define module interactions, enabling reuse across ships and simplifying updates and verification. A federated learning layer supports secure AI training and updates distributed across multiple vessels. An explainable AI layer provides operator insights to foster transparency and trust. The top layer applies dynamic risk management and compliance checks aligned with regulations like IMO MSC.428 and EMSA CISE. An illustrative example of ship propulsion fault detection shows how these layers interoperate—from data integration and modular design to evolving AI, human oversight, and risk management,offering a flexible approach to managing change in maritime AI systems. This framework provides an adaptable and robust foundation for AI solutions that must remain effective and safe in the ever-changing maritime environment.
Keywords: Explainable AI, Knowledge Graph, Assurance, Reliability
A FRAMEWORK FOR TESTING, VERIFICATION, AND VALIDATION OF MARITIME AUTONOMOUS SURFACE SHIPS
Authors: Gorska, Kaja; Theotokatos, Gerasimos; Boulougouris, Evangelos;
University of Strathclyde
Abstract: Maritime Autonomous Surface Ships (MASS) are expected to improve safety, efficiency, and sustainability in the maritime industry. However, their complex nature poses significant challenges to traditional assurance practices designed for conventional vessels. While current research on MASS predominantly focuses on the technological development of individual subsystem, studies dedicated to systematic testing, verification, and validation (TVV) methods remain underrepresented. Specifically, there is a lack of comprehensive TVV framework tailored to MASS. This study aims to propose a structured TVV framework supporting the assurance of MASS. This study conducts a comprehensive review of state-of-the-art and advanced TVV methods used in both the maritime domain and other industries involving autonomous systems, such as automotive and aerospace. The reviewed approaches include traditional verification and validation methods, techniques for machine-learning-based systems, simulations, as well as lab- and full-scale testing. These methods are mapped to the subsystems of a reference MASS architecture to identify coverage, overlaps, and existing gaps. Based on this mapping, a conceptual TVV framework is proposed to support an integrated assurance process for MASS, which serves as a foundational step towards developing a standardised, systemlevel approach to the assurance of autonomous maritime systems, promoting safer and more reliable deployment of MASS technologies.
Keywords: Maritime Autonomous Surface Ships, Testing, Verification, and Validation, Assurance Framework
TOWARDS SAFE DEPLOYMENT OF AUTONOMOUS VEHICLES: RESPONSIBILITIES, HAZARDS, AND METRICS IDENTIFICATION BASED ON STPA METHODOLOGY
Authors: Cheng, Tingting; Correa-Jullian, Camila; Singh, Siddhant; Mosleh, Ali; Ma, Jiaqi;
University of California Los Angeles
Abstract: The autonomous vehicle (AV) industry has advanced substantially since the California Department of Motor Vehicles (DMV) launched its AV testing program in 2012. As developers expand towards commercial passenger services, key methodological and practical challenges remain in systematically assessing and monitoring the safety performance of AV fleets. In the U.S., AV developers primarily report limited incident metrics and offer little insight into how risks may evolve as AV fleets scale. This limits the ability of state and local regulators to make informed decisions about AV integration within complex urban traffic systems. Therefore, developing safety-centric tools to support jurisdictions in monitoring and assessing the safety performance of AV fleets is key. This work proposes System-Theoretic Process Analysis (STPA) and semi-quantitative risk scales to identify, model, and rank safety-critical interactions between AV fleets and other road users to elicit more comprehensive operational safety metrics. We apply STPA to examine AV fleet functions across deployment stages and capture interactions with human agents, identifying hazards that arise from unsafe control actions (UCAs) capable of leading to system-level losses such as increased crash risk or delayed emergency response. The STPA process includes defining system-level hazards and constructing a functional control structure that maps interacting entities (e.g., AVs, human-driven vehicles, remote operators), their responsibilities (preventive and mitigative controls), the control actions they exchange (e.g., remote interventions), and the associated feedback mechanisms (e.g., environmental data shared with operators). UCAs are identified by analyzing these control actions in relevant operational contexts and grounding them in available operational and incident data. Finally, we develop causal scenarios for the identified UCAs, which directly inform measurable, risk-driven safety performance metrics. Outputs include a functional control and interaction model of a generic AV fleet, a structured catalog of UCAs and associated hazard scenarios, and derived metrics linked to system safety objectives.
Keywords: AV Fleet Deployment, System-Theoretic Process Analysis, Safety Metric Derivation
PERSPECTIVES ON RISK ANALYSIS FRAMEWORKS FOR AUTONOMOUS VEHICLES: STATE OF THE ART AND OPEN CHALLENGES
Authors: Doehring, Christopher; Guikema, Seth
University of Michigan
Abstract: Autonomous vehicles (AVs) present unprecedented safety and reliability challenges due to their complex integration of software, hardware, and real-time decision-making in open road environments. Ensuring that AVs operate safely under all conditions requires rigorous risk analysis frameworks to identify potential hazards, assess risks, and implement mitigation measures before deployment. Over the past several years, a variety of frameworks have emerged from government agencies and standards bodies, industry consortiums and companies, and academic research groups. Each framework contributes methodologies for analyzing and reducing the risks associated with AV technology. However, these approaches differ in scope and emphasis from high-level safety assurance processes to technical tools for hazard identification and significant gaps remain in achieving consensus on best practices. This perspectives paper reviews the state of the art in risk analysis frameworks for autonomous vehicles, drawing on representative examples from government and standards organizations, industry efforts, and academic research. Key methodological aspects common to many frameworks are highlighted, such as scenario-based testing and safety assurance cases. We also discuss open challenges that cut across all approaches, including handling the enormous scenario space of driving, ensuring coverage of “edge cases,” and dealing with the uncertainties introduced by machine learning components. The goal is to synthesize current efforts and identify where further work is needed to confidently assess and manage the risks of autonomous driving systems. The remainder of the paper is organized as follows: (1) Introduction, (2) Government and Standards Frameworks, (3) Industry and Consortium Frameworks, (4) Academic and Research Frameworks, (5) Key Methodological Aspects, (6) Gaps and Challenges, and (7) Conclusion.
Keywords: Autonomous Vehicles, AV Risk, Autonomous Vehicle Safety Policy
RISK AWARENESS– BRIDGING THE GAP BETWEEN SAFETY AND AI DOMAINS
Authors: Lee, Paul; Utne, Ingrid Bouwer; Kim, Ekaterina
Department of Marine Technology, Norwegian University of Science and Technology (NTNU)
Abstract: In recent years, significant advancements have been observed in the artificial intelligence (AI) domain with a growing adoption in various applications that necessitate certain level of intelligence. However, numerous AI-related accident cases indicate that its intelligence does not guarantee its safety, which is often overlooked in the present AI arms race. This has raised the need towards AI that is not assessed only in terms of performance-related measures, but also in terms of safety-related measures, such as risk-awareness, as a form of deeper understanding of risk. Considerable effort has been put from the AI domain to tackle this challenge, but often with a considerable difference to what is understood from the safety domain. The aim of this paper is to bridge this gap, by identifying the discrepancy between the risk awareness definition used in the AI and safety domains. First, the general risk awareness definition from the safety domain is given and relevant measures are identified. Second, the concepts of risk awareness in the AI domain are reviewed and assessed against the identified risk awareness measures. The results indicate the gaps in enhancing the risk awareness of AI and current limitations unless new methods and approaches are developed.
Keywords: Safety; Risk Awareness; Artificial Intelligence; Risk Aware AI
