Fliewatüüt meets AI: Can We Trust AI Tools in Safety-Critical Development?

Andreas Gasch • Elena-Flavia Povirnaru - CARIAD SE

The use of AI-based tools in the development and testing of safety-related automotive systems is growing—but how trustworthy/reliable are their answers when safety is on the line? In this talk, we explore the potential and pitfalls of integrating generative AI tools into safety engineering workflows, through a mix of technical inquiry and a playful safety case inspired by "Robbi, Tobbi und das Fliewatüüt."

We pose concrete safety questions to an AI tool, such as:

  • What are the safety-critical aspects of a vehicle that can fly, float, and drive? What do we have to consider/take when working with AI-based tools?
  • Can this system be certified under ISO 26262?
     

We will analyze the responses:

  • Are they technically correct? How realistic they are? Do they reflect safety expectations? What risks arise if relying only on AI-generated answers?
     

Using the imaginative "Fliewatüüt" as a metaphor, we demonstrate where AI tools are helpful—e.g., in brainstorming failure modes, generating test scenarios —and where they fall short, especially in areas requiring domain knowledge, context understanding, etc…

 

Thursday, September 25, 2.00 PM