Fliewatüüt meets AI: Can We Trust AI Tools in Safety-Critical Development?
Andreas Gasch • Elena-Flavia Povirnaru - CARIAD SEThe use of AI-based tools in the development and testing of safety-related automotive systems is growing—but how trustworthy/reliable are their answers when safety is on the line? In this talk, we explore the potential and pitfalls of integrating generative AI tools into safety engineering workflows, through a mix of technical inquiry and a playful safety case inspired by "Robbi, Tobbi und das Fliewatüüt."
We pose concrete safety questions to an AI tool, such as:
- What are the safety-critical aspects of a vehicle that can fly, float, and drive? What do we have to consider/take when working with AI-based tools?
- Can this system be certified under ISO 26262?
We will analyze the responses:
- Are they technically correct? How realistic they are? Do they reflect safety expectations? What risks arise if relying only on AI-generated answers?
Using the imaginative "Fliewatüüt" as a metaphor, we demonstrate where AI tools are helpful—e.g., in brainstorming failure modes, generating test scenarios —and where they fall short, especially in areas requiring domain knowledge, context understanding, etc…