Institution:
![Institution](https://x4hpc.cat/custom/themes/100x100/assets/images/logos/bsc.png)
Smart and Safe Autonomous Systems
Jaume Abella
Deep Learning (DL) techniques are at the heart of most future advanced software functions in Critical Autonomous AI-based Systems (CAIS), where they also represent a major competitive factor. Hence, the economic success of CAIS industries (e.g., automotive, space, railway) depends on their ability to design, implement, qualify, and certify DL-based software products under bounded effort/cost. However, there is a fundamental gap between Functional Safety (FUSA) requirements on CAIS and the nature of DL solutions. This gap stems from the development process of DL libraries and affects high-level concepts such as (1) explainability and traceability, (2) suitability for varying safety requirements, (3) FUSA-compliant implementations, and (4) real-time constraints. As a matter of fact, the data-dependent and stochastic nature of DL algorithms clashes with current FUSA practice, which instead builds on deterministic, verifiable, and pass/fail test-based software.
The safe use of AI in critical systems such as cars, trains and satellites
The SAFEXPLAIN project tackles the challenges by providing a novel and flexible approach to allow the certification – hence adoption – of DL-based solutions in CAIS building on (1) DL solutions that provide end-to-end traceability, with specific approaches to explain whether predictions can be trusted and strategies to reach (and prove) correct operation, in accordance to certification standards; (2) alternative and increasingly sophisticated design safety patterns for DL with varying requirements of criticality and fault tolerance; (3) DL library implementations that adhere to safety requirements; and (4) computing platform configurations, to regain determinism, and probabilistic timing analyses, to handle the remaining non-determinism.
Any system with safety requirements (transportation, industrial, medical, etc.).
Unleash the potential of AI for autonomous operation (e.g., autonomous driving) while preserving safety.
Some open-source libraries (with permissive libraries), and some software with commercial licenses. Software still being developed (early stages), so no explicit protection yet.
Automotive, space, railway, avionics, industries.
TRL: 4
CRL: N/A
BRL: N/A
IPRL: N/A
TmRL: N/A
FRL: N/A
Impacted SDGs:
N/A
if you want to know more about this project do not hesitate to contact us