The research objectives and expected outcome

The project deals with the approval of machine learning (ML) technology for systems intended for use in safety-related applications in all domains covered by the EASA Basic Regulation (Regulation (EU) 2018/1139).

Data-driven learning techniques are a major opportunity for the aviation industry but come also with a significant number of challenges with respect to the trustworthiness of ML and deep learning (DL) solutions.

EASA published its Artificial Intelligence Roadmap in February 2020, followed by a first major deliverable, a Concept Paper ‘First usable guidance for level 1 machine learning applications’ in December 2021. This concept paper lays down the basis of EASA future guidance for ML applications approval thanks to a W-shaped process and identifies a number of areas in which further research is necessary to identify efficient and practicable means of compliance with the defined ‘AI trustworthiness’ objectives.

MLEAP project is a research project initiated by EASA and funded under the Horizon Europe framework. MLEAP has been tailored to investigate the challenging objectives of the W-shaped process at the core of EASA AI Concept Paper.

The intended short-term effect of this project will be to streamline the certification and approval processes by identifying concrete means of compliance with the learning assurance objectives of the EASA guidance for ML applications (levels 1, 2 and 3 as defined in the EASA AI Roadmap), with a specific focus on Level 1B and Level 2.

The achieved medium-term effect of the project will be to alleviate some remaining limitations on the acceptance of ML applications in safety-critical applications.

The requested output

The research results will be a set of reports identifying a set of methods and tools to address the following three important topics:
•    Guarantees on ‘machine learning model generalisation’
•    Guarantees on ‘Data completeness and representativeness’
•    Guarantees on algorithm and model robustness

The reports will highlight  a set of anticipated concepts for the evaluation and certification of AI-based systems supporting the EASA roadmap deliverables, and help industry stakeholders in planning new strategies for deploying AI in their human and technical organisations.

Along with the project, three aviation use-cases will be developed to demonstrate the effectivity and usability of the proposed methods and tools. They will be   developed to demonstrate the effectivity and usability of the proposed methods and tools. Those use cases should be developed in a software and hardware environment, accessible remotely by EASA or through software package deliveries to EASA. The essential life cycle artefacts developed in the project to address the different steps of the W-shaped process should be made available to EASA.

The work break-down structure of this project is the following:
•    Task 1: Methods and tools for the assessment of completeness and representativeness of data sets (training, validation, and test) in data-driven ML and DL
•    Task 2: Methods and tools for quantification of generalisation guarantees for ML and DL models
•    Task 3: Methods and tools for the verification of an ML algorithm and model robustness/stability
•    Task 4: Communication, dissemination, knowledge-sharing, stakeholder management
•    Task 5: Project management
MLEAP timeframe is established to support key EASA AI programme milestones as depicted in figure below:

First MLEAP public deliverable

Halfway through the project, the MLEAP consortium published its interim report on Thursday 11 May 2023. This 258-page report provides a detailed state of the art for the three tasks of interest of MLEAP, a selection of the most promising methods and tools to be explored, and a description of the aerospace use cases that will be used to evaluate these selected methods and tools. Preliminary results of this interim report have already been used to enrich EASA AI concept paper issue 2 currently under public consultation.

One of the main results is a first version of a general framework to cover most of the ML-related steps the W-shaped process of the EASA AI Concept paper.

This process will be further refined in the next phases of the project based on the first experimentation  and the interim report findings related to the data evaluation and preparation (Chapter 3) and considering recommendation about robustness and performance stability evaluation (Chapter 5).  

MLEAP project will now proceed with evaluating the selected methods and tools to the three aeronautical AI use cases (Chapter 2) retained for the project:

• Speech to text in Air Traffic Control communication
• Drone collision avoidance (ACAS-XU)
• Vision-based maintenance inspection

The results for the EASA - Collins Aerospace ”Formula” IPC report published in April 2023, related to the identification of formal methods approaches as efficient means to meet certification objectives from the EASA AI concept paper, will also be considered in the next steps of the project, in particular for MLEAP Task 3.

This interim report will be presented and discussed during  public events in May and June (see Next MLEAP public events Section below).

MLEAP Stakeholders day 2022 – EASA AI Roadmap

On Thursday 24 November 2022, the first dissemination event of the MLEAP project, which, was held at EASA.   The aim of this "Stakeholders day 2022" was to present in detail the objectives of the project to the various parties (aircraft and ATM/ANS system designers, aircraft manufacturers, certification specialists, authorities, standardisation bodies) directly interested in the results of this research project.

Around 60 participants with different backgrounds attended this hybrid event. The EASA AI team briefed the audience with its vision, and how the MLEAP project will deliver on the AI roadmap, before the MLEAP consortium presented the project workplan, the objectives of each technical task and the first results achieved. Stakeholders' reactions and questions were collected in a Q&A session and will serve as an input in future phases of the project.

The meeting material is available under 'Related Content'.

Next MLEAP public events

The next MLEAP public event will take place at EASA headquarters in Cologne, Germany, on 17 May 2023 during EASA Artificial Intelligence Days. This will be an opportunity to present and answer questions of interested parties on the first public deliverables of the MLEAP project to be published in May 2023 on this page. The agenda and registration information can be found on the EASA AI days 2023 event page.

EASA and MLEAP consortium will also give a public conference on 21 June 2023 at the Paris Airshow 2023:

• Awareness session Conference: June 21st from 10 to 11am - at Airbus Pavillon – Invitation only – see dedicated page.
• Knowledge Sharing Conference & Networking: June 21st from 3 to 5pm - at VIPARIS Conference center - Conference room N°2 - This conference is virtual and on-site.

The agenda and registration information for Paris Airshow can be found on this dedicated page.