AMC 20-189 The Management of Open Problem Reports (OPRs)
ED Decision 2020/010/R
1. PURPOSE
This AMC describes an acceptable means, but not the only means, for showing compliance with the applicable airworthiness regulations for the management of open problem reports (OPRs) in ETSO authorisations and type certification, for the system, software and airborne electronic hardware (AEH) domains. Compliance with this AMC is not mandatory, and an applicant may elect to use an alternative means of compliance. However, the alternative means of compliance must meet the relevant requirements, ensure an equivalent level of safety, and be approved by EASA on a product or ETSO article basis.
2. APPLICABILITY
This AMC may be used by applicants, design approval holders, and developers of airborne systems and equipment to be installed on type-certified aircraft, engines, and propellers. This AMC applies to all airborne electronic systems and equipment, including to the software and AEH components contained in those systems, which could cause or contribute to Catastrophic, Hazardous, or Major failure conditions.
3. BACKGROUND
3.1. Each of the system, software and AEH domains relies on problem report (PR) management to ensure the proper management of open problem reports (OPRs) and to help ensure safe products at the time of approval. However, the existing guidance on PR and OPR management is inconsistent and incomplete across domains. Therefore, this AMC provides consistent guidance across these domains for PR management, OPR management, stakeholder responsibilities, reporting, and other aspects of OPR management. This AMC complements but does not alleviate the project-applicable system, software and AEH guidance.
3.2. The technical content of this AMC has been jointly developed with the Federal Aviation Administration (FAA), in order to harmonise as far as practicable.
4. DEFINITIONS
4.1. Terms defined in this AMC
Approval: the term ‘approval’ in this document addresses the approval by EASA of a product or of changes to a product, or authorisation of an ETSO article or of changes to an ETSO article.
Article: refer to Article 1(2)(f) of Regulation (EU) No 748/2012 (‘EASA Part 21’).
Development assurance: all of those planned and systematic actions used to substantiate, with an adequate level of confidence, that errors in requirements, design, and implementation have been identified and corrected such that the system satisfies the applicable certification basis (source: ARP4754A/ED-79A).
Equipment: an item or collection of items with a defined set of requirements.
Error: a mistake in the requirements, design, or implementation with the potential of producing a failure.
Failure: the inability of a system or system component to perform a function within specified limits (source: DO-178C/ED-12C and DO-254/ED-80).
Item: a hardware or software element that has bounded and well-defined interfaces (source: ARP4754A/ED-79A).
Open problem report (OPR): a problem report that has not reached the state ‘Closed’ at the time of approval.
Problem report (PR): a means to identify and record the resolution of anomalous behaviour, process non-compliance with development assurance plans and standards, and deficiencies in life-cycle data (adapted from DO-178C/ED-12C).
Product: refer to Article 3(3) of Regulation (EU) 2018/1139 (the ‘EASA Basic Regulation’).
System: a combination of interrelated equipment, article(s), and/or items arranged to perform a specific function (or functions) within a product.
4.2. States of PRs/OPRs
Recorded: a problem that has been documented using the problem-reporting process.
Classified: a problem report that has been categorised in accordance with an established classification scheme.
Resolved: a problem report that has been corrected or fully mitigated, for which the resolution of the problem has been verified but not formally reviewed and confirmed.
Closed: a resolved problem report that underwent a formal review and confirmation of the effective resolution of the problem.
4.3. Classifications of PRs/OPRs
‘Significant’: assessed at the product, system, or equipment level, a PR that has an actual or potential effect on the product, system, or equipment function that may lead to a Catastrophic, Hazardous or Major failure condition, or may affect compliance with the operating rules.
‘Functional’: a PR that has an actual or a potential effect on a function at the product, system, or equipment level.
‘Process’: a PR that records a process non-compliance or deficiency that cannot result in a potential safety, nor a potential functional, effect.
‘Life-cycle data’: a PR that is linked to a deficiency in a life-cycle data item but not linked to a process non-compliance or process deficiency.
5. PROBLEM REPORT MANAGEMENT
The PR management process is a key enabler for the management of OPRs. The PR management process enables the consistent and timely management of problems encountered across the system, software and AEH domains. Consequently, this process reduces the risk of a loss of visibility of critical issues remaining at the time of approval.
5.1 A PR management process across the system, software and AEH domains should be established and used during the development (both for initial certification and subsequent changes) of a product or an ETSO article. The PR management process should address the review and resolution of PRs that impact the transition to other development assurance processes.
5.2 A problem recorded after approval should also be managed through the PR management process, and any related systemic process issues should be identified and corrected.
5.3 PRs that cannot be resolved by the current stakeholder should be reported in a manner that is understandable to the affected stakeholders.
5.4 For PRs that may have an impact on other products or articles that are developed within an organisation, a means should be established for sharing PR information so that any necessary corrective actions can be taken.
6. OPR MANAGEMENT
An OPR management process, based on the PR management process, should be established across the system, software and AEH domains, including the following process steps:
6.1 The classification of OPRs
6.1.1 The applicant should establish an OPR classification scheme including, at a minimum, the following classifications: ‘Significant’, ‘Functional’, ‘Process’ and ‘Life-cycle data’. Other classifications or subclassifications may be created as needed. The classification scheme should be described in the appropriate planning document(s).
6.1.2 Each OPR should be assigned a single classification per the classification scheme. When multiple classifications apply, the OPR should be assigned the classification with the highest priority. The priority from highest to lowest (including the defined subclassifications) is:
1. ‘Significant’;
2. ‘Functional’;
3. ‘Process’;
4. ‘Life-cycle data’;
5. any other OPR classification.
Note: The classification of an individual OPR may differ from one stakeholder to another, depending on the known mitigations at the time of classification.
6.1.3 The classification of an OPR should account for and document all the mitigations known at the time of classification that are under the control of the classifying stakeholder. A mitigation that is controlled by another stakeholder may be considered in the classification only if validated with that stakeholder, and provided this mitigation remains acceptable in the frame of the type certificate (TC) / supplemental type certificate (STC) approval or European technical standard order (ETSO) article authorisation, as applicable.
6.1.4 A stakeholder, other than the aircraft TC or STC applicant, should classify as ‘Significant’ any OPR for which the classification may vary between ‘Functional’ and ‘Significant’, depending on the installation.
6.2 The assessment of OPRs
Each OPR should be assessed to determine:
1. any resulting functional limitations or operational restrictions at the equipment level (for ETSOs) or at the product level (for other types of approvals);
2. relationships that may exist with other OPRs; and
3. for a ‘Significant’ or ‘Functional’ OPR, the underlying technical cause of the problem.
6.3 Disposition: OPRs classified as ‘Significant’ per the classification in Section 6.1, for which no sufficient mitigation or justification exists to substantiate the acceptability of the safety effect, should be resolved prior to approval. The disposition of OPRs may involve coordination with the certification authority.
6.4 Reporting: an OPR summary report should be prepared and provided to the affected stakeholder(s), and to the certification authority upon request. The OPR summary report may be an aggregation of summaries (e.g. Software/Hardware Accomplishment Summaries or system-level OPR reports) prepared by all the involved stakeholders. The summary report should provide access to the following information for each OPR:
6.4.1 The identification of the OPR (for example, the OPR ID);
6.4.2 The identification of the affected configuration item(s) (for example, the item part number, component name, artefact name) or of the affected process(es);
6.4.3 Title or a summary of the problem, formulated in a manner understandable by the affected stakeholder(s);
6.4.4 Description of the problem, formulated in a manner understandable by the affected stakeholder(s);
6.4.5 The conditions under which the problem occurs;
6.4.6 The OPR classification and assessment results (per Sections 6.1 and 6.2), including:
1. for each OPR, regardless of its classification:
a. the classification of the OPR, and
b. the relationships that are known to exist with other OPRs;
2. for OPRs classified as ‘Significant’:
a. a description of any mitigations or justifications used to substantiate the acceptability of the OPR safety effect (per Section 6.3), and
b. the functional limitations and operational restrictions, if any;
3. for OPRs classified as ‘Functional’:
a. a description of any mitigations or justifications used to reduce the safety effect to Minor or No Safety Effect, and
b. the functional limitations and operational restrictions, if any;
4. for OPRs classified as ‘Process’, a description of the extent or nature of the process non-compliance or deficiency that might contribute to not satisfying the applicable development assurance objectives; and
5. for each OPR not classified as ‘Significant’ or ‘Functional’, the justification that the error cannot have a safety or functional effect.
6.5 ETSO specifics: The ETSO authorisation holder may exclude from the reporting process (per Section 6.4) any OPRs classified as ‘Process’ or ‘Life-cycle data’ that are not necessary for the installation approval. However, all OPRs should be available upon request by the certification authority for assessment in the frame of the ETSO approval.
7. STAKEHOLDER RESPONSIBILITIES
The levels of stakeholders include: item, equipment or ETSO article, system and product. The actual stakeholders for a specific project depend on the project organisation.
7.1 The levels of stakeholders include: item, equipment or ETSO article, system and product. The actual stakeholders for a specific project depend on the project organisation.
7.2 OPR management (per Section 6) should be performed, at a minimum, at the ETSO article level, at the level of each individual system within a product, and at the product level.
8. RELATED REGULATORY, ADVISORY AND INDUSTRY MATERIAL
(a) Related EASA Certification Specifications (CSs)
(1) CS-23, Certification Specifications and Acceptable Means of Compliance for Normal Category Aeroplanes
(2) CS-25, Certification Specifications and Acceptable Means of Compliance for Large Aeroplanes
(3) CS-27, Certification Specifications and Acceptable Means of Compliance for Small Rotorcraft
(4) CS-29, Certification Specifications and Acceptable Means of Compliance for Large Rotorcraft
(5) CS-E, Certification Specifications and Acceptable Means of Compliance for Engines, and AMC 20-3A, Certification of Engines Equipped with Electronic Engine Control Systems
(6) CS-P, Certification Specifications for Propellers, and AMC 20-1, Certification of Aircraft Propulsion Systems Equipped with Electronic Control Systems
(7) CS-ETSO, Certification Specifications for European Technical Standard Orders
(8) CS-APU, Certification Specifications for Auxiliary Power Units, and AMC 20-2A, Certification of Essential APU Equipped with Electronic Controls
(b) EASA Acceptable Means of Compliance (AMC)
(1) AMC 20-115( ), Airborne Software Development Assurance Using EUROCAE ED-12 and RTCA DO-178
(2) AMC 20-152( ), Development Assurance for Airborne Electronic Hardware
(c) FAA ACs
Refer to latest version.
(1) AC 20-115( ), Airborne Software Development Assurance Using EUROCAE ED-12( ) and RTCA DO-178( )
(2) AC 20-152( ), Development Assurance for Airborne Electronic Hardware
(3) AC 27-1309( ), Equipment, Systems, and Installations (included in AC 27-1, Certification of Normal Category Rotorcraft)
(4) AC 29-1309( ), Equipment, Systems, and Installations (included in AC 29-2, Certification of Transport Category Rotorcraft)
(d) Industry Documents
(1) EUROCAE ED-12, Software Considerations in Airborne Systems and Equipment Certification, dated May 1982 (no longer in print)
(2) EUROCAE ED-12A, Software Considerations in Airborne Systems and Equipment Certification, dated October 1985 (no longer in print)
(3) EUROCAE ED-12B, Software Considerations in Airborne Systems and Equipment Certification, dated December 1992
(4) EUROCAE ED-12C, Software Considerations in Airborne Systems and Equipment Certification, dated January 2012
(5) EUROCAE ED-79A, Guidelines for Development of Civil Aircraft and Systems, dated December 2010
(6) EUROCAE ED-80, Design Assurance Guidance for Airborne Electronic Hardware, dated April 2000
(7) EUROCAE ED-94C, Supporting Information for ED-12C and ED-109A, dated January 2012
(8) EUROCAE ED-215, Software Tool Qualification Considerations, dated January 2012
(9) EUROCAE ED-216, Formal Methods Supplement to ED-12C and ED-109A, dated January 2012
(10) EUROCAE ED-217, Object-Oriented Technology and Related Techniques Supplement to ED-12C and ED-109A, dated January 2012
(11) EUROCAE ED-218, Model-Based Development and Verification Supplement to ED-12C and ED-109A, dated January 2012
(12) RTCA DO-178, Software Considerations in Airborne Systems and Equipment Certification, dated January 1982 (no longer in print)
(13) RTCA DO-178A, Software Considerations in Airborne Systems and Equipment Certification, dated March 1985 (no longer in print)
(14) RTCA DO-178B, Software Considerations in Airborne Systems and Equipment Certification, dated 1 December 1992
(15) RTCA DO-178C, Software Considerations in Airborne Systems and Equipment Certification, dated 13 December 2011
(16) RTCA DO-248C, Supporting Information for DO-178C and DO-278A, dated 13 December 2011
(17) RTCA DO-254, Design Assurance Guidance for Airborne Electronic Hardware, dated April 19, 2000
(18) RTCA DO-297, Integrated Modular Avionics (IMA) Development Guidance and Certification Considerations, dated 8 November 2005
(19) RTCA DO-330, Software Tool Qualification Considerations, dated 13 December 2011
(20) RTCA DO-331, Model-Based Development and Verification Supplement to DO-178C and DO-278A, dated 13 December 2011
(21) RTCA DO-332, Object-Oriented Technology and Related Techniques Supplement to DO178C and DO-278A, dated 13 December 2011
(22) RTCA DO-333, Formal Methods Supplement to DO-178C and DO-278A, dated 13 December 2011
(23) SAE International Aerospace Recommended Practice (ARP) 4754A, Guidelines for Development of Civil Aircraft and Systems
9. AVAILABILITY OF DOCUMENTS
(1) EASA Certification Specifications (CSs) and Acceptable Means of Compliance (AMC) may be downloaded from the EASA website: www.easa.europa.eu
(2) FAA Advisory Circulars (ACs) may be downloaded from the FAA website: www.faa.gov
(3) EUROCAE documents may be purchased from:
European Organisation for Civil Aviation Equipment
9-23 rue Paul Lafargue
"Le Triangle" building
93200 Saint-Denis, France
Telephone: +33 1 49 46 19 65
(Email: eurocae [at] eurocae.net, website: www.eurocae.net)
(4) RTCA documents may be purchased from:
RTCA, Inc.1150 18th Street NW, Suite 910, Washington DC 20036, USA
(Email: info [at] rtca.org, website: www.rtca.org)
10. GUIDANCE MATERIAL
GM 20-189 The Management of Open Problem Reports
GM1 to AMC 20-189 — PR management
ED Decision 2020/010/R
Typically, PR processes include the following aspects:
1. PR Recording: a means to document problems resulting from the execution of life-cycle processes.
2. PR Classification: a means to classify PRs prior to the time of approval of the product or of the ETSO article, as early in the life cycle as practical. While early classification may be preliminary, it will help to focus attention on PRs with a potential safety or functional effect, as well as process PRs that may impact the development or development assurance processes.
3. PR Assessment: a means to assess the effect of having a PR remain open at the time of approval. The assessment of PRs classified as ‘Significant’, ‘Functional’ or ‘Process’ would typically be performed by a review board. The assessment of PRs classified as ‘Life-cycle data’ may be performed within the peer-review process instead of a review board.
4. PR Resolution: a means to correct or mitigate PRs prior to the time of approval, as early in the life cycle as practical. The PR resolution process may depend on the classification of the PR; for example, shorter closure loops could be set for PRs classified as ‘Life-cycle data’.
5. PR Closure: a means to close PRs, which includes the review and confirmation of the resolution of the problem, and indicated through a documented authorisation process (e.g. Change Control Board sign-off).
GM2 to AMC 20-189 — OPR classification
ED Decision 2020/010/R
The following paragraph links the classifications presented in DO-248C/ED-94C, DP #9 to those defined in AMC 20-189, subparagraph 6.1. This paragraph highlights the clarifications made to the former scheme (e.g. removing the overlaps between the classifications).
1. The most important clarification compared with the former classification scheme is to give each OPR a single classification using a given order of priority as reflected in AMC 20-189 subparagraph 6.1.2. This promotes visibility of the most relevant issues and helps to prevent inconsistencies in classification. For example, a missing or incorrect requirement issue can be classified as ‘Life-cycle data’ only if it is confirmed that it cannot be classified as ‘Significant’, ‘Functional’, or ‘Process’, in that order of priority.
2. Type ‘Significant’: this typically maps to ‘Type 0’. However, some applicants may have used ‘Type 1A’ to characterise some PRs, for instance, those linked to Major failure conditions. The AMC 20-189 scheme clarifies that those PRs potentially causing or contributing to Catastrophic, Hazardous or Major failure conditions belong to the class ‘Significant’.
3. Type ‘Functional’: this typically maps to ‘Type 1A’ or ‘Type 1B’, that is, a problem that results in a failure with a minor or no adverse impact on safety. A PR whose consequences are a failure that can potentially lead to a Minor failure condition could be mapped to ‘Type 1A’, and a PR leading to a failure having No Safety Effect could be mapped to ‘Type 1B’. Two separate subclassifications could therefore be created in the applicant’s classification scheme to ease the mapping: problems having a functional effect leading to a Minor failure condition could be classified separately (e.g. ‘Functional 1’) from the ones having No Safety Effect (e.g. ‘Functional 2’). Moreover, an important clarification is that AMC 20-189 does not explicitly consider the ‘operational’ nature of a PR in the classification scheme to avoid creating overlaps, as a PR with operational consequences could either be classified as ‘Significant’ or ‘Functional’. Creating an ‘Operational’ subclassification within the classification ‘Significant’ or ‘Functional’ is nevertheless an option available to stakeholders to create a specific emphasis on operational issues within the proposed classification scheme.
4. Type ‘Process’: this may map to ‘Type 3A’; however, not in cases where the process non-compliance or deficiency could result in either not detecting a failure or creating a failure. An important clarification in AMC 20-189 is the removal of the ambiguous notion of ‘significant deviation from the plans or standards’ used in the definition of ‘Type 3A’. The ‘Process’ classification in AMC 20-189 should be used for PRs that record a process non-compliance or deficiency, provided they cannot result in a potential safety or potential functional effect. An example of an OPR that should not be classified as a ‘Process’ PR is one related to a requirement that was not completely verified due to a process deficiency, because the potential safety or functional impact remains undetermined. Considering the highest priority classification would, in such a case, lead to a ‘Significant’ or ‘Functional’ classification, thus putting even more emphasis on the need to resolve the shortcoming in the verification activities.
5. Type ‘Life-cycle data’: this typically maps to ‘Type 2’ or ‘Type 3B’. Since ‘Life-cycle data’ OPRs may range widely, subclassifications may be proposed by stakeholders to distinguish the different types of OPRs. Examples of OPRs classified as ‘Life-cycle data’ may range from issues in a component having no potential safety or functional impact to PRs on pure documentary issues. Moreover, the removal of the notion of ‘non-significant deviation from the plans or standards’ from the definition of ‘Type 3B’ helps to remove the ambiguity and overlap between the ‘Process’ and ‘Life-cycle data’ classifications.
6. Other OPR classification: additional classifications of OPRs may be created to cover ‘Type 4’ or any other classification not specified in AMC 20-189, paragraph 6.1.1.
GM3 to AMC 20-189 — Additional GM related to the ‘Significant’ classification
ED Decision 2020/010/R
In the frame of an engine or propeller TC/STC or of an ETSO article authorisation, the definition of ‘Significant’ is based on the anticipation of a potential effect on the product, system, or equipment function that could lead to a Catastrophic, Hazardous or Major failure condition. The goal is to identify and enhance the visibility of OPRs that may pose potential safety risks at the aircraft installation level (see AMC 20-189 paragraph 6.1.4).
For example, in the case of an engine TC, a partial or complete loss of thrust or power is regarded as a Minor Engine Effect, whereas it may have a more severe effect at the aircraft level. Unless the engine manufacturer can confirm that the effect at the installation level is no more than Minor, the OPR would be classified as ‘Significant’. The associated assumptions or mitigations are usually recorded through instructions for installing and operating the engine, e.g. in an engine installation manual.
In the case of an ETSO authorisation, classification of the failure condition is either based on assumptions defined by the applicant, or mandated through the ETSO standard, and is the basis of the safety analysis at the ETSO article level. An OPR is classified as ‘Significant’ when the OPR may lead to a functional failure associated with a Catastrophic, Hazardous or Major failure condition.
[Amdt 20/19]