T/CESA 1026-2018 Artificial intelligence and Deep Learning Algorithm Evaluation Specification

Scope

Scope: This standard proposes an evaluation index system and evaluation process for deep learning algorithms, as well as content such as evaluation at the requirement stage, design stage, implementation stage, and operation stage. This standard is applicable to guiding organizations such as deep learning algorithm developers, users, and third parties in conducting reliability evaluations of deep learning algorithms. Main technical content: This standard primarily draws on the ideas of the international standard DO-178C-2011 "Software Consideration in Airborne Systems and Equipment Certification". It defines multiple different objectives for different stages of the algorithm to meet different levels, while realizing the concept of forward and backward tracking of reliability indicators for deep learning algorithms, making it easier to verify and evaluate the algorithm development process and achieve the goal of reliability evaluation. This standard combines the characteristics of deep learning algorithms and the traditional reliability evaluation system and process to formulate the standard content, which mainly includes the following aspects: - **Evaluation Index System for Algorithm Reliability**: Based on the internal and external impacts on the reliability of deep learning algorithms and combined with the actual application scenarios of users, this standard proposes a set of reliability evaluation indicators for deep learning algorithms. The index system includes 7 first-level indicators and 20 second-level indicators. - **Evaluation Process for Algorithm Reliability**: Based on the reliability evaluation index system for deep learning algorithms, this standard conducts reliability evaluations of deep learning algorithms. The evaluation process includes seven stages: determining reliability objectives, selecting evaluation indicators, evaluation at the requirement stage, evaluation at the design stage, evaluation at the implementation stage, evaluation at

Development Information

Related Standards