T/SAITA 001-2023 Natural language processing system evaluation specification
T/SAITA 001-2023 Natural language processing system evaluation specification
Basic Information
Scope
Scope: It specifies the basic framework of natural language processing systems, the evaluation index system, the evaluation process, and the evaluation methods, and provides evaluation guidelines for natural language processing systems. Main technical content: It specifies the basic framework of natural language processing systems; proposes an evaluation index system for natural language processing systems that includes 10 first-level indicators and 36 second-level indicators, covering indicators such as functional effectiveness, performance, compatibility, maintainability, interpretability, and security; specifies the evaluation process and testing methods for natural language processing systems. The appendix specifies the calculation methods for the functional effectiveness of natural language understanding and natural language generation, and provides relevant reference datasets and evaluation guidelines for natural language processing systems.