T/AI 117-2022 Deep Learning inference engine application programming interface (API)

Scope

Scope: The "Deep Learning Inference Engine Application Programming Interface (API) Specification" defines the application programming interface (API) specification for deep learning inference engines, including four types of interfaces: model management interface, device management interface, input/output interface, and inference interface. This standard applies to deep learning inference engines deployed on devices, including cloud-side (IDC) inference engines and edge-side (mobile, edge) inference engines. It can be used to guide the design and development of inference engine APIs. However, some provisions may not apply to special industries or professional applications. Main technical content: A deep learning inference engine is a software framework specifically designed for deep learning prediction or inference. It provides a series of high-level interfaces for loading deep learning models, mounting and adapting all operations involved in hardware execution of models, and obtaining inference results. It can be applied to scenarios such as image recognition and natural language processing.

Development Information

Related Standards