RKNN Runtime C API V2.5.0 — Memory ownership and general/zero-copy flows

Platform scope and source

This article follows the RKNN V2.5.0 vendor PDF. Only the chips and interfaces listed in the source are in scope. For an ArmSoM board, check the SoC, NPU driver, Toolkit/Runtime, SDK/BSP and kernel versions first. No ArmSoM hardware validation has been performed; source examples and measurements are not ArmSoM test results.

Series navigation

• Platforms, headers and linking
• Memory ownership and general/zero-copy flows
• Device, context and initialization flags
• Model, tensor and dynamic shape queries
• Input, execution, output and dynamic shapes
• Tensor memory allocation and synchronization
• Matrix multiplication APIs and dynamic shapes
• Custom operator registration and attributes
• Status codes, tensors, memory and extension types
• Matrix multiplication structures
• Custom operator types and deprecated API migration

3. C API Common Conventions

3.1 Platform Differences

Due to the different hardware characteristics of different chip platforms, the RKNN C API interface and interface parameter support are also different. The RKNN C API interface support of each hardware platform is shown in Table 3-1:

Table 3-1 RKNN C API interface support for each hardware platform

	RKNN C API	RV1126B/RK3562/ RK3566/RK3568	RK3576/RK3588	RV1106/RV1103/ RV1103B/RV1106B	RK2118
1	rknn_init	√	√	√	√
2	rknn_set_core_mask	×	√	×	×
3	rknn_dup_context	√	√	×	×
4	rknn_destroy	√	√	√	√
5	rknn_query	√	√	√	√
6	rknn_inputs_set	√	√	×	×
7	rknn_run	√	√	√	√
8	rknn_wait	×	×	×	×
9	rknn_outputs_get	√	√	×	×
10	rknn_outputs_release	√	√	×	×
11	rknn_create_mem_from_mb_blk	×	×	×	×
12	rknn_create_mem_from_phys	√	√	√	√
13	rknn_create_mem_from_fd	√	√	√	×
14	rknn_create_mem	√	√	√	√
15	rknn_destroy_mem	√	√	√	√
16	rknn_set_weight_mem	√	√	√	√
17	rknn_set_internal_mem	√	√	√	√
18	rknn_set_io_mem	√	√	√	√
19	rknn_set_input_shapes	√	√	×	×
20	rknn_mem_sync	√	√	√	√
21	rknn_matmul_create	√	√	×	×
22	rknn_matmul_set_io_mem	√	√	×	×
23	rknn_matmul_set_core_mask	×	√	×	×
24	rknn_matmul_run	√	√	×	×
25	rknn_matmul_destroy	√	√	×	×
26	rknn_register_custom_ops	√	√	×	×
27	rknn_custom_op_get_op_attr	√	√	×	×
28	rknn_set_batch_core_num	×	√	×	×
29	rknn_matmul_set_quant_params	√	√	×	×
30	rknn_matmul_get_quant_params	√	√	×	×
31	rknn_matmul_create_dynamic_shape	√	√	×	×
32	rknn_matmul_set_dynamic_shape	√	√	×	×
33	rknn_B_normal_layout_to_native_layout	√	√	×	×
34	rknn_create_mem2	√	√	√	√

The query parameters supported by each hardware platform using the rknn_query function are shown in Table 3-2:

Table 3-2 Query parameters supported by the rknn_query function on each hardware platform

	rknn_query params	RV1126B/RK3562/ RK3566/RK3568	RK3576/ RK3588	RV1106/RV1103/ RV1103B/RV1106B/RK2118
1	RKNN_QUERY_IN_OUT_NUM	√	√	√
2	RKNN_QUERY_INPUT_ATTR	√	√	√
3	RKNN_QUERY_OUTPUT_ATTR	√	√	√
4	RKNN_QUERY_PERF_DETAIL	√	√	×
5	RKNN_QUERY_PERF_RUN	√	√	×
6	RKNN_QUERY_SDK_VERSION	√	√	√
7	RKNN_QUERY_MEM_SIZE	√	√	√
8	RKNN_QUERY_CUSTOM_STRING	√	√	√
9	RKNN_QUERY_NATIVE_INPUT_ATTR	√	√	√
10	RKNN_QUERY_NATIVE_OUTPUT_ATTR	√	√	√
11	RKNN_QUERY_NATIVE_NC1HWC2_INPUT_ATTR	√	√	√
12	RKNN_QUERY_NATIVE_NC1HWC2_OUTPUT_ATTR	√	√	√
13	RKNN_QUERY_NATIVE_NHWC_INPUT_ATTR	√	√	√
14	RKNN_QUERY_NATIVE_NHWC_OUTPUT_ATTR	√	√	√
15	RKNN_QUERY_INPUT_DYNAMIC_RANGE	√	√	×
16	RKNN_QUERY_DEVICE_MEM_INFO	×	×	√
17	RKNN_QUERY_CURRENT_INPUT_ATTR	√	√	×
18	RKNN_QUERY_CURRENT_OUTPUT_ATTR	√	√	×
19	RKNN_QUERY_CURRENT_NATIVE_INPUT_ATTR	√	√	×
20	RKNN_QUERY_CURRENT_NATIVE_OUTPUT_ATTR	√	√	×

3.2 Memory Ownership

Memory allocated by the API: The tensor memory allocated via rknn_create_mem is owned by the caller and must be released by calling rknn_destroy_mem when no longer used. Avoid double release.

User-allocated memory: The user-allocated memory accepted by rknn_set_weight_mem, rknn_set_internal_mem, and rknn_set_io_mem remains owned by the caller and must be released by the caller after use.

External memory wrappers: rknn_create_mem_from_fd, rknn_create_mem_from_phys, and rknn_create_mem_from_mb_blk associate already-allocated memory (represented by a file descriptor, physical address, or mb_blk) with the RKNN framework for zero-copy. The wrapped rknn_tensor_mem object must still be released via rknn_destroy_mem.

3.3 Input and Output Data Validity

The inputs array and the input data pointed to by buf of rknn_inputs_set must remain valid during the call.

The outputs array of rknn_outputs_get must be created by the user in advance with index set; when is_prealloc is 1, buf is pre-allocated by the user and must remain valid during the call; when is_prealloc is 0, buf is allocated by the interface and must remain valid until released by rknn_outputs_release. After release, the buf pointer becomes invalid and must no longer be used.

The input/output tensor memory set via rknn_set_io_mem must remain valid during the call and must be set before calling rknn_run.

3.4 Return Values and Error Handling

Most interfaces return an int status code: 0 indicates success and a negative value indicates an error. A few interfaces return pointers (such as rknn_create_mem, rknn_create_mem_from_fd, rknn_create_mem_from_phys, rknn_create_mem_from_mb_blk) and return NULL on failure.

For the complete status code definitions, see 6.1 Status Codes.

The caller should check the return value of every call and, when handling errors, locate the problem based on the status code (for example, RKNN_ERR_DEVICE_UNMATCH indicates a version mismatch, and RKNN_ERR_CTX_INVALID indicates an invalid context).

3.5 General API and Zero-copy API

When inferring an RKNN model, the raw data goes through three main flows: input processing, NPU running the model, and output processing. Depending on the model input format and quantization method, there are two processing flows inside the interface: the general API and the zero-copy API. The main differences between them in usage and call flow are as follows:

General API: Provides a simple and easy-to-use set of inference interfaces (rknn_inputs_set/rknn_outputs_get/rknn_outputs_release). Every time frame data is updated, the data allocated by the external module must be copied into the NPU runtime’s input memory. Normalization, quantization, data layout conversion and dequantization are mainly performed on the CPU, while model inference runs on the NPU.

Zero-copy API: Creates or wraps memory via interfaces such as rknn_create_mem/rknn_create_mem_from_fd/rknn_create_mem_from_phys, then sets the input/output memory via rknn_set_io_mem. It directly uses pre-allocated memory (created by the NPU runtime or by another external framework such as DRM), reducing memory copies, delivering better performance and lower bandwidth. Normalization, quantization and model inference all run on the NPU.

Interface selection: When the user input data only has a virtual address, only the general API can be used; when the input data has a physical address or fd, the zero-copy API is used. The general API and the zero-copy API cannot be mixed.

General API call flow (as shown in Figure 3-1):

1. Call rknn_init() to initialize the model;

1. Call rknn_query() to query the input/output attributes of the model;

1. Preprocess the input;

1. Call rknn_inputs_set() to set the input data;

1. Call rknn_run() to perform model inference;

1. Call rknn_outputs_get() to get the inference result data;

1. Postprocess the output;

1. Call rknn_outputs_release() to release the output data memory;

1. Call rknn_destroy() to destroy RKNN.

Figure 3-1 General API call flow

Zero-copy API call flow (taking input/output memory allocated by the runtime as an example, as shown in Figure 3-2):

1. Call rknn_init() to initialize the model;

1. Call rknn_query() to query the native input/output attributes of the model (RKNN_QUERY_NATIVE_INPUT_ATTR/RKNN_QUERY_NATIVE_OUTPUT_ATTR);

1. Preprocess the input (optional);

1. Call rknn_create_mem() to create the input/output rknn_tensor_mem structures (use rknn_create_mem_from_fd()/rknn_create_mem_from_phys() for externally allocated memory);

1. Call rknn_set_io_mem() to set the input/output memory;

1. Call rknn_run() to perform model inference;

1. Read the output memory information from the rknn_tensor_mem structure and postprocess;

1. Call rknn_destroy_mem() to release the input/output memory;

1. Call rknn_destroy() to destroy RKNN.

Figure 3-2 Zero-copy API call flow (input/output allocated internally)

For the detailed usage of rknn_set_io_mem(), see rknn_set_io_mem in 5.4 Tensor and Memory.

Source and editorial notes

Technical chapters are split by use. Code, identifiers and parameters are retained. Chinese follows the Chinese source; English, Russian and German follow the English source. No invented figures are added to chapters without figures.

Source caveats: Chapter numbers and example output are retained; old SDK versions in example logs are not the current document version. The Runtime source jumps from chapter 3 to 5 and repeats query subsection numbers. Its rknn_init parameter table calls extend disabled, while later types describe user_key and shared-weight fields. The native dynamic-output prose uses RKNN_QUERY_CURRENT_OUTPUT_ATTR, but its example uses RKNN_QUERY_CURRENT_NATIVE_OUTPUT_ATTR. Some native NHWC/NC1HWC2 descriptions also confuse input and output. Verify against actual V2.5.0 headers and the target SDK; these are not validated behaviors. The RK2118 cross-reference says RKNN3_RK2118_Quick_Start.pdf, while the current directory contains an RKNN2 quick-start guide.

RKNN-Toolkit2 · ArmSoM documentation