Platform reference for RKNN Toolkit Lite2 V2.5.0, Debian 10/11/12 aarch64 and the SoCs listed in the source. For an ArmSoM board, verify SoC, architecture, Python wheel, BSP NPU driver and Runtime compatibility. No hardware tests were performed. The source encryption section lists more platforms than the Lite2 installation table; both are retained without extending Lite2 device support by inference.
Adapted from the local Chinese and English V2.5.0 sources, retaining section numbers. Chinese uses Chinese figures; English, Russian and German use English-source figures. PDF wraps, continued tables and code indentation are restored. Example outputs are not ArmSoM tests.
RKNN Toolkit Lite2 V2.5.0, part 1: installation, ResNet18 and return conventions
RKNN Toolkit Lite2 V2.5.0, part 2: lifecycle, input layout and output reuse
RKNN Toolkit Lite2 V2.5.0, part 3: model loading, encryption and Runtime initialization
RKNN Toolkit Lite2 V2.5.0, part 4: inference, device and version query APIs
RKNN Toolkit Lite2 V2.5.0, part 5: error checks, Core Mask and logging
3.3 Model Inference
inference
| API | inference |
|---|---|
| Function | Run model inference on the specified inputs and return the output results. |
| Parameters | inputs: Input data. Type: list; each item is an ndarray. |
| data_type: Input data type. Default: None. The interface lists int8, uint8, int16, float16, and float32. |
|
| data_format: Input data layout. Default: None. Currently supports “nhwc” and “nchw”. |
|
| inputs_pass_through: Pass Through flag list for the inputs. Each element uses 0 / 1 to indicate disabled / enabled. Default: None. |
|
| get_frame_id: Whether to obtain the Frame ID. Default: False. Mainly used in asynchronous mode. |
|
| reuse_outputs: Whether to reuse the internal output buffers. Default: False. When True, the API returns views of the internal reusable buffers, suitable for long-running inference loops; note that the next inference overwrites the returned results. |
|
| Return Value |
Normally returns a list of output ndarray objects. If get_frame_id=True, the Frame ID is appended to the end of the list. Returns None if the Runtime is not initialized or an exception occurs during inference. |
Example:
outputs = rknn_lite.inference(inputs=[img])
if outputs is None:
print('Inference failed')
else:
show_top5(outputs)
To reduce memory allocation in a long-running inference loop, enable output reuse:
for frame in frames:
outputs = rknn_lite.inference(
inputs=[frame],
reuse_outputs=True
)
show_top5(outputs) # consume immediately; results are overwritten
# by the next inference
Reference output:
-----TOP 5-----
[812]: 0.999442994594574
[404]: 0.0004096269840374589
[657]: 3.284541890025139e-05
[833]: 2.6112385967280716e-05
[895]: 1.8509887013351545e-05
Usage Notes:
init_runtime() must complete successfully before this API is called; otherwise, the API returns None .
inputs , data_type , data_format , and inputs_pass_through are passed to the Runtime input-setting logic.
The API returns None if an exception occurs while setting inputs, running the model, or retrieving outputs.
When get_frame_id=False , the API returns the normal output list. When get_frame_id=True , the Frame ID returned by the Runtime is appended to the end of the output list.
reuse_outputs defaults to False , and the returned arrays own their memory and stay valid across calls. Only enable reuse_outputs=True when explicitly reducing memory allocation; the returned arrays are then views of the internal reusable buffers and will be overwritten by the next inference, so copy the results first if they must be retained.
After inference succeeds, perform the required post-processing according to the model type.
3.4 Device, Version, and Platform Queries
list_devices
| API | list_devices |
|---|---|
| Function | Enumerate devices currently accessible through ADB and NTB and print the device lists. |
| Parameters | None |
| Return Value |
(adb_devices, ntb_devices): two list objects. Devices that also appear in the NTB list are removed from the returned ADB list. |
Example:
adb_devices, ntb_devices = rknn_lite.list_devices()
get_sdk_version
| API | get_sdk_version |
|---|---|
| Function | Get SDK version information. |
| Parameters | None |
| Return Value |
Normally returns an sdk_version string; returns None if the Runtime is not initialized or the query fails. |
The Runtime must be initialized successfully before calling this API. Because init_runtime() requires a model to be loaded, the standard sequence is load_rknn() → init_runtime() → get_sdk_version() .
Example:
sdk_version = rknn_lite.get_sdk_version()
if sdk_version is None:
print('Get SDK version failed')
Reference return information:
I RKNN: [10:47:23.097] RKNN Runtime Information:
librknnrt version: 1.2.0b1
(a47985372@2021-12-06T10:20:14)
I RKNN: [10:47:23.097] RKNN Driver Information:
version: 0.6.2
I RKNN: [10:47:23.098] RKNN Model Information:
version: 1,
toolkit version: 1.1.2
(compiler version: 1.1.2b13
(1e5726f57@2021-11-29T14:13:55)),
target: RKNPU v2,
target platform: rk3588,
framework name: PyTorch,
framework layout: NCHW
list_support_target_platform
| API | list_support_target_platform |
|---|---|
| Function | Query the chip platforms on which a given RKNN model can run, or list the currently supported platforms. |
| Parameters | rknn_model: RKNN model path. Default: None. When it is None, the platform query function is called directly to list the currently supported platforms. |
| Return Value |
Normally returns support_target, a dict. Returns None if an explicitly specified model path does not exist. |
Example:
support_target = rknn_lite.list_support_target_platform(
rknn_model='mobilenet_v1.rknn'
)
if support_target is None:
print('Query target platform failed')
Reference result:
**************************************************
Target platforms filled in RKNN model: ['RK3588']
Target platforms supported by this RKNN model: ['RK3588']
**************************************************
Usage Notes:
When rknn_model=None , file-existence checking is not performed; the underlying platform query function is called directly.
When an explicitly specified model path does not exist, the API records an error and returns None .