RKNN2 V2.5.0 · RK2118：Activations, normalization and extensions

This RKNN2 V2.5.0 platform reference helps ArmSoM developers select and adapt models. It does not establish support on every ArmSoM board: verify the board SoC, actual Toolkit2, Runtime, NPU driver and BSP versions. The source specifies no universal Linux kernel version. No ArmSoM hardware validation was performed.

Source: the supplied Chinese and English RKNN2 Supported Operators and Constraints Reference V2.5.0, 374 and 384 pages respectively. Chinese follows the Chinese edition; English, Russian and German follow the English edition. Platform constraints remain separate. The English PDF itself contains Chinese I/O figures and output tables; their provenance is retained with editable localized tables. Source spellings such as ConvTranpose and the original LSTM/GRU equations are preserved; these equations are not corrections to SDK implementations.

Source pages: CN 307–317; EN 316–326.

Rockchip RKNN2

Sigmoid

Given an input tensor, the function y=1/(1+exp(-x)) is applied element-wise to the input tensor to produce the output tensor

Inputs

• input:T1
  • shape:[batch,channel,height,width]
  • Quantization support
    • per-layer

Outputs

• output:T1
  • shape:[batch,channel,height,width]
  • Quantization support
    • per-layer

Data Type Constraints

• T1:float16(Tensor)

Tanh

Computes the hyperbolic tangent of the given input tensor element-wise

Inputs

• X:T1
  • shape:[batch,channel,height,width]
  • Quantization support:per-layer

Outputs

• Y:T1
  • shape:[batch,channel,height,width]
  • Quantization support:per-layer

Data Type Constraints

• T1:float16(Tensor)

Softplus

Applies the Softplus function element-wise, y=ln(exp(x)+1)

Inputs

• X:T1
  • shape:[batch,channel,height,width]
  • Quantization support:per-layer

Outputs

• Y:T1
  • shape:[batch,channel,height,width]
  • Quantization support:per-layer

Data Type Constraints

• T1:float16(Tensor)

HardSigmoid

Given an input tensor, the function y=max(0,min(1,alpha*x+beta)) is applied element-wise to the input tensor to produce the output tensor, where alpha=1/6, beta=0.5

Inputs

• input:T1
  • shape:[batch,channel,height,width]
  • Quantization support
    • per-layer

Outputs

• output:T1
  • shape:[batch,channel,height,width]
  • Quantization support
    • per-layer

Data Type Constraints

• T1:float16(Tensor)

HardSwish

Applies the HardSwish function element-wise, y = x * max(0, min(1, alpha * x + beta)) = x * HardSigmoid(x), where alpha=1/6, beta=0.5

Inputs

• X:T1
  • shape:[batch,channel,height,width]
  • Quantization support:per-layer

Outputs

• Y:T1
  • shape:[batch,channel,height,width]
  • Quantization support:per-layer

Data Type Constraints

• T1:float16(Tensor)

Elu

Given an input tensor, the function f(x)=alpha*(exp(x)-1.) for x<0, f(x)=x for x>=0 is applied element-wise to the input tensor to produce the output tensor, where alpha=1

Inputs

• input:T1
  • shape:[batch,channel,height,width]
  • Quantization support
    • per-layer

Outputs

• output:T1
  • shape:[batch,channel,height,width]
  • Quantization support
    • per-layer

Data Type Constraints

• T1:float16(Tensor)

exSwish

Applies the Sigmoid Linear Unit function element-wise. The Swish function is also known as the SiLU function, y=x*sigmoid(x)

Inputs

• X:T1
  • shape:[batch,channel,height,width]
  • Quantization support:per-layer

Outputs

• Y:T1
  • shape:[batch,channel,height,width]
  • Quantization support:per-layer

Data Type Constraints

• T1:float16(Tensor)

exMish

Given an input tensor, the function mish(x)=xtanh(softplus(x))=xtanh(ln(1+e^{x})) is applied element-wise to the input tensor to produce the output tensor

Inputs

• input:T1
  • shape:[batch,channel,height,width]
  • Quantization support
    • per-layer

Outputs

• output:T1
  • shape:[batch,channel,height,width]
  • Quantization support
    • per-layer

Data Type Constraints

• T1:float16(Tensor)

exGelu

Given an input tensor, y=x*Φ(x) is applied element-wise to produce the output tensor. When Φ(x) is set to tanh: y=0.5*x*(1+Tanh(sqrt(2/π)*(x+0.044715*x^3))).

Inputs

• input:T1
  • shape:[batch,channel,height,width]
  • Quantization support
    • per-layer

Outputs

• output:T1
  • shape:[batch,channel,height,width]
  • Quantization support
    • per-layer

Data Type Constraints

• T1:float16(Tensor)

Where

Obtains values from x_tensor or y_tensor according to mask_tensor. When the position of mask_tensor is True, the value of x_tensor at the corresponding position is taken; otherwise, the value of y_tensor is taken.

Inputs

• mask_tensor:T1
  • shape:[batch,channel,height,width]
  • Broadcasting support
    • Supports broadcasting along arbitrary dimensions
  • Broadcasting constraints
    • Broadcasting constraints are the same as the Expand operator
• x_tensor:*T2
  • shape:[batch,channel,height,width]
  • Quantization support
    • per-layer
  • Broadcasting support
    • Supports broadcasting along arbitrary dimensions
  • Broadcasting constraints
    • Broadcasting constraints are the same as the Expand operator
• y_tensor:T1
  • shape:[batch,channel,height,width]
  • Quantization support
    • per-layer
  • Broadcasting support
    • Supports broadcasting along arbitrary dimensions
  • Broadcasting constraints
    • Broadcasting constraints are the same as the Expand operator

Outputs

• output:T2
  • shape:[batch,channel,height,width]
  • Quantization support
    • per-layer

Data Type Constraints

• T1:bool(Tensor)
• T2:float16(Tensor)

exConvStreaming

To reduce the latency of speech models, the input of the original Convolution is divided into several time-frame slices. Each time-frame slice is used as input to compute the convolution, and the results are cached for subsequent time frames, thereby achieving streaming audio processing.

Inputs

• input_tensor:T1
  • shape:[batch,channel,height,width]
    • width: currently width=1 is supported.
  • Quantization support
    • per-layer
• weight:T1
  • shape:[output_channel,input_channel,kernel_h,kernel_w]
    • kernel_h∈[1,31]
    • kernel_w∈[1,31]
  • Quantization support
    • per-layer
    • per-channel

Outputs

• output:T1
  • shape:[batch,channel,height,width]
  • Quantization support
    • per-layer

Attributes

• strides:int(strides_h,strides_w)
  • stride_h∈[1,7]
  • stride_w∈[1,7]
• pads:int(pads_top,pads_left,pads_bottom,pads_right)
  • pads_top∈[0,15]
  • pads_left∈[0,15]
  • pads_bottom∈[0,15]
  • pads_right∈[0,15]
• group:int
• dilations:int(dilations_h,dilations_w)
  • dilations_h∈[1,32]
  • dilations_w∈[1,32]

Data Type Constraints

• T1:float16(Tensor)

Other Support

• Multi-core: Not yet supported

exConvStreamingPRelu/LeakyRelu

exConvStreaming and PRelu/LeakyRelu fused computation, Output = PRelu(exConvStreaming(Input)) or Output= LeakyRelu(exConvStreaming(Input)). Description and specification constraints are the same as exConvStreaming

exGlu

Gated Linear Unit function, where a is the first half of the input matrix and b is the second half GLU(a,b)=a⊗σ(b)

Inputs

• Input:T1
  • shape:[batch,channel,height,width]
  • Quantization support:per-layer

Outputs

• Output:T1
  • shape:[batch,channel/2,height,width]
  • Quantization support:per-layer

Data Type Constraints

• T1:float16(Tensor)

Attributes

• axis: int (default value = 1)
  • The dimension to split the input
  • Constraint: only 1 is supported

exMatMul

Matrix multiplication operation

Y = A·B + C    if c_type = "add"
Y = A·B * C    if c_type = "mul"

Inputs

• A:T1
  • shape:[b,k,1,n]
    • k ∈(0,8192]
  • Quantization support
    • per-layer
• B:T1
  • shape:[b,k,1,m]
  • Quantization support
    • per-layer
• C(optional):T1,T2
  • shape:[b,n,1,m]
  • Quantization support
    • per-layer
  • Broadcasting support
    • Supports broadcasting along arbitrary dimensions
    • Broadcasting constraints are the same as the Expand operator

Outputs

• Y:T1
  • shape:[b,n,1,m]
  • Quantization support
    • per-layer

Data Type Constraints

• T1:float16(Tensor)
• T2:float(Tensor)

Attributes

• c_type: strings (default “add”)
  • add and mul are optional, indicating whether C is added or multiplied

exWindow

exWindow originates from the window_partition and window_reverse operations of swin_transformer. In the original paper, input features are divided into multiple non-overlapping windows, each of size window_size × window_size. Both input and output features are required to be 4-dimensional shapes, partitioned in both height and width directions. The pytorch implementation is as follows:

def window_partition(x, window_size):
    """
    Args:
        x: (B, C, H, W)
        window_size (int): window size
    Returns:
        windows: (B, C, num_windows * num_windows, window_size * window_size)
    """
    B, C, H, W = x.shape
    x = x.view(B, C, H // window_size, window_size, W // window_size, window_size)
    windows = x.permute(0, 1, 2, 4, 3, 5).contiguous().view(B, C, -1, window_size * window_size)
    return windows

def window_reverse(windows, window_size, H, W):
    """
    Args:
        windows: (B, C, num_windows * num_windows, window_size * window_size)
        window_size (int): Window size
        H (int): Height of image
        W (int): Width of image
    Returns:
        x: (B, C, H, W)
    """
    B, C, _, _ = windows.shape
    x = windows.view(B, C, H // window_size, W // window_size, window_size, window_size)
    x = x.permute(0, 1, 2, 4, 3, 5).contiguous().view(B, C, H, W)
    return x

Inputs

• X:T1
  • shape:[batch,channel,height,width]

Outputs

• Y:T1
  • shape:[batch,channel,height,width]

Data Type Constraints

• T1:float16(Tensor)

Attributes

• mode:string
  • Supported modes: partition, reverse, partition_num_first
    • partition: implemented with reference to window_partition;
    • reverse: implemented with reference to window_reverse;
    • partition_num_first: the difference from partition is that the H,W of partition_num_first are divided according to num_windows, while the H,W of partition are divided according to window_sizes.
• window_sizes:listofints
  • A 2-dimensional vector, the window size. Defines the height and width of each window. Attention computation and feature extraction within the window are based on this size.
• num_windows:listofints
  • A 2-dimensional vector, the number of windows. Defines how many windows the height and width of the input feature are divided into.

Other Support

• Multi-core: Not yet supported