RKNN2 V2.5.0 · RK3566/RK3568：Activations, normalization and extensions

This RKNN2 V2.5.0 platform reference helps ArmSoM developers select and adapt models. It does not establish support on every ArmSoM board: verify the board SoC, actual Toolkit2, Runtime, NPU driver and BSP versions. The source specifies no universal Linux kernel version. No ArmSoM hardware validation was performed.

Source: the supplied Chinese and English RKNN2 Supported Operators and Constraints Reference V2.5.0, 374 and 384 pages respectively. Chinese follows the Chinese edition; English, Russian and German follow the English edition. Platform constraints remain separate. The English PDF itself contains Chinese I/O figures and output tables; their provenance is retained with editable localized tables. Source spellings such as ConvTranpose and the original LSTM/GRU equations are preserved; these equations are not corrections to SDK implementations.

Source pages: CN 58–69; EN 60–71.

Rockchip RKNN2

Sigmoid

Given an input tensor, the function y=1/(1+exp(-x)) is applied element-wise to the input tensor to produce the output tensor

Inputs

• input:T1
  • shape:[batch,channel,height,width]
  • Quantization support
    • per-layer

Outputs

• output:T1
  • shape:[batch,channel,height,width]
  • Quantization support
    • per-layer

Data Type Constraints

• T1:int8(Tensor),float16(Tensor)

Tanh

Computes the hyperbolic tangent of the given input tensor element-wise

Inputs

• X:T1
  • shape:[batch,channel,height,width]
  • Quantization support:per-layer

Outputs

• Y:T1
  • shape:[batch,channel,height,width]
  • Quantization support:per-layer

Data Type Constraints

• T1:int8(Tensor),float16(Tensor)

Softplus

Applies the Softplus function element-wise, y=ln(exp(x)+1)

Inputs

• X:T1
  • shape:[batch,channel,height,width]
  • Quantization support:per-layer

Outputs

• Y:T1
  • shape:[batch,channel,height,width]
  • Quantization support:per-layer

Data Type Constraints

• T1:int8(Tensor),float16(Tensor)

HardSigmoid

Given an input tensor, the function y=max(0,min(1,alpha*x+beta)) is applied element-wise to the input tensor to produce the output tensor, where alpha=1/6, beta=0.5

Inputs

• input:T1
  • shape:[batch,channel,height,width]
  • Quantization support
    • per-layer

Outputs

• output:T1
  • shape:[batch,channel,height,width]
  • Quantization support
    • per-layer

Data Type Constraints

• T1:int8(Tensor),float16(Tensor)

HardSwish

Applies the HardSwish function element-wise, y = x * max(0, min(1, alpha * x + beta)) = x * HardSigmoid(x), where alpha=1/6, beta=0.5

Inputs

• X:T1
  • shape:[batch,channel,height,width]
  • Quantization support:per-layer

Outputs

• Y:T1
  • shape:[batch,channel,height,width]
  • Quantization support:per-layer

Data Type Constraints

• T1:int8(Tensor),float16(Tensor)

Elu

Given an input tensor, the function f(x)=alpha*(exp(x)-1.) for x<0, f(x)=x for x>=0 is applied element-wise to the input tensor to produce the output tensor, where alpha=1

Inputs

• input:T1
  • shape:[batch,channel,height,width]
  • Quantization support
    • per-layer

Outputs

• output:T1
  • shape:[batch,channel,height,width]
  • Quantization support
    • per-layer

Data Type Constraints

• T1:int8(Tensor),float16(Tensor)

exSwish

Applies the Sigmoid Linear Unit function element-wise. The Swish function is also known as the SiLU function, y=x*sigmoid(x)

Inputs

• X:T1
  • shape:[batch,channel,height,width]
  • Quantization support:per-layer

Outputs

• Y:T1
  • shape:[batch,channel,height,width]
  • Quantization support:per-layer

Data Type Constraints

• T1:int8(Tensor),float16(Tensor)

exMish

Given an input tensor, the function mish(x)=xtanh(softplus(x))=xtanh(ln(1+e^{x})) is applied element-wise to the input tensor to produce the output tensor

Inputs

• input:T1
  • shape:[batch,channel,height,width]
  • Quantization support
    • per-layer

Outputs

• output:T1
  • shape:[batch,channel,height,width]
  • Quantization support
    • per-layer

Data Type Constraints

• T1:int8(Tensor),float16(Tensor)

exGelu

Given an input tensor, y=x*Φ(x) is applied element-wise to produce the output tensor. When Φ(x) is set to tanh: y=0.5*x*(1+Tanh(sqrt(2/π)*(x+0.044715*x^3))).

Inputs

• input:T1
  • shape:[batch,channel,height,width]
  • Quantization support
    • per-layer

Outputs

• output:T1
  • shape:[batch,channel,height,width]
  • Quantization support
    • per-layer

Data Type Constraints

• T1:int8(Tensor),float16(Tensor)

exSwooshR

Given an input tensor, the following function from the paper https://arxiv.org/abs/2310.11230 is applied element-wise to the input tensor to produce the output tensor SwooshR(x)=log(1+exp(x−1))−0.08x−0.313261687

Inputs

• input:T1
  • shape:[batch,channel,height,width]
  • Quantization support
    • per-layer

Outputs

• output:T1
  • shape:[batch,channel,height,width]
  • Quantization support
    • per-layer

Data Type Constraints

• T1:int8(Tensor),float16(Tensor)

exSwooshL

Given an input tensor, the following function from the paper https://arxiv.org/abs/2310.11230 is applied element-wise to the input tensor to produce the output tensor SwooshL(x)=log(1+exp(x−4))−0.08x−0.035

Inputs

• input:T1
  • shape:[batch,channel,height,width]
  • Quantization support
    • per-layer

Outputs

• output:T1
  • shape:[batch,channel,height,width]
  • Quantization support
    • per-layer

Data Type Constraints

• T1:int8(Tensor),float16(Tensor)

Where

Obtains values from x_tensor or y_tensor according to mask_tensor. When the position of mask_tensor is True, the value of x_tensor at the corresponding position is taken; otherwise, the value of y_tensor is taken.

Inputs

• mask_tensor:T1
  • shape:[batch,channel,height,width]
  • Broadcasting support
    • Supports broadcasting along arbitrary dimensions
  • Broadcasting constraints
    • Broadcasting constraints are the same as the Expand operator
• x_tensor:*T2
  • shape:[batch,channel,height,width]
  • Quantization support
    • per-layer
  • Broadcasting support
    • Supports broadcasting along arbitrary dimensions
  • Broadcasting constraints
    • Broadcasting constraints are the same as the Expand operator
• y_tensor:T1
  • shape:[batch,channel,height,width]
  • Quantization support
    • per-layer
  • Broadcasting support
    • Supports broadcasting along arbitrary dimensions
  • Broadcasting constraints
    • Broadcasting constraints are the same as the Expand operator

Outputs

• output:T2
  • shape:[batch,channel,height,width]
  • Quantization support
    • per-layer

Data Type Constraints

• T1:bool(Tensor)
• T2:int8(Tensor),float16(Tensor)

exGlu

Gated Linear Unit function, where a is the first half of the input matrix and b is the second half GLU(a,b)=a⊗σ(b)

Inputs

• Input:T1
  • shape:[batch,channel,height,width]
  • Quantization support:per-layer

Outputs

• Output:T1
  • shape:[batch,channel/2,height,width]
  • Quantization support:per-layer

Data Type Constraints

• T1:int8(Tensor),float16(Tensor)

Attributes

• axis: int (default value = 1)
  • The dimension to split the input
  • Constraint: only 1 is supported

exMatMul

Matrix multiplication operation

Y = A·B + C    if c_type = "add"
Y = A·B * C    if c_type = "mul"

Inputs

• A:T1
  • shape:[b,k,1,n]
    • k ∈(0,8192]
  • Quantization support
    • per-layer
• B:T1
  • shape:[b,k,1,m]
  • Quantization support
    • per-layer
• C(optional):T1,T2
  • shape:[b,n,1,m]
  • Quantization support
    • per-layer
  • Broadcasting support
    • Supports broadcasting along arbitrary dimensions
    • Broadcasting constraints are the same as the Expand operator

Outputs

• Y:T1
  • shape:[b,n,1,m]
  • Quantization support
    • per-layer

Data Type Constraints

• T1:int8(Tensor),float16(Tensor)
• T2:float(Tensor)

Attributes

• c_type: strings (default “add”)
  • add and mul are optional, indicating whether C is added or multiplied

exNorm

Performs various normalization operations on the input data. Currently three types are supported: LayerNormalization, RMSNormalization, and MeanVarianceNormalization:

LayerNorm(x) = ((x - μ) / sqrt(σ^2 + ε)) * W + B

RMSNorm(x) = (x / sqrt(RMS^2 + ε)) * W + B

MeanVarianceNorm(x) = ((x - μ) / sqrt(Variance + ε)) * W + B

Where: x is the input vector. μ and σ2 are the mean and variance of the input vector respectively:

μ = (1/H) * Σ[i=1..H](x_i)

σ^2 = (1/H) * Σ[i=1..H]((x_i - μ)^2)

RMS is the root mean square of the input vector:

RMS = sqrt((1/H) * Σ[i=1..H](x_i^2))

Variance is the equivalent variance of the input vector:

Variance = (1/H) * Σ[i=1..H](x_i^2) - μ^2

ε is a small number used to prevent division by zero. W and B are optional parameters.

Inputs

• X:T1
  • shape:[batch,channel,height,width]
• W(optional):T1
  • shape:[batch,channel,height,width]
• B(optional):T1
  • shape:[batch,channel,height,width]

Outputs

• Y:T1
  • shape:[batch,channel,height,width]

Data Type Constraints

• T1:float16(Tensor)

Attributes

• epsilon: float (default value 1e-05)
  • The epsilon value used to prevent division by 0
• mode:string
  • Three normalization modes are available: layer, rms, and mean_var
• axes: int64[] (default value [1])
  • Normalization axes, currently only [1] and [1,2,3] are supported

Other Support

• Multi-core: Not yet supported

exWindow

exWindow originates from the window_partition and window_reverse operations of swin_transformer. In the original paper, input features are divided into multiple non-overlapping windows, each of size window_size × window_size. Both input and output features are required to be 4-dimensional shapes, partitioned in both height and width directions. The pytorch implementation is as follows:

def window_partition(x, window_size):
    """
    Args:
        x: (B, C, H, W)
        window_size (int): window size
    Returns:
        windows: (B, C, num_windows * num_windows, window_size * window_size)
    """
    B, C, H, W = x.shape
    x = x.view(B, C, H // window_size, window_size, W // window_size, window_size)
    windows = x.permute(0, 1, 2, 4, 3, 5).contiguous().view(B, C, -1, window_size * window_size)
    return windows

def window_reverse(windows, window_size, H, W):
    """
    Args:
        windows: (B, C, num_windows * num_windows, window_size * window_size)
        window_size (int): Window size
        H (int): Height of image
        W (int): Width of image
    Returns:
        x: (B, C, H, W)
    """
    B, C, _, _ = windows.shape
    x = windows.view(B, C, H // window_size, W // window_size, window_size, window_size)
    x = x.permute(0, 1, 2, 4, 3, 5).contiguous().view(B, C, H, W)
    return x

Inputs

• X:T1
  • shape:[batch,channel,height,width]

Outputs

• Y:T1
  • shape:[batch,channel,height,width]

Data Type Constraints

• T1:int8(Tensor),float16(Tensor)

Attributes

• mode:string
  • Supported modes: partition, reverse, partition_num_first
    • partition: implemented with reference to window_partition;
    • reverse: implemented with reference to window_reverse;
    • partition_num_first: the difference from partition is that the H,W of partition_num_first are divided according to num_windows, while the H,W of partition are divided according to window_sizes.
• window_sizes:listofints
  • A 2-dimensional vector, the window size. Defines the height and width of each window. Attention computation and feature extraction within the window are based on this size.
• num_windows:listofints
  • A 2-dimensional vector, the number of windows. Defines how many windows the height and width of the input feature are divided into.

Other Support

• Multi-core: Not yet supported