Rockchip SED: AED, baby cry, alarms and glass-break APIs

Complete Chinese V1.6.0 content; source scope all series/general. ArmSoM platform reference only: verify SoC, SDK/BSP, audio input, libraries and RKNN/NPU model versions. No universal board support. Only a Chinese PDF exists; other languages are full translations. Source performance/latency is not ArmSoM measurement; no audio/NPU commands executed.

Source inconsistencies retained: an 8bit char result describes four statuses as bytes; SedBuzParam/RKAGCParam/RKSedEnable differ from later type names; EN_BCD examples differ from enum EN_SED. Recommended frm_len90 conflicts with buzzer_decision_len100 required to be below frm_len; verify headers and configure jointly. Formula dB=20*log10(X) omits reference amplitude while text defines32767 as0dB; verify normalization.

ArmSoM Docs · Rockchip Linux

1. Overview

This guide describes Sound Event Detection: Baby Cry Detection, Abnormal Event Detection and Buzz Detection. The three modules share an interface but operate independently; each can be enabled or disabled.

2. Functions

2.1 Abnormal Event Detection(AED)

AED detects abnormal sounds in real time: loud-sound level and signal-to-noise ratio, supporting 8k and 16k input. Loud-sound detection returns 1 above the configured dB threshold, otherwise 0. SNR detection distinguishes signal from steady environmental noise and recording noise floor, returning 1 above its threshold, otherwise 0. Decibels measure the ratio of quantities with identical units; amplitude is the absolute audio-data magnitude. The source relation is:

dB = 20 * log10(X)

Amplitude ratios correspond to dB addition/subtraction: doubling amplitude adds 6dB. Full-scale 16bit amplitude 32767 is defined as 0dB, so the dB values here are below 0dB. SNR is the signal/noise ratio: with noise at 0dB and signal 6dB higher, SNR is 6dB.

2.2 AI sound-event detectionSound Event Detection(SED)

SED detects baby crying, buzzer alarms and breaking glass in real time, using only 16k input. An RCNN model with multi-head attention analyzes about 1.5s of audio. The source reports better results above 6dB SNR.

2.2.1 Baby Cry Detection(BCD)

BCD uses deep learning to detect baby crying in real time. Results improve with high SNR; detection latency from cry onset is about 2s.

2.2.2 Buzz Detection(BUZ)

BUZ detects common alarms, including smoke, air-raid and burglar alarms, using deep learning. Higher SNR improves results; latency from alarm onset is about 2s.

2.2.3 Glass broken Detection(GBS)

GBS uses deep learning for real-time glass-break detection. Higher SNR improves results; latency from breaking-glass onset is about 0.6s.

3. APIs

APIs: rkaudio_sed_init initializes SED; rkaudio_sed_destroy destroys it; rkaudio_sed_process runs detection; rkaudio_sed_init_res returns initialization results.

3.1 rkaudio_sed_init

Description: initialize and return the SED handle used by rkaudio_sed_process. Destroy it with rkaudio_sed_destroy after use. Syntax: void *rkaudio_sed_init(int fs, int bit, int chan, RKAudioSedParam *param). Parameters:

Return value:

3.2 rkaudio_sed_destroy

Description: destroy the SED handle. Syntax: void rkaudio_sed_destroy(void *st_). Parameters:

Return value: none.

3.3 rkaudio_sed_init_res

Description: report SED-handle initialization results. Syntax: char rkaudio_sed_init_res(void* st_). Parameters:

Returns an 8bit char. From the low end, the original calls the first field a byte for agc, the second for aed, third for sed and fourth for fir; 1 means successful initialization and 0 failure. The byte terminology conflicts with an 8bit return type and requires SDK verification.

3.4 rkaudio_sed_process

Description: detect sound events and store results in res. Syntax: int rkaudio_sed_process(void *st_, short *in, int in_size, RKAudioSedRes *res). Parameters:

Return value:

3.5 rkaudio_sed_param_init

Initialize SED parameters, enable submodules and call their parameter initialization functions. Source code and default values are provided and may be adjusted for actual data. Destroy with rkaudio_sed_param_destroy after use. To change parameters or module enablement during operation, destroy and reinitialize SED. Syntax: RKAudioSedParam *rkaudio_sed_param_init(). Returns the parameter pointer; see RKAudioSedParam.

3.6 rkaudio_sed_param_destroy

Destroy SED parameters. Syntax: void rkaudio_sed_param_destroy(RKAudioSedParam *param). Parameter: SED parameter pointer.

3.7 rkaudio_sed_param_aed

Initialize AED parameters, called by rkaudio_sed_param_init and destroyed by rkaudio_sed_param_destroy. Syntax: SedAedParam *rkaudio_sed_param_aed(). Returns the AED parameter pointer.

3.8 rkaudio_sed_param

Initialize BCD/BUZ/GBS parameters, called by rkaudio_sed_param_init and destroyed by rkaudio_sed_param_destroy. Source syntax: SedBuzParam *rkaudio_sed_param(). Source return description: BUZ parameter pointer.

3.9 rkaudio_agc_param_init

Initialize AGC parameters within rkaudio_sed_param_init; destroyed by rkaudio_sed_param_destroy. Syntax: RKAGCParam* rkaudio_agc_param_init().

Returns the AGC parameter pointer.

3.10 rkaudio_sed_lsd_db

Return AED loud-sound level in dB. Syntax: float rkaudio_sed_lsd_db(void *st_). Return: dbfs value.

3.11 rkaudio_sed_bcd_model_set

Set the baby-cry RKNN model path. Syntax: int rkaudio_sed_bcd_model_set(char *model_path). Returns 0 on success, -1 on failure.

4. Parameters

4.1 JUMP_FRAME

SED detection interval in frames. Larger values reduce detections per second and computation. For 20 frames, the interval is

0.016*20 = 0.32s, roughly three detections per second. Recommended range: 15-25. Skipped frames retain the preceding result. Definition:

#define JUMP_FRAME 20

4.2 RKAudioSedParam

SED algorithm parameters. Definition:

typedef struct RKAudioSedParam_
{
    int model_en;
    RKSEDAGCParam *agc_param;
    SedAedParam   *aed_param;
    SedParam      *sed_param;
    RKFIRParam    *fir_param;
} RKAudioSedParam;

Members:

4.3 SedAedParam

AED algorithm parameters. Definition:

typedef struct SedAedParam_
{
    float    snr_db;
    float    lsd_db;
    int      policy;
    float    smooth_param;
} SedAedParam;

Members:

4.4 SedParam

BCD/BUZ/GBS parameters. Definition:

typedef struct SedParam_
{
    int      frm_len;
    int      nclass;
    int      babycry_decision_len;
    int      buzzer_decision_len;
    int      glassbreaking_decision_len;
    float    babycry_confirm_prob;
    float    buzzer_confirm_prob;
    float    glassbreaking_confirm_prob;
} SedParam;

Members:

4.5 RKAudioSedRes

SED result structure. Definition:

typedef struct RKAudioSedRes_
{
    int snr_res;
    int lsd_res;
    int bcd_res;
    int buz_res;
    int gbs_res;
} RKAudioSedRes;

Members:

4.6 RKAudioSedEnable

Assign to model_en to enable the corresponding module. Combine flags for multiple modules, e.g. EN_AED | EN_BCD as written in the source. Definition:

typedef enum RKAudioSedEnable_
{
    EN_AGC = 1 << 0,
    EN_AED = 1 << 1,
    EN_SED = 1 << 2,
    EN_FIR = 1 << 3,
} RKAudioSedEnable;

Members:

4.7 RKSEDAGCParam

AGC parameters; changes are not recommended. Definition:

typedef struct RKSEDAGCParam_
{
    float    attack_time;
    float    release_time;
    float    max_gain;
    float    max_peak;
    float    fRth0;
    float    fRk0;
    float    fRth1;
    int      fs;
    int      frmlen;
    float    attenuate_time;
    float    fRth2;
    float    fRk1;
    float    fRk2;
    float    fLineGainDb;
    int      swSmL0;
    int      swSmL1;
    int      swSmL2;
} RKSEDAGCParam;

Members:

4.8 RKFIRParam

FIR parameters; changes are not recommended. Definition:

typedef struct RKFIRParam_
{
    int fir_len;
    float* fir_coeffs;
} RKFIRParam;

Members: