Edge AI SDK/AI Framework/RK3588 - Revision history

Zhihao.zhu: /* Applications */

2025-09-22T00:39:32Z

‎Applications

Eric.liang at 07:48, 9 July 2025

2025-07-09T07:48:04Z

Zhihao.zhu at 01:04, 9 July 2025

2025-07-09T01:04:38Z

Zhihao.zhu at 08:06, 7 July 2025

2025-07-07T08:06:27Z

Zhihao.zhu at 07:51, 7 July 2025

2025-07-07T07:51:41Z

Zhihao.zhu at 05:52, 7 July 2025

2025-07-07T05:52:54Z

Zhihao.zhu: Created page with " = Requirement = {| border="1" cellpadding="1" cellspacing="1" style="width: 900;" |- | style="width: 100px;" | Device | style="width: 250px;" | EMMC (recommend physical size..."

2025-07-07T05:08:57Z

Created page with " = Requirement = {| border="1" cellpadding="1" cellspacing="1" style="width: 900;" |- | style="width: 100px;" | Device | style="width: 250px;" | EMMC (recommend physical size..."

New page

= Requirement =

{| border="1" cellpadding="1" cellspacing="1" style="width: 900;"
|-
| style="width: 100px;" | Device
| style="width: 250px;" | EMMC (recommend physical size)
| style="width: 250px;" | Full Installed Size
|-
| style="width: 100px;" | ASR-A501
| style="width: 250px;" | 64 GB
| style="width: 250px;" | 135 MB
|-
| style="width: 100px;" | AOM-3821
| style="width: 250px;" | 32 GB
| style="width: 250px;" | 135 MB
|}

= Version =

{| border="1" cellpadding="1" cellspacing="1" style="width: 1050;"
|-
| style="width: 100px;" | Device
| style="width: 150px;" | EdgeAISDK
| style="width: 300px;" | OS
| style="width: 500px;" | Support Framework Version
|-
| style="width: 100px;" | AOM-3821
| style="width: 150px;" | v3.3.1
| style="width: 300px;" | Debian 12 kernel:6.1.75
| style="width: 500px;" |  
|-
| style="width: 100px;" | ASR-A501
| style="width: 150px;" | v3.3.1
| style="width: 300px;" | Debian 12 kernel:6.1.75
| style="width: 500px;" |  
|}

= RKNPU2 =

NVIDIA JetPack SDK is the most comprehensive solution for building end-to-end accelerated AI applications. JetPack provides a full development environment for hardware-accelerated AI-at-the-edge development on Nvidia Jetson modules. JetPack includes Jetson Linux with bootloader, Linux kernel, Ubuntu desktop environment, and a complete set of libraries for acceleration of GPU computing, multimedia, graphics, and computer vision. It also includes samples, documentation, and developer tools for both host computer and developer kit, and supports higher level SDKs such as DeepStream for streaming video analytics, Isaac for robotics, and Riva for conversational AI.

More Info refer to [https://developer.nvidia.com/embedded/jetpack https://developer.nvidia.com/embedded/jetpack]

JetPack [https://developer.nvidia.com/embedded/jetpack-sdk-512 5.1.2] includes CUDA 11.4.19, TensorRT 8.5.2, DeepStream 6.2, 

= RKNN =

[https://developer.nvidia.com/tensorrt TensorRT] is a high performance deep learning inference runtime for image classification, segmentation, and object detection neural networks. TensorRT is built on CUDA, NVIDIA’s parallel programming model, and enables you to optimize inference for all deep learning frameworks. It includes a deep learning inference optimizer and runtime that delivers low latency and high-throughput for deep learning inference applications.

 

= Applications =

== Edge AI SDK / Vision Application ==

{| border="1" cellpadding="1" cellspacing="1" style="width: 900px;"
|-
| style="width: 154px;" | Application
| style="width: 179px;" | Model
| style="width: 179px;" | AOM-3821 FPS (video file)
| style="width: 179px;" | ASR-A501 FPS (video file)
|-
| style="width: 154px;" | Object Detection
| style="width: 154px;" | yolov10
| style="width: 154px;" | 30
| style="width: 154px;" | 30
|-
| style="width: 154px;" | Person Detection
| style="width: 154px;" | yolov10
| style="width: 154px;" | 30
| style="width: 154px;" | 30
|-
| style="width: 154px;" | Face Detection
| style="width: 154px;" | retinaface
| style="width: 154px;" | 30
| style="width: 154px;" | 30
|-
| style="width: 154px;" | Pose Estimation
| style="width: 154px;" | yolov8_pose
| style="width: 154px;" | 30
| style="width: 154px;" | 30
|}

 

== Edge AI SDK / GenAI Application ==

{| border="1" cellpadding="1" cellspacing="1" style="width: 500px;"
|-
| style="width: 154px;" | Version
| style="width: 154px;" | Application
| style="width: 179px;" | Model
| style="width: 179px;" | Note
|-
| style="width: 154px;" | v3.0.0
| style="width: 154px;" | LLM Chatbot
| style="width: 154px;" | Llama-2-7b
| style="width: 154px;" |  
|-
| style="width: 154px;" | v3.3.0
| style="width: 154px;" | GenAI Chatbot
| style="width: 154px;" | Gemma3:4b
| style="width: 154px;" | Refer to [[Edge_AI_SDK/GenAIChatbot|Link]]
|}

= Benchmark =

Jetson is used to deploy a wide range of popular DNN models and ML frameworks to the edge with high performance inferencing, for tasks like real-time classification and object detection, pose estimation, semantic segmentation, and natural language processing (NLP). More Info refer to [ [https://developer.nvidia.com/embedded/jetson-benchmarks https://developer.nvidia.com/embedded/jetson-benchmarks] ] More Info refer to [ [https://github.com/NVIDIA-AI-IOT/jetson_benchmarks https://github.com/NVIDIA-AI-IOT/jetson_benchmarks] ]

cd /opt/Advantech/EdgeAISuite/Jetson/benchmark ./exec_benchmark.sh [[File:EAS Jetson Benchmark S1.png|600px|EAS Jetson Benchmark S1.png]]

= Tegrastats Utility =

This SDK provides the tegrastats utility, which reports memory usage and processor usage for Tegra-based devices. You can find the utility in your package at the following location.

More Info refer to [https://docs.nvidia.com/drive/drive_os_5.1.6.1L/nvvib_docs/index.html#page/DRIVE_OS_Linux_SDK_Development_Guide/Utilities/util_tegrastats.html Link]  

[[File:EAS Jetson SystemInfo Tegrastats.png|600px|EAS Jetson SystemInfo Tegrastats.png]]

@@ Line 60: / Line 60: @@
 |-
 | style="width: 154px;" | Person Detection
-| style="width: 154px;" | yolov10
+| style="width: 154px;" | yolov5
 | style="width: 154px;" | 25
 | style="width: 154px;" | 25

@@ Line 1: / Line 1: @@
 = RKNN SDK =
@@ Line 120: / Line 83: @@
 Advantech, based on the rknn_common_test command, has encapsulated the npu_stress_test.sh to be used for testing the NPU.
 = Utility =
-*'''sysstat''': This SDK provides the mpstat utility, which reports memory usage and processor usage for devices. You can find the utility in your package at the following location. More Info refer to&nbsp;[https://github.com/sysstat/sysstat Link]<br/> &nbsp;<br>
-*'''OpenCV4.6''': OpenCV (Open Source Computer Vision Library: http://opencv.org) is an open-source library that includes several hundreds of computer vision algorithms. More Info refer to&nbsp;[https://github.com/opencv/opencv/wiki/OpenCV-Change-Logs-v2.2%E2%80%90v4.10 Link]<br/> &nbsp;
+*'''sysstat''': This SDK provides the mpstat utility, which reports memory usage and processor usage for devices. You can find the utility in your package at the following location. More Info refer to&nbsp;[https://github.com/sysstat/sysstat Link]<br/> &nbsp;

@@ Line 9: / Line 9: @@
 |-
 | style="width: 100px;" | ASR-A501
-| style="width: 250px;" | 64 GB
+| style="width: 250px;" | 16 GB+
 | style="width: 250px;" | 135 MB
 |-
 | style="width: 100px;" | AOM-3821
-| style="width: 250px;" | 32 GB
+| style="width: 250px;" | 16 GB+
 | style="width: 250px;" | 135 MB
 |}
@@ Line 120: / Line 120: @@
 Advantech, based on the rknn_common_test command, has encapsulated the npu_stress_test.sh to be used for testing the NPU.
 = Utility =
+*'''sysstat''': This SDK provides the mpstat utility, which reports memory usage and processor usage for devices. You can find the utility in your package at the following location. More Info refer to&nbsp;[https://github.com/sysstat/sysstat Link]<br/> &nbsp;<br>
-This SDK provides the mpstat utility, which reports memory usage and processor usage for devices. You can find the utility in your package at the following location. More Info refer to&nbsp;[https://github.com/sysstat/sysstat Link]<br/> &nbsp;
+*'''OpenCV4.6''': OpenCV (Open Source Computer Vision Library: http://opencv.org) is an open-source library that includes several hundreds of computer vision algorithms. More Info refer to&nbsp;[https://github.com/opencv/opencv/wiki/OpenCV-Change-Logs-v2.2%E2%80%90v4.10 Link]<br/> &nbsp;

@@ Line 37: / Line 37: @@
 |}
-== RKNN SDK ==
+= RKNN SDK =
 RKNN SDK （[https://pan.baidu.com/s/1R0DhNJU56Uhp4Id7AzNfgQ Baidu] Password: a887）include two parts:
-*rknpu2  (on the Board End)
-*rknn-toolkit2  (on the PC)
+*rknpu2 (on the Board End)
-<pre>
+*rknn-toolkit2 (on the PC)
-├── rknn-toolkit2
+<pre>├── rknpu2
-│   ├── doc
+│   ├── Driver
-│   ├── examples
+│   └── RKNPU2 Environment
-│   ├── packages
+└── rknn-toolkit2
 </pre>
-= '''rknpu2''' =
+== RKNPU2 ==
-'rknpu2' include driver and runtime-lib to help to fast develop AI applications using rknn model(*.rknn).
+=== RKNPU2 Environment ===
 Here are two basic concepts in the RKNPU2 environment:
-More Info refer to&nbsp;[https://developer.nvidia.com/embedded/jetpack https://developer.nvidia.com/embedded/jetpack]
+*'''RKNN Server''': A background proxy service running on the development board. The main function of this service is to call the interface corresponding to the board end Runtime to process the data transmitted by the computer through USB, and return the processing results to the computer.
-JetPack&nbsp;[https://developer.nvidia.com/embedded/jetpack-sdk-512 5.1.2]&nbsp;includes CUDA 11.4.19, TensorRT 8.5.2, DeepStream 6.2,&nbsp;
-= '''rknn-toolkit2''' =
+== RKNN-TOOLKIT2 ==
 RKNN-Toolkit2 is a development kit that provides users with model conversion, inference and performance evaluation on PC platforms. Users can easily complete the following functions through the Python interface provided by the tool:
 #<span style="color:#0000ff;">'''Model conversion'''</span>: support to convert Caffe / TensorFlow / TensorFlow Lite / ONNX / Darknet / PyTorch model to RKNN model, support RKNN model import/export, which can be used on Rockchip NPU platform later.
 #<span style="color:#0000ff;">'''Quantization'''</span>: support to convert float model to quantization model, currently support quantized methods including asymmetric quantization (asymmetric_quantized-8). and support hybrid quantization.
@@ Line 78: / Line 74: @@
 #<span style="color:#0000ff;">'''Performance & Memory evaluation'''</span>: distribute the RKNN model to the specified NPU device to run, and evaluate the model performance and memory consumption in the actual device.
 #<span style="color:#0000ff;">'''Quantitative error analysis'''</span>: This function will give the Euclidean or cosine distance of each layer of inference results before and after the model is quantized. This can be used to analyze how quantitative error occurs, and provide ideas for improving the accuracy of quantitative models.
 #<span style="color:#0000ff;">'''Model encryption'''</span>: Use the specified encryption method to encrypt the RKNN model as a whole.
 &nbsp;
@@ Line 96: / Line 93: @@
 | style="width: 154px;" | Object Detection
 | style="width: 154px;" | yolov10
-| style="width: 154px;" | 30
+| style="width: 154px;" | 25
-| style="width: 154px;" | 30
+| style="width: 154px;" | 25
 |-
 | style="width: 154px;" | Person Detection
 | style="width: 154px;" | yolov10
-| style="width: 154px;" | 30
+| style="width: 154px;" | 25
-| style="width: 154px;" | 30
+| style="width: 154px;" | 25
 |-
 | style="width: 154px;" | Face Detection
 | style="width: 154px;" | retinaface
-| style="width: 154px;" | 30
+| style="width: 154px;" | 25
-| style="width: 154px;" | 30
+| style="width: 154px;" | 25
 |-
 | style="width: 154px;" | Pose Estimation
 | style="width: 154px;" | yolov8_pose
-| style="width: 154px;" | 30
+| style="width: 154px;" | 25
-| style="width: 154px;" | 30
+| style="width: 154px;" | 25
 |}
 &nbsp;
 = Benchmark =
-Jetson is used to deploy a wide range of popular DNN models and ML frameworks to the edge with high performance inferencing, for tasks like real-time classification and object detection, pose estimation, semantic segmentation, and natural language processing (NLP).<br/> More Info refer to&nbsp;[ [https://developer.nvidia.com/embedded/jetson-benchmarks https://developer.nvidia.com/embedded/jetson-benchmarks] ]<br/> More Info refer to&nbsp;[ [https://github.com/NVIDIA-AI-IOT/jetson_benchmarks https://github.com/NVIDIA-AI-IOT/jetson_benchmarks] ]
+RK3588 is used to deploy a wide range of popular DNN models and ML frameworks to the edge with high performance inferencing, for tasks like real-time classification and object detection, pose estimation, semantic segmentation, and natural language processing (NLP). More Info refer to&nbsp;[ [https://github.com/airockchip https://github.com/airockchip] ]
-= Tegrastats Utility =
+Advantech, based on the rknn_common_test command, has encapsulated the npu_stress_test.sh to be used for testing the NPU.
-This SDK provides the tegrastats utility, which reports memory usage and processor usage for Tegra-based devices. You can find the utility in your package at the following location.
+cd /opt/Advantech/EdgeAISuite/Rockchip/benchmark<br/> ./exec_benchmark.sh<br/> &nbsp;
-More Info refer to&nbsp;[https://docs.nvidia.com/drive/drive_os_5.1.6.1L/nvvib_docs/index.html#page/DRIVE_OS_Linux_SDK_Development_Guide/Utilities/util_tegrastats.html Link]<br/> &nbsp;
+= Utility =
-[[File:EAS Jetson SystemInfo Tegrastats.png|600px|EAS Jetson SystemInfo Tegrastats.png]]
+This SDK provides the mpstat utility, which reports memory usage and processor usage for devices. You can find the utility in your package at the following location. More Info refer to&nbsp;[https://github.com/sysstat/sysstat Link]<br/> &nbsp;

@@ Line 34: / Line 34: @@
 | style="width: 150px;" | v3.3.1
 | style="width: 300px;" | Debian 12 kernel:6.1.75
-| style="width: 500px;" | &nbsp;
+| style="width: 500px;" | librknnrt version: 2.0.0b0、 rknpu 0.9.7&nbsp;
 |}
-= RKNPU2 =
+== RKNN SDK ==
-NVIDIA JetPack SDK is the most comprehensive solution for building end-to-end accelerated AI applications. JetPack provides a full development environment for hardware-accelerated AI-at-the-edge development on Nvidia Jetson modules. JetPack includes Jetson Linux with bootloader, Linux kernel, Ubuntu desktop environment, and a complete set of libraries for acceleration of GPU computing, multimedia, graphics, and computer vision. It also includes samples, documentation, and developer tools for both host computer and developer kit, and supports higher level SDKs such as DeepStream for streaming video analytics, Isaac for robotics, and Riva for conversational AI.
+= '''rknpu2''' =
 More Info refer to&nbsp;[https://developer.nvidia.com/embedded/jetpack https://developer.nvidia.com/embedded/jetpack]
-JetPack&nbsp;[https://developer.nvidia.com/embedded/jetpack-sdk-512 5.1.2]&nbsp;includes CUDA 11.4.19, TensorRT 8.5.2, DeepStream 6.2,&nbsp;
+= '''rknn-toolkit2''' =
-= RKNN =
+RKNN-Toolkit2 is a development kit that provides users with model conversion, inference and performance evaluation on PC platforms. Users can easily complete the following functions through the Python interface provided by the tool:
 &nbsp;