模型评估

RKNN提供(模拟器和板端)精度评估、耗时评估和内存评估的功能，辅助RKNN模型的优化和部署

模型精度评估

进入 rknn_model_zoo/examples/mobilenet/python 目录

cd rknn_model_zoo/examples/mobilenet/python

执行模型连板精度分析

python mobilenet.py --target rk3588 --accuracy_analysis --npu_device_test

模型连板精度分析的输出结果如下：

# simulator_error: calculate the output error of each layer of the simulator (compared to the 'golden' value).
# entire: output error of each layer between 'golden' and 'simulator', these errors will accumulate layer by layer.
# single: single-layer output error between 'golden' and 'simulator', can better reflect the single-layer accuracy of
the simulator.
layer_name simulator_error
entire single
cos euc cos euc
---------------------------------------------------------------
...
[Conv] 464 0.99202 | 4.1079 0.99998 | 0.1981
[Conv] output_conv 0.99308 | 13.235 0.99992 | 1.4133
[Reshape] output_int8 0.99308 | 13.235 0.99993 | 1.3043
[exDataConvert] output 0.99308 | 13.235 0.99993 | 1.3043
# runtime_error: calculate the output error of each layer of the runtime.
# entire: output error of each layer between 'golden' and 'runtime', these errors will accumulate layer by layer.
# single_sim: single-layer output error between 'simulator' and 'runtime', can better reflect the single-layer accuracy of
runtime.
layer_name runtime_error
entire single_sim
cos euc cos euc
---------------------------------------------------------------
...
[Conv] 464 0.99210 | 4.2718 1.00000 | 0.0
[Conv] output_conv 0.99203 | 14.847 1.00000 | 0.2007
[Reshape] output_int8
[exDataConvert] output 0.99203 | 14.847 1.00000 | 0.0

基于rknn_model_zoo/examples/yolov5/cpp进行改造，采用gstreamer采集usb摄像头数据，并通过yolov5模型进行实时目标检测。最后通过mediamtx将检测结果推流到rtmp服务器。

耗时评估

进入 rknn_model_zoo/examples/mobilenet/python 目录

cd rknn_model_zoo/examples/mobilenet/python

执行模型耗时评估

python mobilenet.py --target rk3588 --eval_perf

模型耗时评估的输出结果如下：

---------------------------------------------------------------
Network Layer Information Table
---------------------------------------------------------------
ID OpType DataType Target Time(us) ...
---------------------------------------------------------------
1 InputOperator UINT8 CPU 17
2 ConvClip UINT8 NPU 331
3 ConvClip INT8 NPU 429
4 Conv INT8 NPU 292
....
55 Conv INT8 NPU 374
56 Reshape INT8 CPU 61
57 OutputOperator INT8 CPU 11
---------------------------------------------------------------
Total Operator Elapsed Per Frame Time(us): 12631
Total Memory Read/Write Per Frame Size(KB): 10563
---------------------------------------------------------------
---------------------------------------------------------------
Operator Time Consuming Ranking Table
---------------------------------------------------------------
OpType CallNumber ... NPUTime(us) TotalTime(us) TimeRatio(%)
---------------------------------------------------------------
ConvClip 35 8436 8436 66.79%
Conv 9 2093 2093 16.57%
ConvAdd 10 2013 2013 15.94%
Reshape 1 0 61 0.48%
InputOperator 1 0 17 0.13%
OutputOperator 1 0 11 0.09%
---------------------------------------------------------------

内存评估

进入 rknn_model_zoo/examples/mobilenet/python 目录

cd rknn_model_zoo/examples/mobilenet/python

执行模型内存评估

python mobilenet.py --target rk3588 --eval_memory

模型内存评估的输出结果如下：

===================================
Memory Profile Info Dump
===================================
NPU model memory detail(bytes):
Weight Memory: 3.53 MiB
Internal Tensor Memory: 1.53 MiB
Other Memory: 377.19 KiB
Total Memory: 5.43 MiB
INFO: When evaluating memory usage, we need consider
the size of model, current model size is: 3.98 MiB
===================================

说明

更多资料，请参考文档《Rockchip_RKNPU_User_Guide_RKNN_SDK_V2.3.0_CN.pdf》