关于YOLOV5模型

2024-10-13 13:41

请问这个问题该如何解决在yolov5模型中时，使用luckfox_pico_rtsp_yolov5历程，我将模型更换为自己训练的模型，已经通过luckfox_rknn将我自己训练的yolov5s模型转换为rknn格式，我将luckfox_pico_rtsp_yolov5/model目录下的yolov5.rknn和yolov5s_relu.onnx换成我自己的并且修改了coco_80_labels_list.txt文件

与include/rknn目录下的postprocess.h文件

但是在便于时出现了警告

2024-10-14 1:16

您好，请问您使用的是什么型号的luckfox pico？根据打印的log来看应该设计内存分配问题，可以通过增大板级支持文件中的RK_BOOTARGS_CMA_SIZE 或降低推流分辨率来解决

2024-10-14 8:58

您好我使用的是Luckfox Pico Max Max 256MB的板子系统是Buildroot 我尝试修改了main.c文件中的

但是还是出现一样的问题
请问我是应该修改这个吗，我将他修改为99M 无效果

Code: Select all

[root@luckfox luckfox_rtsp_yolov5_demo]# ./luckfox_rtsp_yolov5
E RKNN: unsupport cpu Pow op, op name: Pow:Pow_236 in current, please try updating to the latest version of the toolkit2 and runtime from: https://console.zbox.filez.com/l/I00fc3 (PWD: rknn)
E RKNN: rknn fallback cpu failed
rknn_init fail! ret=-1
init rknn model success!
load lable ./model/coco_80_labels_list.txt
rkaiq log level ff0
ID: 0, sensor_name is m00_b_sc3336 4-0030, iqfiles is /etc/iqfiles
rk_aiq_uapi2_sysctl_init/prepare succeed
rk_aiq_uapi2_sysctl_start succeed
rockit log path (null), log_size = 0, can use export rt_log_path=, export rt_log_size= change
log_file = (nil)
RTVersion        03:31:51-681 {dump              :064} ---------------------------------------------------------
RTVersion        03:31:51-684 {dump              :065} rockit version: git-8cb4d25b8 Tue Feb 28 11:12:39 2023 +0800
RTVersion        03:31:51-686 {dump              :066} rockit building: built- 2023-02-28 15:23:19
RTVersion        03:31:51-686 {dump              :067} ---------------------------------------------------------
(null)           03:31:51-686 {log_level_init    :203}

 please use echo name=level > /tmp/rt_log_level set log level
        name: all cmpi mb sys vdec venc rgn vpss vgs tde avs wbc vo vi ai ao aenc adec
        log_level: 0 1 2 3 4 5 6

rockit default level 4, can use export rt_log_level=x, x=0,1,2,3,4,5,6 change
(null)           03:31:51-687 {read_log_level    :093} text is all=4
(null)           03:31:51-687 {read_log_level    :095} module is all, log_level is 4
RTIsp3x          03:31:51-691 {ispInitDevice     :208} sensor name = m00_b_sc3336 4-0030
RTIsp3x          03:31:51-692 {ispInitDevice     :211} sensor_index = 0
(null)           03:31:51-695 {monitor_log_level :144} #Start monitor_log_level thread, arg:(nil)
RTIsp3x          03:31:51-698 {ispInitDevice     :208} sensor name = m00_b_sc3336 4-0030
RTIsp3x          03:31:51-699 {ispInitDevice     :211} sensor_index = 0
[INFO  rtsp_demo.c:280:rtsp_new_demo] rtsp server demo starting on port 554
[DEBUG rtsp_demo.c:480:rtsp_new_session] add session path: /live/0
rtsp streamq_alloc size is 1753484
vi_dev_init
RKViDev          03:31:51-709 {vi_set_dev_attr   :440} VI_DEV_ATTR_S all parameter reserved
  === VI ATTRS INFO: ===
        devId  :      0
        chnId  :      0
        buffcnt:      0
        buffSize:     0
        width:        0
        height:       0
        Maxwidth:     0
        Maxwidth:     0
        streaming:    0
RKViChn          03:31:51-712 {prepareRuntime    :334} ---------------------------------------------------------
RKViChn          03:31:51-712 {prepareRuntime    :336} vi version: 1.86, name:vvi
RKViChn          03:31:51-713 {prepareRuntime    :337} rockit-ko version: vmpi:fb2eed2be49e
RKViChn          03:31:51-713 {prepareRuntime    :338} rockit-ko building: -2023-02-09-11:04:49
RKViChn          03:31:51-713 {prepareRuntime    :339} ---------------------------------------------------------
RKViChn          03:31:51-713 {prepareRuntime    :358} mb pool create success, MBCnt= 2
cmpi             03:31:51-714 {createRuntime     :546} [non-WRAP MODE]: buff size = 132480
RTIsp3x          03:31:51-716 {ispInitDevice     :208} sensor name = m00_b_sc3336 4-0030
RTIsp3x          03:31:51-717 {ispInitDevice     :211} sensor_index = 0
RTDeviceV4L2     03:31:51-720 {open              :138} open video name(/dev/video11)
RKViChn          03:31:51-722 {setFrameRate      :1153} [vi] dev(0) ch(0) illegal param s32SrcFrameRate(0) s32DstFrameRate(0)
RTDeviceV4L2     03:31:51-723 {ispCameraInfo     :549} current device:/dev/video11 isn't compatible(cap:0x84201000) device,memoryType:4, retry:0
RTDeviceV4L2     03:31:51-724 {ispInitFormat     :726} ioctl VIDIOC_S_FMT OK
RKViChn          03:31:51-725 {start             :813} =========== vi Start startRuntime ===========
RTDeviceV4L2     03:31:51-726 {ispStreamOn       :440} do ispStreamOn start
RTDeviceV4L2     03:31:51-745 {ispStreamOn       :493} do ispStreamOn done
vpss_init
=== 0 ===
rga_api version 1.10.0_[2]
====RK_MPI_SYS_Bind vi0 to vpss0====
venc_init
MpiAdapterEncode 03:31:51-764 {ma_encode_create  :1653} ringbuf buf_size 131072
venc init success
=== 1 ===
cmpi             03:31:51-827 {mb_get_buffer_by_i:422} allocated buffer(this=0x2da2a0, data=(nil), size=0, id=-1)
Segmentation fault (core dumped)
[root@luckfox luckfox_rtsp_yolov5_demo]#

请问我更换为SD卡版本的系统会有帮助呢

2024-10-14 9:25

Code: Select all

E RKNN: unsupport cpu Pow op, op name: Pow:Pow_236 in current, please try updating to the latest version of the toolkit2 and runtime from: https://console.zbox.filez.com/l/I00fc3 (PWD: rknn)
E RKNN: rknn fallback cpu failed
rknn_init fail! ret=-1

根据运行的提示信息来看rknn初始化阶段就出问题了，提示的信息是不支持Pow算子，应该是训练阶段时出现的问题，请参考rknn_model_zoo的说明进行模型的训练，在最后转换为ONNX时rockchip有提供一个脚本进行处理，将模型的后处理部分去掉在使用例程时在cpu上运行，避免算子不兼容的情况

2024-10-14 10:30

您好请问这个问题是出在我训练模型的时候还是我pt转onnx时或者是onnx转rknn呢

2024-10-14 10:49

zzzzzz wrote: ↑2024-10-14 10:30 您好请问这个问题是出在我训练模型的时候还是我pt转onnx时或者是onnx转rknn呢

这个根据已知信息我无法判断，您可以用Netron查看一下ONNX的网络结构，对比一下和官方的差别，onnx转rknn出现问题的概率较小
rockchip 官方提供的 ONNX 获取方法

Code: Select all

wget -O ./yolov5s_relu.onnx https://ftrg.zbox.filez.com/v2/delivery/data/95f00b0fc900458ba134f8b180b3f7a1/examples/yolov5/yolov5s_relu.onnx

2024-10-14 13:05

您好我按照RKNN推理测试的过程走了一遍放入我的模型后还是存在问题如下

Code: Select all

[root@luckfox rknn_yolov5_demo]# ./rknn_yolov5_demo model/yolov5.rknn model/bus.jpg
load lable ./model/coco_80_labels_list.txt
model input num: 1, output num: 3
input tensors:
  index=0, name=images, n_dims=4, dims=[1, 640, 640, 3], n_elems=1228800, size=1228800, fmt=NHWC, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922
output tensors:
  index=0, name=output0, n_dims=4, dims=[1, 80, 80, 48], n_elems=307200, size=307200, fmt=NHWC, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922
  index=1, name=367, n_dims=4, dims=[1, 40, 40, 48], n_elems=76800, size=76800, fmt=NHWC, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922
  index=2, name=369, n_dims=4, dims=[1, 20, 20, 48], n_elems=19200, size=19200, fmt=NHWC, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922
input_attrs[0].size_with_stride=1228800
model is NHWC input fmt
model input height=640, width=640, channel=3
origin size=640x640 crop size=640x640
input image: 640 x 640, subsampling: 4:2:0, colorspace: YCbCr, orientation: 1
scale=1.000000 dst_box=(0 0 639 639) allow_slight_change=1 _left_offset=0 _top_offset=0 padding_w=0 padding_h=0
rga_api version 1.10.1_[0]
rknn_run
Segmentation fault (core dumped)

请问会是什么问题呢是我的模型训练存在问题吗

2024-10-15 1:44

从日志上看模型加载成功了，出现段错误一般是数据非法读取写入，我无法判断哪里出现问题。
模型的输入数据就算错误输入了只要满足输入大小的要求最后也会有推理结果，大概率是模型输出的后处理出现的问题，这个请自行使用调试方法判断了。

2024-10-15 1:48

Crocodile wrote: ↑2024-10-15 1:44 从日志上看模型加载成功了，出现段错误一般是数据非法读取写入，我无法判断哪里出现问题。
模型的输入数据就算错误输入了只要满足输入大小的要求最后也会有推理结果，大概率是模型输出的后处理出现的问题，这个请自行使用调试方法判断了。

好的谢谢您了

2024-10-17 8:57

您好我找到了那个出错的函数但是我不知道该怎么解决请问可以给点建议吗函数是postproess.cc中的process_i8_rv1106函数

我把它屏蔽了之后令validCount=20就有结果输出了但是图片就没有标注请问我该怎么做