Hi,
I inspected the luckfox_pico_rtsp_yolov5 demo and it clearly uses OpenCV on CPU inside the main loop:
VI provides NV12 (YUV420sp) frame (RK_MPI_VI_GetChnFrame)
then it does CPU cv::cvtColor(NV12→BGR)
then CPU resize + letterbox + memcpy() into RKNN input (RGB)
then CPU overlay (rectangle, putText)
then it encodes RGB888 (not NV12) by copying frame.data into a DMA buffer and calling RK_MPI_VENC_SendFrame() with RK_FMT_RGB888
So this demo is not a “full hardware pipeline”, it is more like a visualization-friendly example.
Questions:
Why is OpenCV used here? Is it only for simplicity, or because a full hardware path is not supported / not recommended on RV1106?
Is there an official/reference demo showing a true HW pipeline:
VI(NV12) → VPSS
VPSS channel A → NPU inference (preferably NV12/zero-copy if possible)
VPSS channel B → VENC H264 (NV12)
then RTSP stream
Thanks for your help
Why does luckfox_pico_rtsp_yolov5 use OpenCV (CPU) instead of full HW? Any official full-HW demo ?
Hello, our sample does not aim for high performance. It is mainly designed to help customers with limited experience in using the Rockchip library to quickly get started. Introducing OpenCV-Mobile can lower the learning threshold. Meanwhile, there is currently a bug in VPSS, which prevents images from being retrieved from the memory cache pool. Images can only be passed to the next stage（such as:venc）.
Currently, the most ideal way to implement real-time YOLOv5 using rkmpi + rknn is to utilize multi-threading + librga + zero-copy. One thread uses VI + VENC + RTSP streaming, while the other thread uses VI to convert the format to RGB888 via librga and input it into RKNPU. VGN binds VENC to display the detection results of YOLOv5 in the form of OSD. If you are interested, you can refer to the implementation part of YOLOv5 in https://github.com/luckfox-eng29/kvm_video.
Currently, the most ideal way to implement real-time YOLOv5 using rkmpi + rknn is to utilize multi-threading + librga + zero-copy. One thread uses VI + VENC + RTSP streaming, while the other thread uses VI to convert the format to RGB888 via librga and input it into RKNPU. VGN binds VENC to display the detection results of YOLOv5 in the form of OSD. If you are interested, you can refer to the implementation part of YOLOv5 in https://github.com/luckfox-eng29/kvm_video.