RV1106 RKMPI video overlay

2025-10-16 21:06

Hello! I'm having a problem setting up a video overlay on my Luckfox Pico Max. I have two cameras connected to the board, and I can get video stream from both of them using VI-VPSS-VENC pipeline, I am also able to send frames from both cameras VENC streams to RTSP and receive both RTSP streams on my PC.

However I can't quite figure out how to make a video overlay from one camera stream to another. In other words what I'm trying to do is to insert footage from the first camera into the video stream of the second camera so that in the lower right corner of the first camera's video stream I can see the image from the other camera.

So far I've tried using RGN overlay like in simple_vi_bind_venc_osd.c example, but instead of loading image from file, in a separate pthread I'm trying to:
- capture a frame using RK_MPI_VENC_GetStream()
- get it's virtual address using pData = RK_MPI_SYS_Handle2VirAddress()
- write frame data using memcpy(stBitmap.pData, pData, width*height*4) (I'm using ARGB888 format)
- reload overlay by using RK_MPI_RGN_SetBitmap()
- release stream using RK_MPI_VENC_ReleaseStream()
but when I connect to my RTSP stream I can only see a video from main camera and no overlay.

Can someone please give me some suggestions on how can I achieve the desired overlay on RTSP stream? Thank you in advance!

2025-10-17 7:12

I'm not sure if it's feasible since I haven't tried it myself. But I previously noticed that the `imblend` function in RGA can achieve image overlay. For details, you can refer to the official documentation: https://github.com/airockchip/librga/bl ... _RGA_EN.md

2025-10-17 11:24

Hello, Pointer! Thank you fro your reply, I will definitely check RGA later, it looks like it has many useful features that I can use in my project)

I have solved the issue with overlay using RGN. I made a pretty dumb mistake: I was receiving frame data from coded VENC stream of the second camera

Of course it has a totally different output and isn't compatible with my RGN overlay! I switched to receiving frame data from VPSS using RK_MPI_VPSS_GetChnFrame(), and now everything works fine! My current pipeline looks like this:

1st camera: Sensor->VI->VPSS->VENCChn1-RTSP
---------------------------------------|->VENCChn2->RTSP
-------------------------------|->GetChnFrame->^
2nd cam: Sensor->VI-VPSS->VENCChn3->RTSP

So now I have 3 RTSP streams: one for first cam, one for second cam and one for combined video (full scale video from first camera and cropped video from second camera in the bottom right corner).