Dmabuf mode for v4l2

2024-07-25 11:39

Hi everyone! Another chapter of my pico mini B research. I found part of rockchip mpp software in pico buildroot sdk (luckfox-pico/media/mpp/release_mpp_rv1106_arm-rockchip830-linux-uclibcgnueabihf). According to https://github.com/rockchip-linux/mpp readme file there is no support for rv1103 soc, But there is mpp_info_test and mpi_enc_test binaries that are working at my pico board. Obviously, after testing this binaries to work, i found rockchip mpp library (/oem/usr/lib/librockchip_mpp.so.0) at my board. According to their documentation https://github.com/rockchip-linux/mpp/b ... _MPP_EN.md, hardware encoder does not support space, allocated by CPU, but v4l2-ctl using --stream-mmap option, which means that all the pointers to actual raw image buffers are in user-space, and after calling

Code: Select all

 encode_put_frame()

function with this pointer my program crashes with this messages:

Now im trying to figure out how to configure v4l2 driver in DMABUF mode, and i have problems, such as:
1. Trying to configure it from my own application, according to linux kernel documentation https://docs.kernel.org/userspace-api/m ... tml#dmabuf fails on VIDIOC_EXPBUF system call.
2. v4l2-ctl tool working in dmabuf mode with command:

Code: Select all

v4l2-ctl -E11 --stream-dmabuf --stream-to=out.raw --stream-count=100 --set-fmt-video=width=480,height=320,pixelformat=NV21 --verbose --stream-poll

But after encoding raw file at my pc with ffmpeg with this command:

Code: Select all

ffmpeg -f rawvideo -pixel_format nv21 -framerate 25 -video_size 480x320 -i out.raw out.mp4 -loglevel debug

I have output video like that:

So now i'm pretty confused, if this method possibly can work? Can i use mpp library code at this board? If i can, how can i configure DMABUF mode for capturing? There is so many questions, and absolutely no answers in the internet, so i need help. Maybe someone can share experience in this area, as i am completely newbee in embedded linux and digital video. Thanks in advance!

2024-07-29 15:32

What i also tried is to capture video using V4L2 dirver kernel api. It's working fine in MMAP mode, but with DMABUF it fails on VIDIOC_EXPBUF system call. According to v4l2-ctl util in --verbose mode the sequence of ioctl's is:

Code: Select all

VIDIOC_REQBUFS returned 0 (Success)
		VIDIOC_QUERYBUF returned 0 (Success)
		VIDIOC_QUERYBUF returned 0 (Success)
		VIDIOC_QUERYBUF returned 0 (Success)
		VIDIOC_QUERYBUF returned 0 (Success)
		VIDIOC_REQBUFS returned 0 (Success)
		VIDIOC_QUERYBUF returned 0 (Success)
		VIDIOC_QUERYBUF returned 0 (Success)
		VIDIOC_QUERYBUF returned 0 (Success)
		VIDIOC_QUERYBUF returned 0 (Success)
		VIDIOC_EXPBUF returned 0 (Success)
		VIDIOC_EXPBUF returned 0 (Success)
		VIDIOC_EXPBUF returned 0 (Success)
		VIDIOC_EXPBUF returned 0 (Success)
		VIDIOC_QBUF returned 0 (Success)
		VIDIOC_QBUF returned 0 (Success)
		VIDIOC_QBUF returned 0 (Success)
		VIDIOC_QBUF returned 0 (Success)
		VIDIOC_STREAMON returned 0 (Success)

And looking at v4l2-utils source code (https://gitlab.collabora.com/dbrouwer/v ... ree/master) it is really doing something that it has to:

Code: Select all

	if (options[OptStreamDmaBuf]) {
		if (exp_q.reqbufs(&exp_fd, reqbufs_count_cap))
			goto done;
		if (q.export_bufs(&exp_fd, exp_fd.g_type()))
			goto done;
	}

Inside of reqbufs function there is VIDIOC_REQBUFS and VIDIOC_QUERYBUF ioctls:

Code: Select all

static inline int v4l_queue_reqbufs(struct v4l_fd *f,
		struct v4l_queue *q, unsigned count, unsigned int flags = 0)
{
	struct v4l2_requestbuffers reqbufs;
	int ret;

	reqbufs.type = q->type;
	reqbufs.memory = q->memory;
	reqbufs.count = count;
	reqbufs.flags = flags;
	/*
	 * Problem: if REQBUFS returns an error, did it free any old
	 * buffers or not?
	 */
	ret = v4l_ioctl(f, VIDIOC_REQBUFS, &reqbufs);
	if (ret)
		return ret;
	q->buffers = reqbufs.count;
	q->capabilities = reqbufs.capabilities;
	return v4l_queue_querybufs(f, q, 0);
}

Code: Select all

static inline int v4l_queue_querybufs(struct v4l_fd *f, struct v4l_queue *q, unsigned from)
{
	unsigned b, p;
	int ret;

	for (b = from; b < v4l_queue_g_buffers(q); b++) {
		struct v4l_buffer buf;

		v4l_buffer_init(&buf, v4l_queue_g_type(q), v4l_queue_g_memory(q), b);
		ret = v4l_ioctl(f, VIDIOC_QUERYBUF, &buf.buf);
		if (ret)
			return ret;
		if (b == 0) {
			q->num_planes = v4l_buffer_g_num_planes(&buf);
			for (p = 0; p < v4l_queue_g_num_planes(q); p++)
				q->lengths[p] = v4l_buffer_g_length(&buf, p);
		}
		if (q->memory == V4L2_MEMORY_MMAP)
			for (p = 0; p < q->num_planes; p++)
				q->mem_offsets[b][p] = v4l_buffer_g_mem_offset(&buf, p);
	}
	return 0;
}

Code: Select all

static inline int v4l_queue_export_bufs(struct v4l_fd *f, struct v4l_queue *q,
					unsigned exp_type)
{
	struct v4l2_exportbuffer expbuf;
	unsigned b, p;
	int ret = 0;

	expbuf.type = exp_type ? : f->type;
	expbuf.flags = O_RDWR;
	memset(expbuf.reserved, 0, sizeof(expbuf.reserved));
	for (b = 0; b < v4l_queue_g_buffers(q); b++) {
		expbuf.index = b;
		for (p = 0; p < v4l_queue_g_num_planes(q); p++) {
			expbuf.plane = p;
			ret = v4l_ioctl(f, VIDIOC_EXPBUF, &expbuf);
			if (ret)
				return ret;
			v4l_queue_s_fd(q, b, p, expbuf.fd);
		}
	}
	return 0;
}

But doing the same in my own application causes an error. Also v4l2-compliance say this thing about VIDIOC_EXPBUF ioctl that causes an error in my code:

Had anyone experienced this thing? There is such a feeling that i'm the only person working with camera on this board...

2024-07-31 7:04

Hello,
Although the SDK provides dynamic libraries for mpp, it's challenging to use them effectively in a real project with only the header files. Referencing the usage of the mpp library on other Rockchip platforms might lead to some progress, but currently, the mpi has more extensive examples and resources available that can serve as alternatives.

I have only tested using v4l2 to capture raw images, and there currently isn't a successful solution for involving the ISP or enabling dmabuf mode. On the RV1103/RV1106, memory allocation for the camera is handled through CMA, which differs from the CMA in standard Linux. This might be the reason your experiments haven't been successful. The unique design of the RV1106 in multimedia provides good performance, but it significantly increases the difficulty of operations that do not use Rockchip libraries.