Luckfox Forums

I followed the instructions in:
https://forums.luckfox.com/viewtopic.php?p=7829

to add hardware video encoding/decoding to the standard Debian Trixie image.

The test code pipeline:

gst-launch-1.0 videotestsrc num-buffers=600 ! video/x-raw,format=NV12,width=640,height=320,framerate=60/1 ! mpph264enc ! h264parse ! qtmux ! filesink location=h264.mp4

works!

However, when I alter it to encode from the camera:

v4l2src device="/dev/video13" ! video/x-raw,width=640,height=320,format=NV12,framerate=10/1 ! mpph264enc ! h264parse ! qtmux ! filesink sync=true location=/mnt/cam-h264.mp4

the framerate of the stream is 19.4fps.

I've tried using v4l2-ctl to change the framerate on the camera, but:

v4l2-ctl -d /dev/video13 --set-output-parm 10

responds with:

VIDIOC_S_PARM: failed: Inappropriate ioctl for device

The camera seems stuck at 19.4 frames per second.
Can anyone provide an example gstreamer pipeline to stream video at 10 frames per second?

I've figured out how to slow the frame rate by increasing the vertical blanking interval with:

v4l2-ctl -d /dev/v4l-subdev4 --set-ctrl vertical_blanking=200  #any value >58

although the system responds with:

VIDIOC_SUBDEV_S_CLIENT_CAP: failed: Inappropriate ioctl for device

it still performs the requested change change.

Is there any better way?

The IMX415 recommended for use with this board is limited to 19.4fps due to the processing limits of the RV1126B.

We do not need the sensor's full 4K resolution.
How does one configure the sensor for 2x2 binning to reduce its readout to HD
(1920x1080) resolution?

The maximum frame rate and resolution output by the sensor are fixed after being written into the driver's registers during initialization. The frame rate can be reduced by adjusting the blanking interval. For example, using the command
v4l2-ctl -d /dev/v4l-subdev4 --set-ctrl "vertical_blanking=2000"
can reduce the frame rate to around 10 frames per second.
You can use the command
v4l2-ctl -d /dev/v4l-subdev4 -L
to confirm that the setting has been successfully applied. The prompt
VIDIOC_SUBDEV_S_CLIENT_CAP: failed: Inappropriate ioctl for device
can be ignored.

If the ISP needs to be enabled, you can run the following command to let the service run in the background:
rkaiq_3A_server &
Then, modify the frmRate mode in /etc/iqfiles/imx415_default_default.json to ae_frmRate_fix_mode and set its value to 16(note that there are two frmRate entries; it is recommended to modify both). This will allow the ISP to fix the frame rate to around 10 frames per second.

If 1080P output is required, you can use the scaling function of GStreamer. If the camera needs to natively output 1920x1080, the driver configuration will need to be modified. For more details, refer to:
https://forums.luckfox.com/viewtopic.php?t=2577

We do need the native 1920x1080 image format and have modified the imx415.c driver accordingly.
The system now streams HD video at 38fps.
The vertical_blanking control is set to its minimum value of 1154.

Like the poster of https://forums.luckfox.com/viewtopic.php?t=2577
we would like to get the frame rate up to 60hz.

If we can stream 3840x2160 @19fps, I would expect to be able to steam 1920x1080 roughly 4 times the rate (which would be >70hz).

I suspect the MIPI clock rate is too low at 891Mhz.
Have anyone managed to get HD at 60fps with this hardware?