USB-A abnormal behavior

2025-03-04 15:43

Hi,

When usb camera is attached, and relevant modules are loaded no video source is detected. However I can see the usb device in LSUSB stdout.

Code: Select all

[root@luckfox root]# insmod /oem/usr/ko/gspca_main.ko 
[root@luckfox root]# insmod /oem/usr/ko/gspca_ov534.ko 

[root@luckfox root]# lsusb
Bus 001 Device 001: ID 1d6b:0002
Bus 001 Device 003: ID 1415:2000
Bus 002 Device 001: ID 1d6b:0003

[root@luckfox root]# v4l2-ctl --list-devices
rkisp-statistics (platform: rkisp):
	/dev/video19
	/dev/video20

rkcif-mipi-lvds (platform:rkcif):
	/dev/media0

rkisp_mainpath (platform:rkisp-vir0):
	/dev/video11
	/dev/video12
	/dev/video13
	/dev/video14
	/dev/video15
	/dev/video16
	/dev/video17
	/dev/video18
	/dev/media1

Failed to open /dev/video0: No such device

After removing both usb and ethernet cable, and attaching the usb cable first and ethernet cable after, video device is available. The board is powered via lab power supply @ 5VDC on POW, PWR pins. How can I troubleshoot the issue?

Code: Select all

[root@luckfox root]# v4l2-ctl --list-devices
...
...
USB Camera-B4.09.24.1 (usb-xhci-hcd.0.auto-1):
	/dev/video21

Failed to open /dev/video0: No such device

I'm not sure if this is related but I also observe abnormal behavior using ffmpeg with this specific usb device.

ffmpeg throws same info on STDOUT, at the same time kernel messages also indicate frame overflow (ov534 1-1:1.0: frame overflow 614872 > 614400).

Code: Select all


frame=  355 fps= 26 q=29.0 size=     675kB time=00:00:13.30 bitrate= 415.5kbits/s s
frame=  370 fps= 26 q=23.1 size=     693kB time=00:00:13.80 bitrate= 411.5kbits/s s
frame=  386 fps= 27 q=16.0 size=     721kB time=00:00:14.33 bitrate= 411.9kbits/s s
frame=  400 fps= 27 q=23.7 size=     738kB time=00:00:14.80 bitrate= 408.4kbits/s s
frame=  416 fps= 27 q=23.2 size=     757kB time=00:00:15.33 bitrate= 404.2kbits/s s
frame=  431 fps= 27 q=25.4 size=     775kB time=00:00:15.86 bitrate= 399.9kbits/s s
frame=  447 fps= 27 q=31.0 size=     803kB time=00:00:16.40 bitrate= 401.0kbits/s s
frame=  462 fps= 27 q=28.7 size=     821kB time=00:00:16.90 bitrate= 397.8kbits/s s
frame=  478 fps= 27 q=29.4 size=     839kB time=00:00:17.43 bitrate= 394.3kbits/s s
frame=  494 fps= 27 q=21.4 size=     865kB time=00:00:17.96 bitrate= 394.4kbits/s s
frame=  508 fps= 27 q=25.8 size=     881kB time=00:00:18.46 bitrate= 391.0kbits/s s
frame=  524 fps= 27 q=26.3 size=     899kB time=00:00:19.00 bitrate= 387.4kbits/s s
frame=  537 fps= 27 q=26.1 size=     914kB time=00:00:19.43 bitrate= 385.2kbits/s s
[video4linux2,v4l2 @ 0x556960] Dequeued v4l2 buffer contains 2064 bytes, but 614400 were expected. Flags: 0x00002001.
frame=  537 fps= 27 q=26.1 size=     914kB time=00:00:19.43 bitrate= 385.2kbits/s s
[video4linux2,v4l2 @ 0x556960] Dequeued v4l2 buffer contains 2072 bytes, but 614400 were expected. Flags: 0x00002001.

Additional info on the usb device :

Code: Select all


[root@luckfox root]# v4l2-ctl --device=/dev/video21 --all
Driver Info:
	Driver name      : ov534
	Card type        : USB Camera-B4.09.24.1
	Bus info         : usb-xhci-hcd.0.auto-1
	Driver version   : 5.10.160
	Capabilities     : 0x85200001
		Video Capture
		Read/Write
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x05200001
		Video Capture
		Read/Write
		Streaming
		Extended Pix Format
Priority: 2
Video input : 0 (ov534: ok)
Format Video Capture:
	Width/Height      : 640/480
	Pixel Format      : 'YUYV' (YUYV 4:2:2)
	Field             : None
	Bytes per Line    : 1280
	Size Image        : 614400
	Colorspace        : sRGB
	Transfer Function : Default (maps to sRGB)
	YCbCr/HSV Encoding: Default (maps to ITU-R 601)
	Quantization      : Default (maps to Limited Range)
	Flags             : 
Streaming Parameters Video Capture:
	Capabilities     : timeperframe
	Frames per second: 30.000 (30/1)
	Read buffers     : 2

User Controls

                     brightness 0x00980900 (int)    : min=0 max=255 step=1 default=0 value=0 flags=slider
                       contrast 0x00980901 (int)    : min=0 max=255 step=1 default=32 value=32 flags=slider
                     saturation 0x00980902 (int)    : min=0 max=255 step=1 default=64 value=64 flags=slider
                            hue 0x00980903 (int)    : min=-90 max=90 step=1 default=0 value=0 flags=slider
        white_balance_automatic 0x0098090c (bool)   : default=1 value=1
                       exposure 0x00980911 (int)    : min=0 max=255 step=1 default=120 value=120 flags=inactive, volatile
                 gain_automatic 0x00980912 (bool)   : default=1 value=1 flags=update
                           gain 0x00980913 (int)    : min=0 max=63 step=1 default=20 value=20 flags=inactive, volatile
                horizontal_flip 0x00980914 (bool)   : default=0 value=0
                  vertical_flip 0x00980915 (bool)   : default=0 value=0
           power_line_frequency 0x00980918 (menu)   : min=0 max=1 default=0 value=0 (Disabled)
				0: Disabled
				1: 50 Hz
                      sharpness 0x0098091b (int)    : min=0 max=63 step=1 default=0 value=0 flags=slider

Camera Controls

                  auto_exposure 0x009a0901 (menu)   : min=0 max=1 default=0 value=0 (Auto Mode) flags=update
				0: Auto Mode
				1: Manual Mode


[root@luckfox root]# v4l2-ctl -d 21 --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
	Type: Video Capture

	[0]: 'GRBG' (8-bit Bayer GRGR/BGBG)
		Size: Discrete 320x240
			Interval: Discrete 0.005s (187.000 fps)
			Interval: Discrete 0.007s (150.000 fps)
			Interval: Discrete 0.007s (137.000 fps)
			Interval: Discrete 0.008s (125.000 fps)
			Interval: Discrete 0.010s (100.000 fps)
			Interval: Discrete 0.013s (75.000 fps)
			Interval: Discrete 0.017s (60.000 fps)
			Interval: Discrete 0.020s (50.000 fps)
			Interval: Discrete 0.027s (37.000 fps)
			Interval: Discrete 0.033s (30.000 fps)
		Size: Discrete 640x480
			Interval: Discrete 0.017s (60.000 fps)
			Interval: Discrete 0.020s (50.000 fps)
			Interval: Discrete 0.025s (40.000 fps)
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
	[1]: 'YUYV' (YUYV 4:2:2)
		Size: Discrete 320x240
			Interval: Discrete 0.005s (187.000 fps)
			Interval: Discrete 0.007s (150.000 fps)
			Interval: Discrete 0.007s (137.000 fps)
			Interval: Discrete 0.008s (125.000 fps)
			Interval: Discrete 0.010s (100.000 fps)
			Interval: Discrete 0.013s (75.000 fps)
			Interval: Discrete 0.017s (60.000 fps)
			Interval: Discrete 0.020s (50.000 fps)
			Interval: Discrete 0.027s (37.000 fps)
			Interval: Discrete 0.033s (30.000 fps)
		Size: Discrete 640x480
			Interval: Discrete 0.017s (60.000 fps)
			Interval: Discrete 0.020s (50.000 fps)
			Interval: Discrete 0.025s (40.000 fps)
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.067s (15.000 fps)

2025-03-05 12:24

Hello, we have only tested cameras with drivers so far, and we have not tested cameras with built-in drivers. I suspect this issue may be related to the RKISP driver. Typically, cameras using a generic driver are directly compiled into the kernel and registered as /dev/video0 upon startup. If no device occupies /dev/video0, the RKISP driver will take over when loaded.
You loaded gspca_ov534.ko after loading the RKISP driver, which may cause some errors.
You can try moving /oem/usr/ko/insmod_ko.sh to another location to prevent it from running at startup and test whether the issue still occurs.

2025-03-07 9:59

Thank You for the information on RKISP.

Disabling the RKISP as suggested makes only USB device appears as Video0 as expected. But it doesn't provide additional outcome on the issues.

Meanwhile I performed more trials and here are the observations:

First, disconnecting the ethernet cable for usb device recognition appeared to be irrelevant. I made several attempts for usb cable disconnect and reconnect trials and found that when I handle the board eg touching pins, components, the shield of usb or ethernet sockets the device can be recognized. If I put and usb extender and reconnect the cable there is no way made usb device to be recognized. I conclude that as; since that particular usb device has a some long cable comparing to other usb devices, the signal might be noisy where the board cannot tolerate and device driver gave up probing. When handling the board I might introduce some sort of impudences, perhaps, and helps it working magically. I don't have proper equipment to measure the usb signal readily available.

I also noticed that, for the frame overflow issue, the cpu has 100% utilization when ffmpeg streams the video. After 5-6 seconds the overflowing occurs and cpu utilization drops to 10~13%. That suggests me ffmpeg does not utilize hw acceleration on the board. Can yo confirm?

I also dig through the MPI development guide ( RK-KF-YF-576, page 61) and as far as I understand the HW acceleration is only provided for MIPI devices. Since the guide is in Chinese I rely on translation and would like to verify if this is accurate. Particularly no hardware acceleration or AI processing is applicable to usb devices? If so I should stop wasting time with usb cams and get proper MIPI cams...

2025-03-10 1:19

1 USB is mainly a differential signal, and the length or quality of the USB cable will affect the communication quality;
2 The hardware encoding of the RV1106 can only be done using the Rockit library provided by Rockchip, and there is currently no method to call the Rockit library when using ffmpeg commands, and can only be achieved by writing code
3 Hardware encoder and AI processing can work as long as the data and configuration information are transmitted, using MIPI camera or USB camera