Apex AD10 GMSL Camera Advanced#

Introduction#

If you are using this document, it means that you have encountered the following issues:

  • High CPU usage: You are experiencing high CPU usage and need guidance on resolving this issue.

  • External trigger and timestamp: You need to operate the camera in external trigger mode and require assistance with obtaining timestamps.

Notice#

Under NTP Mode, timestamp of camera will affected by NTP jitter, which will cause wrong timestamp.

If you want to use timestamp, please use GPS timing mode, or None mode.

Definition#

Term

Explanation

Notes

Self-triggering

It refers to the camera operating without being controlled by an external trigger signal, and instead, it triggers the shutter by itself.

This mode is generally supported by most cameras. As long as no external trigger signal is provided, the camera will operate in this mode.

External triggering

It refers to the camera being controlled by an external signal to trigger the shutter simultaneously.

This mode requires camera firmware support. When purchasing a camera, please confirm with the manufacturer whether this mode is supported.

Performance Issue Resolution#

cameras_egl_demo:OpenCV Demo directly uses v4l and OpenCV to capture camera images.

/opt/miivii/features/gmsl_camera/bin/cameras_opencv_demo -s 1280x720 -d /dev/video0

Performance and Usability Integrated Solution#

cameras_opencv_demo:EGL Demo utilizes EGL as the display implementation, resulting in high efficiency for the display part.

/opt/miivii/features/gmsl_camera/bin/cameras_opencv_demo -s 1280x720 -d /dev/video0

Timestamp Acquisition Solution#

cameras_sdk_demo:SDK Demo that is compatible with GMSL1 SDK and uses ASIC for image format conversion, resulting in high efficiency. It also allows the retrieval of timestamp for triggering the shutter through the SDK.

Please note that obtaining the timestamp requires camera support for external triggering.

#Open a camera channel and retrieve the timestamp.
/opt/miivii/features/gmsl_camera/bin/cameras_sdk_demo -s 1280x720 -d /dev/video0
#Open two cameras and retrieve the timestamps.
/opt/miivii/features/gmsl_camera/bin/cameras_sdk_demo -s 1280x720 -d /dev/video0 -m 2
#Open two camera channels, set the camera frame rate to 30Hz, and retrieve the timestamps.
/opt/miivii/features/gmsl_camera/bin/cameras_sdk_demo -s 1280x720 -d /dev/video0 -m 2 -r 30-0

Compile the sample code.#

cp -r /opt/miivii/features ~/
cd ~/features/gmsl_camera
sudo make;

GMSL/GMSL2 Timestamp Measurement#

How to get detail log#

Command

# export CHECK_TIME=1   Debugging does not need to be executed when using the log test, otherwise too many logs will block the program 

sudo jetson_clocks
rm /tmp/cameras_sdk_demo.log
./bin/cameras_sdk_demo -s 1280x720 -d /dev/video0

Check Result

  • Print log will be added in screen printing

  • in /tmp/,cameras_sdk_demo.log will be generated.

log file description

Field

Unit

physical meaning

Test Methods

Timestamp

nanosecond

the time the frame was triggered

According to the transmission delay, the trigger time obtained from the queue

FrameInterval

nanosecond

frame interval

Trigger time interval between two frames

Difference from the previous Timestamp

FrameTransferDelay

nanosecond

frame transfer delay

LinuxGetFrameTime - Timestamp

LinuxGetFrameTime

nanosecond

System time when Linux gets frames

The Linux system time when the frame was received

LinuxFrameInterval

nanosecond

Linux time interval for getting frames

Difference from the previous LinuxGetFrameTime

How can I confirm that the timestamp is accurate?#

In the code, check the FrameInterval to confirm whether the timestamp is accurate.

# export CHECK_TIME=1   Debugging does not need to be executed when using the log test, otherwise too many logs will block the program

sudo jetson_clocks
rm /tmp/cameras_sdk_demo.log
./bin/cameras_sdk_demo -s 1280x720 -d /dev/video0

If the timestamp is correct, the content of /tmp/cameras_sdk_demo.log is as follows, only one line:

Timestamp : 1620897955083817280 FrameInterval : 1620897955083817280 FrameTransferDelay : 66992720 LinuxGetFrameTime : 1620897955150810000 LinuxFrameInterval : 1620897955150810000

If the timestamp is abnormal, multiple sets of timestamps will be recorded in /tmp/cameras_sdk_demo.log in multiple lines:

Timestamp : 1620958367246484576 FrameInterval  :  1620958367246484576 FrameTransferDelay : 67111424 LinuxGetFrameTime : 1620958367313596000 LinuxFrameInterval : 1620958367313596000
Timestamp : 1620958739646034432 FrameInterval  :  1620958739646034432 FrameTransferDelay : 67403568 LinuxGetFrameTime : 1620958739713438000 LinuxFrameInterval : 1620958739713438000
Timestamp : 1620958748796023808 FrameInterval  :  1620958748796023808 FrameTransferDelay : 80901192 LinuxGetFrameTime : 1620958748876925000 LinuxFrameInterval : 1620958748876925000
Timestamp : 1620958789795973504 FrameInterval  :  1620958789795973504 FrameTransferDelay : 72186496 LinuxGetFrameTime : 1620958789868160000 LinuxFrameInterval : 1620958789868160000
Timestamp : 1620959793244763712 FrameInterval  :  1620959793244763712 FrameTransferDelay : 73185288 LinuxGetFrameTime : 1620959793317949000 LinuxFrameInterval : 1620959793317949000
Timestamp : 1620959854794691840 FrameInterval  :  1620959854794691840 FrameTransferDelay : 68099160 LinuxGetFrameTime : 1620959854862791000 LinuxFrameInterval : 1620959854862791000
Timestamp : 1620960274844196896 FrameInterval  :  1620960274844196896 FrameTransferDelay : 68391104 LinuxGetFrameTime : 1620960274912588000 LinuxFrameInterval : 1620960274912588000
Timestamp : 1620960283994186240 FrameInterval  :  1620960283994186240 FrameTransferDelay : 71857760 LinuxGetFrameTime : 1620960284066044000 LinuxFrameInterval : 1620960284066044000
Timestamp : 1620960291394178080 FrameInterval  :  1620960291394178080 FrameTransferDelay : 68419920 LinuxGetFrameTime : 1620960291462598000 LinuxFrameInterval : 1620960291462598000

How to confirm timestamp accuracy?#

Command

# export CHECK_TIME=1   Debugging does not need to be executed when using the log test, otherwise too many logs will block the program

 sudo jetson_clocks
./bin/cameras_sdk_demo -s 1280x720 -d /dev/video0 > log

Check Result

  • import log to excel

  • get max and min value of FrameInterval

  • The timestamp precision can be obtained by the difference between the maximum and minimum values.

How to confirm whether the image frame transmission delay is stable?#

Prior Knowledge

GMSL Type

Resolution

Typical Camera

Transmit Delay

GMSL1

720p

SG1-AR0143-0101-GMSL-Hxxx

60ms

GMSL1

1080p

SG2-AR0231-0202-GMSL-Hxxx

100ms

Turn on the camera using a frame rate lower than the transmission delay. Since the transmission delay is smaller than the frame interval at this time, the cache of the timestamp is 1, so no other problems will be introduced by the software, and what is measured is the real physical delay.

Please be careful not to use a frame rate that is close to the transmission delay. The jitter of the transmission delay will cause inaccurate timestamps.

Command

# export CHECK_TIME=1   Debugging does not need to be executed when using the log test, otherwise too many logs will block the program
#-r 10 Indicates that the external trigger frame rate of the camera is set to 10.
sudo jetson_clocks
./bin/cameras_sdk_demo -s 1280x720 -d /dev/video0 -r 10

Check Result

Confirm the actual transmission delay through FrameTransferDelay.