Apex Dual Orin GMSL Camera Advanced#
Introduction#
If you are using this document, it means that you have encountered the following issues:
High CPU usage: You are experiencing high CPU usage and need guidance on resolving this issue.
External trigger and timestamp: You need to operate the camera in external trigger mode and require assistance with obtaining timestamps.
Notice#
Under NTP Mode, timestamp of camera will affected by NTP jitter, which will cause wrong timestamp.
If you want to use timestamp, please use GPS timing mode, or None mode.
Definition#
Term |
Explanation |
Notes |
|---|---|---|
Self-triggering |
It refers to the camera operating without being controlled by an external trigger signal, and instead, it triggers the shutter by itself. |
This mode is generally supported by most cameras. As long as no external trigger signal is provided, the camera will operate in this mode. |
External triggering |
It refers to the camera being controlled by an external signal to trigger the shutter simultaneously. |
This mode requires camera firmware support. When purchasing a camera, please confirm with the manufacturer whether this mode is supported. |
Performance Issue Resolution#
cameras_egl_demo：OpenCV Demo directly uses v4l and OpenCV to capture camera images.
/opt/miivii/features/gmsl_camera/bin/cameras_opencv_demo -s 1280x720 -d /dev/video0
Performance and Usability Integrated Solution#
cameras_opencv_demo：EGL Demo utilizes EGL as the display implementation, resulting in high efficiency for the display part.
/opt/miivii/features/gmsl_camera/bin/cameras_opencv_demo -s 1280x720 -d /dev/video0
Timestamp Acquisition Solution#
cameras_sdk_demo：SDK Demo that is compatible with GMSL1 SDK and uses ASIC for image format conversion, resulting in high efficiency. It also allows the retrieval of timestamp for triggering the shutter through the SDK.
Please note that obtaining the timestamp requires camera support for external triggering.
#Open a camera channel and retrieve the timestamp.
/opt/miivii/features/gmsl_camera/bin/cameras_sdk_demo -s 1280x720 -d /dev/video0
#Open two cameras and retrieve the timestamps.
/opt/miivii/features/gmsl_camera/bin/cameras_sdk_demo -s 1280x720 -d /dev/video0 -m 2
#Open two camera channels, set the camera frame rate to 30Hz, and retrieve the timestamps.
/opt/miivii/features/gmsl_camera/bin/cameras_sdk_demo -s 1280x720 -d /dev/video0 -m 2 -r 30-0
Compile the sample code.#
cp -r /opt/miivii/features ~/
cd ~/features/gmsl_camera
sudo make;
GMSL/GMSL2 Timestamp Measurement#
How to get detail log#
Command
# export CHECK_TIME=1 Debugging does not need to be executed when using the log test, otherwise too many logs will block the program
sudo jetson_clocks
rm /tmp/cameras_sdk_demo.log
./bin/cameras_sdk_demo -s 1280x720 -d /dev/video0
Check Result
Print log will be added in screen printing
in /tmp/，cameras_sdk_demo.log will be generated.
log file description
Field |
Unit |
physical meaning |
Test Methods |
|---|---|---|---|
Timestamp |
nanosecond |
the time the frame was triggered |
According to the transmission delay, the trigger time obtained from the queue |
FrameInterval |
nanosecond |
frame interval Trigger time interval between two frames |
Difference from the previous Timestamp |
FrameTransferDelay |
nanosecond |
frame transfer delay |
LinuxGetFrameTime - Timestamp |
LinuxGetFrameTime |
nanosecond |
System time when Linux gets frames |
The Linux system time when the frame was received |
LinuxFrameInterval |
nanosecond |
Linux time interval for getting frames |
Difference from the previous LinuxGetFrameTime |
How can I confirm that the timestamp is accurate?#
In the code, check the FrameInterval to confirm whether the timestamp is accurate.
# export CHECK_TIME=1 Debugging does not need to be executed when using the log test, otherwise too many logs will block the program
sudo jetson_clocks
rm /tmp/cameras_sdk_demo.log
./bin/cameras_sdk_demo -s 1280x720 -d /dev/video0
If the timestamp is correct, the content of /tmp/cameras_sdk_demo.log is as follows, only one line:
Timestamp : 1620897955083817280 FrameInterval : 1620897955083817280 FrameTransferDelay : 66992720 LinuxGetFrameTime : 1620897955150810000 LinuxFrameInterval : 1620897955150810000
If the timestamp is abnormal, multiple sets of timestamps will be recorded in /tmp/cameras_sdk_demo.log in multiple lines:
Timestamp : 1620958367246484576 FrameInterval : 1620958367246484576 FrameTransferDelay : 67111424 LinuxGetFrameTime : 1620958367313596000 LinuxFrameInterval : 1620958367313596000
Timestamp : 1620958739646034432 FrameInterval : 1620958739646034432 FrameTransferDelay : 67403568 LinuxGetFrameTime : 1620958739713438000 LinuxFrameInterval : 1620958739713438000
Timestamp : 1620958748796023808 FrameInterval : 1620958748796023808 FrameTransferDelay : 80901192 LinuxGetFrameTime : 1620958748876925000 LinuxFrameInterval : 1620958748876925000
Timestamp : 1620958789795973504 FrameInterval : 1620958789795973504 FrameTransferDelay : 72186496 LinuxGetFrameTime : 1620958789868160000 LinuxFrameInterval : 1620958789868160000
Timestamp : 1620959793244763712 FrameInterval : 1620959793244763712 FrameTransferDelay : 73185288 LinuxGetFrameTime : 1620959793317949000 LinuxFrameInterval : 1620959793317949000
Timestamp : 1620959854794691840 FrameInterval : 1620959854794691840 FrameTransferDelay : 68099160 LinuxGetFrameTime : 1620959854862791000 LinuxFrameInterval : 1620959854862791000
Timestamp : 1620960274844196896 FrameInterval : 1620960274844196896 FrameTransferDelay : 68391104 LinuxGetFrameTime : 1620960274912588000 LinuxFrameInterval : 1620960274912588000
Timestamp : 1620960283994186240 FrameInterval : 1620960283994186240 FrameTransferDelay : 71857760 LinuxGetFrameTime : 1620960284066044000 LinuxFrameInterval : 1620960284066044000
Timestamp : 1620960291394178080 FrameInterval : 1620960291394178080 FrameTransferDelay : 68419920 LinuxGetFrameTime : 1620960291462598000 LinuxFrameInterval : 1620960291462598000
How to confirm timestamp accuracy?#
Command
# export CHECK_TIME=1 Debugging does not need to be executed when using the log test, otherwise too many logs will block the program
sudo jetson_clocks
./bin/cameras_sdk_demo -s 1280x720 -d /dev/video0 > log
Check Result
import log to excel
get max and min value of FrameInterval
The timestamp precision can be obtained by the difference between the maximum and minimum values.
How to confirm whether the image frame transmission delay is stable?#
Prior Knowledge
GMSL Type |
Resolution |
Typical Camera |
Transmit Delay |
|---|---|---|---|
GMSL1 |
720p |
SG1-AR0143-0101-GMSL-Hxxx |
60ms |
GMSL1 |
1080p |
SG2-AR0231-0202-GMSL-Hxxx |
100ms |
Turn on the camera using a frame rate lower than the transmission delay. Since the transmission delay is smaller than the frame interval at this time, the cache of the timestamp is 1, so no other problems will be introduced by the software, and what is measured is the real physical delay.
Please be careful not to use a frame rate that is close to the transmission delay. The jitter of the transmission delay will cause inaccurate timestamps.
Command
# export CHECK_TIME=1 Debugging does not need to be executed when using the log test, otherwise too many logs will block the program
#-r 10 Indicates that the external trigger frame rate of the camera is set to 10.
sudo jetson_clocks
./bin/cameras_sdk_demo -s 1280x720 -d /dev/video0 -r 10
Check Result
Confirm the actual transmission delay through FrameTransferDelay.