1.2. TIER IV Cameras Getting Started Guide for ADLINK ROSCube RQX-59G#
Attention
This document is for the users of ADLINK RQX-59G
1.2.1. Preparation#
1.2.1.1. Required items#
ADLINK ROSCube-X RQX-59G
BSP version 1.4.2
JetPack5.1.2 (L4T R35.4.1)installed
GMSL2 coaxial cable (FAKRA - mini FAKRA, 1:4)
TIER IV Automotive HDR Camera C1 or C2
1.2.1.2. Camera connection#
Firstly, make sure that the RQX-59G is turned off.
Connect cameras to the FAKRA connectors of the cable(single FAKRA connector side). Then insert the mini FAKRA connectors of the cable to the RQX-59G’s GMSL2 ports.
Please make sure the direction of the lock clown on FAKRA connectors and mini FAKRA connectors is correct (Fig.1~3).
![../_images/c1_start_guide_fig1.png](../_images/c1_start_guide_fig1.png)
Fig. 1.7 RosCube RQX-59G#
![../_images/c1_start_guide_fig2.png](../_images/c1_start_guide_fig2.png)
Fig. 1.8 Connector#
![../_images/c1_start_guide_fig3.png](../_images/c1_start_guide_fig3.png)
Fig. 1.9 Fakra cable insertion direction#
1.2.2. Power on and check the camera image stream output#
1.2.2.1. Power on#
Power on the RQX-59G and confirm that the power-on LED is turned on blue.
1.2.2.2. Login#
On the startup window, type the password to log in.
Default settings are shown below.
user |
ros |
password |
adlinkros |
1.2.2.3. Camera driver installation#
Note
As of 2024-04-01, to use RQX-59G with TIER IV cameras, dedicated driver deb package is needed.
Get the camera driver deb package from Github.
Copy the provided driver package file (tier4-camera-gmsl_1.4.2_59g_arm64.deb
) to any directory in RQX-59G (e.g. ~/driver
). Then follow the command line instructions below:
Updating the apt package. It requires an internet connection. Then, install the driver by using the
apt install
command. Please make sure*.dtbo
files are generated at/boot
.# Install sudo apt update sudo apt install make debhelper dkms sudo apt install ~/driver/tier4-camera-gmsl_1.2.1_arm64.deb # Confirm /boot/tier4-*.dtbo exists ls /boot/*.dtbo /boot/tier4-imx490-gmsl-device-tree-overlay-roscube-orin-r354.dtbo /boot/tier4-isx021-gmsl-device-tree-overlay-roscube-orin-r354.dtbo /boot/tier4-isx021-imx490-gmsl-device-tree-overlay-roscube-orin-r354.dtbo
To assign the camera for each port, you can use
configure-by-hardware.py
script. Please refer to the camera driver’s README page for the details. The user needs to specify an appropriate overlay command for each user’s configuration.# For RQX-59G # Please be noted that you need to specify "1=" (not "2=") sudo /opt/nvidia/jetson-io/config-by-hardware.py -n 1="TIERIV ISX021 GMSL2 Camera Device Tree Overlay" # Confirm kernel_tegra234-p3701-0004-rqx590-nosuspend-user-custom.dtb has been generated ls /boot/*.dtb /boot/kernel_tegra234-p3701-0004-rqx590-nosuspend.dtb /boot/kernel_tegra234-p3701-0004-rqx590-nosuspend-user-custom.dtb # Then, shutdown the system sudo shutdown -h now
You can run
/opt/nvidia/jetson-io/config-by-hardware.py -l
to check available overlay options.sudo /opt/nvidia/jetson-io/config-by-hardware.py -l [sudo] password for ros: Header 1 [default]: Jetson AGX CSI Connector Available hardware modules: 1. TIERIV IMX490 GMSL2 Camera Device Tree Overlay 2. TIERIV ISX021 GMSL2 Camera Device Tree Overlay 3. TIERIV ISX021 IMX490 GMSL2 Camera Device Tree Overlay
For instance, to assign all GMSL ports to the C2 camera, the overlay command should be
sudo /opt/nvidia/jetson-io/config-by-hardware.py -n 1="TIERIV IMX490 GMSL2 Camera Device Tree Overlay"
To Assign GMSL ports for both C1 and C2, the overlay command should be
sudo /opt/nvidia/jetson-io/config-by-hardware.py -n 1="TIERIV ISX021 IMX490 GMSL2 Camera Device Tree Overlay"
In this case, ports 1, 2, 5, and 6 will be assigned to C1, and ports 3, 4, 7, and 8 will be assigned to C2.
Please refer to the device driver GitHub repository for the details of the device overlay.
1.2.2.4. Check if the RQX-59G recognizes cameras#
Open a terminal window and type the following command. If it returns /dev/videoX
, cameras are recognized properly as video devices.
ls /dev/video*
/dev/video0
/dev/video1
.
.
.
/dev/video7 # When 8 cameras are connected
1.2.2.5. Visualize camera output using GStreamer#
Open a terminal window and type the following command. Gstreamer will launch and the camera image stream will appear on a new window(Fig. 4, Fig.5).
Please also refer to GStreamer command examples for various GStreamer examples.
1.2.2.5.1. Case 1: 1 C1 camera is connected#
# If cameras are running in slave mode, execute this i2cset command first
i2cset -f -y 2 0x66 0x04 0xff
# Start streaming
gst-launch-1.0 v4l2src io-mode=0 device=/dev/video0 do-timestamp=true ! 'video/x-raw, width=1920, height=1280, framerate=30/1, format=UYVY' ! videoscale ! xvimagesink sync=false
1.2.2.5.2. Case 2: 8 C1 cameras are connected#
# If cameras are running in slave mode, execute this i2cset command first
i2cset -f -y 2 0x66 0x04 0xff
# Start streaming
gst-launch-1.0 v4l2src io-mode=0 device=/dev/video0 do-timestamp=true ! \
'video/x-raw, width=1920, height=1280, framerate=30/1, format=UYVY' ! \
xvimagesink sync=false v4l2src io-mode=0 device=/dev/video1 do-timestamp=true ! \
'video/x-raw, width=1920, height=1280, framerate=30/1, format=UYVY' ! xvimagesink sync=false \
v4l2src io-mode=0 device=/dev/video2 do-timestamp=true ! \
'video/x-raw, width=1920, height=1280, framerate=30/1, format=UYVY' ! xvimagesink sync=false \
v4l2src io-mode=0 device=/dev/video3 do-timestamp=true ! \
'video/x-raw, width=1920, height=1280, framerate=30/1, format=UYVY' ! xvimagesink sync=false \
v4l2src io-mode=0 device=/dev/video4 do-timestamp=true ! \
'video/x-raw, width=1920, height=1280, framerate=30/1, format=UYVY' ! xvimagesink sync=false \
v4l2src io-mode=0 device=/dev/video5 do-timestamp=true ! \
'video/x-raw, width=1920, height=1280, framerate=30/1, format=UYVY' ! xvimagesink sync=false \
v4l2src io-mode=0 device=/dev/video6 do-timestamp=true ! \
'video/x-raw, width=1920, height=1280, framerate=30/1, format=UYVY' ! xvimagesink sync=false \
v4l2src io-mode=0 device=/dev/video7 do-timestamp=true ! \
'video/x-raw, width=1920, height=1280, framerate=30/1, format=UYVY' ! xvimagesink sync=false \
See also
If there is a problem concerning unstable camera activation when you connect multiple cameras, please refer to this page.
1.2.2.5.3. Case 3: 1 C2 camera is connected#
gst-launch-1.0 v4l2src io-mode=0 device=/dev/video0 do-timestamp=true ! 'video/x-raw, width=2880, height=1860, framerate=30/1, format=UYVY' ! videoscale ! xvimagesink sync=false
When you connect multiple C2 cameras
At RQX-59G, four deserializer are implemented for eight GMSL ports, and two GMSL port shares single deserializer and its bandwidth.
Even though we’ve checked that two C2 cameras can work with one deserializer, it’s best to use just one camera per deserializer, especially if you’re connecting up to four cameras.
For example:
Port 1 and 2 … Not recommended
Port 3 and 4 … Not recommended
Port 1 and port 3 … Recommended
Port 1 and 5 … Recommended
![../_images/c1_start_guide_fig4.png](../_images/c1_start_guide_fig4.png)
Fig. 1.10 Visualized camera output#
![../_images/c1_start_guide_fig5.png](../_images/c1_start_guide_fig5.png)
Fig. 1.11 Visualized camera output (x8 cameras)#
1.2.3. Record video data as rosbag (ROS format log data)#
1.2.3.1. Install ROS melodic and gscam (ROS camera driver for GStreamer)#
Install ROS melodic and gscam by following command line instructions.
Caution
The installation command may change. If the following commands do not work, please refer to the ROS official documentation.
sudo sh -c 'echo "deb http://packages.ros.org/ros/ubuntu $(lsb_release -sc) main" > /etc/apt/sources.list.d/ros-latest.list'
curl -s https://raw.githubusercontent.com/ros/rosdistro/master/ros.asc | sudo apt-key add -
sudo apt update
sudo apt install ros-melodic-desktop
echo "source /opt/ros/melodic/setup.bash" >> ~/.bashrc
sudo apt install -y ros-melodic-gscam
1.2.3.2. Run gscam and visualize the image topic#
Run the gscam by following instructions.
Open a new terminal and execute the following command to start roscore
roscore
Open another terminal and execute the following commands
export GSCAM_CONFIG="v4l2src device=/dev/video0 ! video/x-raw,framerate=30/1 ! videoconvert" rosrun gscam gscam
Now that a topic (ROS format message data) camera/image_raw is published, visualize the topic by rqt_image_view. To launch rqt_image_view, open another terminal and execute the following command.
rosrun rqt_image_view rqt_image_view
This command opens a window to visualize image topics. Select camera/image_raw by clicking the pull-down menu on the upper left of the window (Fig. 6)
![../_images/c1_start_guide_fig6.png](../_images/c1_start_guide_fig6.png)
Fig. 1.12 Select a published image topic#
To run gscam with multiple cameras, create a text file with the following XML message and save it as ~/2cameras.launch
. This .launch file is for streaming with 2 cameras.
<launch>
<arg name="frame_rate" default="30"/>
<node name="gscam_driver_v4l_port_1" pkg="gscam" type="gscam" output="screen" ns="port_1">
<param name="camera_name" value="port_1"/>
<param name="frame_id" value="port_1"/>
<param name="camera_info_url" value="package://gscam/examples/uncalibrated_parameters.ini"/>
<param name="gscam_config" value="v4l2src device=/dev/video0 ! video/x-raw,format=UYVY,width=1920,height=1280,framerate=30/1 ! videoconvert"/>
<param name="sync_sink" value="true"/>
</node>
<node name="gscam_driver_v4l_port_2" pkg="gscam" type="gscam" output="screen" ns="port_2">
<param name="camera_name" value="port_2"/>
<param name="frame_id" value="port_2"/>
<param name="camera_info_url" value="package://gscam/examples/uncalibrated_parameters.ini"/>
<param name="gscam_config" value="v4l2src device=/dev/video1 ! video/x-raw,format=UYVY,width=1920,height=1280,framerate=30/1 ! videoconvert"/>
<param name="sync_sink" value="true"/>
</node>
</launch>
Caution
This is the setting for C1. For C2, please change the width
, height
and framerate
if necessary.
Assuming that you have created /home/ros/2cameras.launch
, execute the following command to run gscam with multiple cameras.
roslaunch /home/ros/2cameras.launch
Note
For more cameras, add nodes with appropriate node name, namespace(ns), camera_name, frame_id, and gscam_config.
1.2.3.3. Record a rosbag#
While running gscam, you may record a rosbag (Log file in ROS format) by
rosbag record -a
Now you may also check the information of rosbag by rosbag info
rosbag info <recorded rosbag file name>
# example output
path: 2022-03-29-17-42-07.bag
version: 2.0
duration: 3.1s
start: Mar 29 2022 17:42:07.97 (1648543327.97)
end: Mar 29 2022 17:42:11.07 (1648543331.07)
size: 365.7 MB
messages: 113
compression: none [53/53 chunks]
types: rosgraph_msgs/Log [acffd30cd6b6de30f120938c17c593fb]
sensor_msgs/CameraInfo [c9a58c1b0b154e0e6da7578cb991d214]
sensor_msgs/Image [060021388200f6f0f447d0fcd9c64743]
topics: /port_1/camera/camera_info 26 msgs : sensor_msgs/CameraInfo
/port_1/camera/image_raw 26 msgs : sensor_msgs/Image
/port_2/camera/camera_info 26 msgs : sensor_msgs/CameraInfo
/port_2/camera/image_raw 26 msgs : sensor_msgs/Image
/rosout 8 msgs : rosgraph_msgs/Log (3 connections)
/rosout_agg 1 msg : rosgraph_msgs/Log
1.2.4. Configuration#
To change driver configuration, edit the configuration file /etc/modprobe.d/tier4-*.conf
. (See following sections).
The configuration file includes the line by default. (This is the case of C1)
options tier4_isx021 trigger_mode=0 enable_auto_exposure=1 enable_distortion_correction=1
To apply the configurations, reboot RQX-59G after editing.
1.2.4.1. Switch drive mode#
Caution
To run the camera in slave mode, you need to input frame synchronization signal from RQX-59G. please check the Shutter triggering over GMSL2 to drive cameras in slave mode.
Please refer to the ADLINK’s documentation regarding trigger input how to generate the trigger signal from RQX-59G.
1.2.4.1.1. The case of C1#
Camera drive mode can be switched from Master mode to Slave mode(and vice versa) by editing the configuration file.
Master mode: Free running mode at 30fps (Default)
Slave mode: Shutter triggering mode, Shutter timing, and the frame rate can be adjusted by the FSYNC signal frequency (configurable with RQX-59G)
To switch modes, edit trigger_mode in /etc/modprobe.d/tier4-isx021.conf
. (See following cases)
Drive mode |
Frame rate |
trigger_mode= |
Master |
30fps |
0 |
Slave |
Depends on the FSYNC input frequency |
1 |
1.2.4.1.2. The case of C2#
In the case of C2, the following modes can be selected by editing /etc/modprobe.d/tier4-imx490.conf
.
The following table shows the available settings.
Drive mode |
Frame rate |
trigger_mode= |
Master |
10fps |
0 |
Slave |
10fps |
1 |
Master |
20fps |
2 |
Slave |
20fps |
3 |
Master |
30fps |
4 |
Slave |
30fps |
5 |
1.2.4.2. Enable/Disable Lens Distortion Correction (LDC)#
Note
This setting is common for both C1 and C2.
To enable LDC, set the flag enable_distortion_correction=1
(default).
To disable LDC, set the flag enable_distortion_correction=0
.
1.2.4.3. Enable/Disable auto exposure (AE)#
Note
This setting is common for both C1 and C2.
To enable AE, set the flag enable_auto_exposure=1
(default).
To disable AE, set the flag enable_auto_exposure=0
.
1.2.4.4. Setting exposure time#
1.2.4.4.1. The case of C1#
In the case of C1, users can specify three types of exposure time via variables named shutter_time_min
, shutter_time_mid
, and shutter_time_max
.
The actual exposure time of C1 transits these three (and linearly interpolated values) corresponding to the illuminance.
The values for the variables should be specified in microseconds. Users who would like to fix exposure time under arbitral illuminance conditions can set the exact same value for these variables.
For example, the following configuration in /etc/modprobe.d/tier4-isx021.conf
fixes exposure time to 11 ms:
shutter_time_min=11000 shutter_time_mid=11000 shutter_time_max=11000
1.2.4.4.2. The case of C2#
Likewise C1, C2 also has capability to set two types of exposure time via variables named shutter_time_min
and shutter_time_max
.
The unit of the value for these variables is also microseconds.
For example, the following configuration in /etc/modprobe.d/tier4-imx490.conf
fixes exposure time to 11 ms:
shutter_time_min=11000 shutter_time_max=11000
1.2.5. Shutter triggering over GMSL2#
Camera operation in slave mode requires triggering signals input (FSYNC input) over GMSL2. Please refer to ADLINK’s documentation for how to configure the FSYNC parameters and GPIO settings.
1.2.5.1. Preparation to drive cameras in slave mode#
Check the HW version of your RQX-59G by executing i2cget -f -y 2 0x66 0x01
on a terminal and follow the instructions below.
Please confirm that returned value is at least 0x24
.
i2cget -f -y 2 0x66 0x01
0x24
1.2.5.2. Input FSYNC frequency for C1#
You can input any frequency that is slower than 30fps.
1.2.5.3. Input FSYNC frequency for C2#
For C2, the allowed FSYNC frequency depends on the drive modes. Please refer to the below list.
In the case of 30fps mode (
trigger_mode=5
): 15 < f <= 30 fpsIn the case of 20fps mode (
trigger_mode=3
): 10 < f <= 20 fpsIn the case of 10fps mode (
trigger_mode=1
): 5 < f <= 10 fps
1.2.5.4. FPS check#
To check the FPS setting with a visualized video, please execute the following commands.
The camera is connected to only port 1
gst-launch-1.0 v4l2src io-mode=0 device=/dev/video0 do-timestamp=true ! 'video/x-raw, width=1920, height=1280, framerate=30/1, format=UYVY' ! fpsdisplaysink video-sink=xvimagesink sync=false
Cameras are connected to ports 1 and 2
gst-launch-1.0 v4l2src io-mode=0 device=/dev/video0 do-timestamp=true ! 'video/x-raw, width=1920, height=1280, framerate=30/1, format=UYVY' ! xvimagesink sync=false v4l2src io-mode=0 device=/dev/video1 do-timestamp=true ! 'video/x-raw, width=1920, height=1280, framerate=30/1, format=UYVY' ! fpsdisplaysink video-sink=xvimagesink sync=false