1.1. TIER IV Cameras Getting Started Guide for ADLINK ROSCube RQX-58G

Attention

This document is for the users of ADLINK RQX-58G

1.1.1. Preparation

1.1.1.1. Required items

• ADLINK ROSCube-X RQX-58G

• JetPack4.5 (L4T 32.5.1) or JetPack4.6 (L4T 32.6.1) installed

• GMSL2 coaxial cable (FAKRA - mini FAKRA, 1:4)

• TIER IV Automotive HDR Camera C1 or C2

1.1.1.2. Camera connection

1. Firstly, make sure that the ROSCube-X is turned off.

2. Connect cameras to the FAKRA connectors of the cable(single FAKRA connector side). Then insert the mini FAKRA connectors of the cable to the ROSCube-X’s GMSL2 ports.

3. Please make sure the direction of the lock clown on FAKRA connectors and mini FAKRA connectors is correct (Fig.1~3).

Fig. 1.1 RosCube

Fig. 1.2 Connector

Fig. 1.3 Fakra cable insertion direction

1.1.2. Power on and check the camera image stream output

1.1.2.1. Power on

Power on the ROSCube-X and confirm that the power-on LED is turned on blue.

1.1.2.2. Login

On the startup window, type the password to log in.

Default settings are shown below.

user	ros
password	adlinkros

1.1.2.3. Camera driver installation

Note

For C2, camera driver v.1.4.1 (or higher) is needed to set drive mode properly. Please upgrade the camera driver if needed.

If proper camera driver has already been installed, please skip this section.

Get the camera driver deb package from Github. The latest release has been confirmed to work with RQX-58G.

Copy the provided driver package file (tier4-isx021-gmsl_*.*.*_arm64.deb) to any directory in ROSCube-X (e.g. ~/c1_driver). Then follow the command line instructions below:

1. Updating the apt package. It requires an internet connection. Then, install the driver by using the apt install command. Please make sure *.dtbo files are generated at /boot.

   # Install 
   sudo apt update
   sudo apt install make debhelper dkms
   sudo apt install  ~/c1_driver/tier4-camera-gmsl_1.2.1_arm64.deb
   
   # Confirm  /boot/tier4-*.dtbo exists
   ls /boot/*.dtbo
   

2. The command for the device tree overlay differs for the L4T version. To check the L4T version, execute $ cat /etc/nv_tegra_release and check the result. For example, If it returns # R32 (release), REVISION: 5.1..., the installed L4T version is 32.5.1.

3. Choose appropriate instructions for the installed L4T version. Also, to assign the camera for each port, please refer to the camera driver’s README page. The user needs to specify an appropriate overlay command for each user’s configuration.

   L4T 32.5.1

   # For L4T 32.5.1
   sudo /opt/nvidia/jetson-io/config-by-hardware.py -n "TIERIV ISX021 GMSL2 Camera Device Tree Overlay"
   
   # Confirm　/boot/kernel_tegra194-rqx-58g-tier4-isx021-gmsl2-camera-device-tree-overlay-roscube-r32_x.dtb has been generated
   ls /boot/kernel_tegra194-rqx-58g-tier4-isx021-gmsl2-camera-device-tree-overlay*.dtb
   
   # Then, shutdown the system
   sudo shutdown -h now
   

   L4T 32.6.1 or later

   # For L4T 32.6.1
   sudo /opt/nvidia/jetson-io/config-by-hardware.py -n 2="TIERIV ISX021 GMSL2 Camera Device Tree Overlay"
   
   # Confirm　/boot/kernel_tegra194-rqx-58g-user-custom.dtb has been generated
   ls /boot/kernel_tegra194-rqx-58g-user-custom.dtb
   
   # Then, shutdown the system
   sudo shutdown -h now
   

You can run /opt/nvidia/jetson-io/config-by-hardware.py -l to check available overlay options.

$ sudo /opt/nvidia/jetson-io/config-by-hardware.py -l
[sudo] password for ros:
Header 1 [default]: Jetson 40pin Header
No hardware configurations found!
Header 2: Jetson AGX Xavier CSI Connector
Available hardware modules:
1. TIERIV IMX490 GMSL2 Camera Device Tree Overlay
2. TIERIV ISX021 GMSL2 Camera Device Tree Overlay
3. TIERIV ISX021 IMX490 GMSL2 Camera Device Tree Overlay

For instance, to assign all GMSL ports to the C2 camera, the overlay command should be

sudo /opt/nvidia/jetson-io/config-by-hardware.py -n 2="TIERIV IMX490 GMSL2 Camera Device Tree Overlay"

To Assign GMSL ports for both C1 and C2, the overlay command should be

sudo /opt/nvidia/jetson-io/config-by-hardware.py -n 2="TIERIV ISX021 IMX490 GMSL2 Camera Device Tree Overlay"

In this case, ports 1, 2, 5, and 6 will be assigned to C1, and ports 3, 4, 7, and 8 will be assigned to C2.

Please refer to the device driver GitHub repository for the details of the device overlay.

1.1.2.4. Check if the ROSCube-X recognizes cameras

Open a terminal window and type the following command. If it returns /dev/videoX, cameras are recognized properly as video devices.

ls /dev/video*
/dev/video0
/dev/video1
.
.
.
/dev/video7 # When 8 cameras are connected

1.1.2.5. Visualize camera output using GStreamer

Open a terminal window and type the following command. Gstreamer will launch and the camera image stream will appear on a new window（Fig. 4, Fig.5）

1.1.2.5.1. Case 1: 1 C1 camera is connected

# If cameras are running in slave mode, execute this i2cset command first
i2cset -f -y 2 0x66 0x04 0xff

# Start streaming
gst-launch-1.0 v4l2src io-mode=0 device=/dev/video0 do-timestamp=true ! 'video/x-raw, width=1920, height=1280, framerate=30/1, format=UYVY' ! videoscale ! xvimagesink sync=false

1.1.2.5.2. Case 2: 8 C1 cameras are connected

# If cameras are running in slave mode, execute this i2cset command first
i2cset -f -y 2 0x66 0x04 0xff

# Start streaming
gst-launch-1.0 v4l2src io-mode=0 device=/dev/video0 do-timestamp=true ! \
'video/x-raw, width=1920, height=1280, framerate=30/1, format=UYVY' ! \
xvimagesink sync=false v4l2src io-mode=0 device=/dev/video1 do-timestamp=true ! \
'video/x-raw, width=1920, height=1280, framerate=30/1, format=UYVY' ! xvimagesink sync=false \
v4l2src io-mode=0 device=/dev/video2 do-timestamp=true ! \
'video/x-raw, width=1920, height=1280, framerate=30/1, format=UYVY' ! xvimagesink sync=false \
v4l2src io-mode=0 device=/dev/video3 do-timestamp=true ! \
'video/x-raw, width=1920, height=1280, framerate=30/1, format=UYVY' ! xvimagesink sync=false \
v4l2src io-mode=0 device=/dev/video4 do-timestamp=true ! \
'video/x-raw, width=1920, height=1280, framerate=30/1, format=UYVY' ! xvimagesink sync=false \
v4l2src io-mode=0 device=/dev/video5 do-timestamp=true ! \
'video/x-raw, width=1920, height=1280, framerate=30/1, format=UYVY' ! xvimagesink sync=false \
v4l2src io-mode=0 device=/dev/video6 do-timestamp=true ! \
'video/x-raw, width=1920, height=1280, framerate=30/1, format=UYVY' ! xvimagesink sync=false \
v4l2src io-mode=0 device=/dev/video7 do-timestamp=true ! \
'video/x-raw, width=1920, height=1280, framerate=30/1, format=UYVY' ! xvimagesink sync=false \

See also

If there is a problem concerning unstable camera activation when you connect multiple cameras, please refer to this page.

1.1.2.5.3. Case 3: 1 C2 camera is connected

gst-launch-1.0 v4l2src io-mode=0 device=/dev/video0 do-timestamp=true ! 'video/x-raw, width=2880, height=1860, framerate=30/1, format=UYVY' ! videoscale ! xvimagesink sync=false

Restriction regarding C2 camera

Currently, with ROSCube L4T 32.6.1, two C2 cameras cannot be connected to single deserializer board.

For example:

• Port 1 and 2 … NG

• Port 3 and 4 … NG

• Port 1 and port 3 … OK

• Port 1 and 5 … OK

Fig. 1.4 Visualized camera output

Fig. 1.5 Visualized camera output (x8 cameras)

Constraint on camera boot order (For driver version earlier than v1.1.1)

If you are using a driver version earlier than v1.1.1, please follow the specific instructions for booting. For driver versions v1.2.1 and later, there are no ordering or constraint requirements

---

To run multiple cameras, cameras need to be launched at once. For example, to run 2 cameras, the start-streaming command has to be a one liner

# This one-liner command works
gst-launch-1.0 v4l2src io-mode=0 device=/dev/video0 do-timestamp=true ! \
'video/x-raw, width=1920, height=1280, framerate=30/1, format=UYVY' ! \
xvimagesink sync=false v4l2src io-mode=0 device=/dev/video1 do-timestamp=true ! \
'video/x-raw, width=1920, height=1280, framerate=30/1, format=UYVY' ! xvimagesink sync=false \

These separate commands do not work

# These separated commands may not work

# On a terminal
gst-launch-1.0 v4l2src io-mode=0 device=/dev/video0 do-timestamp=true ! \
'video/x-raw, width=1920, height=1280, framerate=30/1, format=UYVY' ! \
xvimagesink sync=false

# On another terminal
gst-launch-1.0 v4l2src io-mode=0 device=/dev/video1 do-timestamp=true ! \
'video/x-raw, width=1920, height=1280, framerate=30/1, format=UYVY' ! \
xvimagesink sync=false

1.1.3. Record video data as rosbag (ROS format log data)

1.1.3.1. Install ROS melodic and gscam (ROS camera driver for GStreamer)

Install ROS melodic and gscam by following command line instructions.

Caution

The installation command may change. If the following commands do not work, please refer to the ROS official documentation.

sudo sh -c 'echo "deb http://packages.ros.org/ros/ubuntu $(lsb_release -sc) main" > /etc/apt/sources.list.d/ros-latest.list'
curl -s https://raw.githubusercontent.com/ros/rosdistro/master/ros.asc | sudo apt-key add -
sudo apt update
sudo apt install ros-melodic-desktop
echo "source /opt/ros/melodic/setup.bash" >> ~/.bashrc
sudo apt install -y ros-melodic-gscam

1.1.3.2. Run gscam and visualize the image topic

Run the gscam by following instructions.

1. Open a new terminal and execute the following command to start roscore

   roscore
   

2. Open another terminal and execute the following commands

   export GSCAM_CONFIG="v4l2src device=/dev/video0 ! video/x-raw,framerate=30/1 ! videoconvert"
   rosrun gscam gscam
   

3. Now that a topic (ROS format message data) camera/image_raw is published, visualize the topic by rqt_image_view. To launch rqt_image_view, open another terminal and execute the following command.

   rosrun rqt_image_view rqt_image_view
   

   This command opens a window to visualize image topics. Select camera/image_raw by clicking the pull-down menu on the upper left of the window (Fig. 6)

Fig. 1.6 Select a published image topic

To run gscam with multiple cameras, create a text file with the following XML message and save it as ~/2cameras.launch. This .launch file is for streaming with 2 cameras.

<launch>
 <arg name="frame_rate" default="30"/>
 <node name="gscam_driver_v4l_port_1" pkg="gscam" type="gscam" output="screen" ns="port_1">
   <param name="camera_name" value="port_1"/>
   <param name="frame_id" value="port_1"/>
   <param name="camera_info_url" value="package://gscam/examples/uncalibrated_parameters.ini"/>
   <param name="gscam_config" value="v4l2src device=/dev/video0 ! video/x-raw,format=UYVY,width=1920,height=1280,framerate=30/1 ! videoconvert"/>
   <param name="sync_sink" value="true"/>
</node>
 <node name="gscam_driver_v4l_port_2" pkg="gscam" type="gscam" output="screen" ns="port_2">
   <param name="camera_name" value="port_2"/>
   <param name="frame_id" value="port_2"/>
   <param name="camera_info_url" value="package://gscam/examples/uncalibrated_parameters.ini"/>
   <param name="gscam_config" value="v4l2src device=/dev/video1 ! video/x-raw,format=UYVY,width=1920,height=1280,framerate=30/1 ! videoconvert"/>
   <param name="sync_sink" value="true"/>
</node>
</launch>

Caution

This is the setting for C1. For C2, please change the width, height and framerate if necessary.

Assuming that you have created /home/ros/2cameras.launch, execute the following command to run gscam with multiple cameras.

roslaunch /home/ros/2cameras.launch

Note

For more cameras, add nodes with appropriate node name, namespace(ns), camera_name, frame_id, and gscam_config.

1.1.3.3. Record a rosbag

While running gscam, you may record a rosbag (Log file in ROS format) by

rosbag record -a

Now you may also check the information of rosbag by rosbag info

rosbag info <recorded rosbag file name>
# example output
path:        2022-03-29-17-42-07.bag
version:     2.0
duration:    3.1s
start:       Mar 29 2022 17:42:07.97 (1648543327.97)
end:         Mar 29 2022 17:42:11.07 (1648543331.07)
size:        365.7 MB
messages:    113
compression: none [53/53 chunks]
types:       rosgraph_msgs/Log      [acffd30cd6b6de30f120938c17c593fb]
             sensor_msgs/CameraInfo [c9a58c1b0b154e0e6da7578cb991d214]
             sensor_msgs/Image      [060021388200f6f0f447d0fcd9c64743]
topics:      /port_1/camera/camera_info   26 msgs    : sensor_msgs/CameraInfo
             /port_1/camera/image_raw     26 msgs    : sensor_msgs/Image     
             /port_2/camera/camera_info   26 msgs    : sensor_msgs/CameraInfo
             /port_2/camera/image_raw     26 msgs    : sensor_msgs/Image     
             /rosout                       8 msgs    : rosgraph_msgs/Log      (3 connections)
             /rosout_agg                   1 msg     : rosgraph_msgs/Log

1.1.4. Configuration

To change driver configuration, edit the configuration file /etc/modprobe.d/tier4-*.conf. (See following sections). The configuration file includes the line by default. (This is the case of C1)

options tier4_isx021 trigger_mode=0 enable_auto_exposure=1 enable_distortion_correction=1

To apply the configurations, reboot ROSCube-X after editing.

1.1.4.1. Switch drive mode

Caution

To run the camera in slave mode, you need to input frame synchronization signal from ROSCube-X. please check the Shutter triggering over GMSL2 to drive cameras in slave mode

1.1.4.1.1. The case of C1

Camera drive mode can be switched from Master mode to Slave mode(and vice versa) by editing the configuration file.

• Master mode: Free running mode at 30fps (Default)

• Slave mode: Shutter triggering mode, Shutter timing, and the frame rate can be adjusted by the FSYNC signal frequency (configurable with ROSCube-X)

To switch modes, edit trigger_mode in /etc/modprobe.d/tier4-isx021.conf. (See following cases)

Drive mode	Frame rate	trigger_mode=
Master	30fps	0
Slave	Depends on the FSYNC input frequency	1

1.1.4.1.2. The case of C2

In the case of C2, the following modes can be selected by editing /etc/modprobe.d/tier4-imx490.conf. The following table shows the available settings.

Drive mode	Frame rate	trigger_mode=
Master	10fps	0
Slave	10fps	1
Master	20fps	2
Slave	20fps	3
Master	30fps	4
Slave	30fps	5

1.1.4.2. Enable/Disable Lens Distortion Correction (LDC)

Note

This setting is common for both C1 and C2.

To enable LDC, set the flag enable_distortion_correction=1 (default). To disable LDC, set the flag enable_distortion_correction=0.

1.1.4.3. Enable/Disable auto exposure (AE)

Note

This setting is common for both C1 and C2.

To enable AE, set the flag enable_auto_exposure=1 (default). To disable AE, set the flag enable_auto_exposure=0.

1.1.4.4. Setting exposure time

1.1.4.4.1. The case of C1

In the case of C1, users can specify three types of exposure time via variables named shutter_time_min, shutter_time_mid, and shutter_time_max. The actual exposure time of C1 transits these three (and linearly interpolated values) corresponding to the illuminance. The values for the variables should be specified in microseconds. Users who would like to fix exposure time under arbitral illuminance conditions can set the exact same value for these variables. For example, the following configuration in /etc/modprobe.d/tier4-isx021.conf fixes exposure time to 11 ms:

shutter_time_min=11000 shutter_time_mid=11000 shutter_time_max=11000

1.1.4.4.2. The case of C2

Likewise C1, C2 also has capability to set two types of exposure time via variables named shutter_time_min and shutter_time_max. The unit of the value for these variables is also microseconds. For example, the following configuration in /etc/modprobe.d/tier4-imx490.conf fixes exposure time to 11 ms:

shutter_time_min=11000 shutter_time_max=11000

1.1.5. Shutter triggering over GMSL2

Camera operation in slave mode requires triggering signals input (FSYNC input) over GMSL2. Please refer to ADLINK’s documentation for how to configure the FSYNC parameters and GPIO settings.

1.1.5.1. Preparation to drive cameras in slave mode

Check the HW version of your ROSCube-X by executing i2cget -f -y 2 0x66 0x01 on a terminal and follow the instructions below.

When i2cget -f -y 2 0x66 0x01 returns 0x23

Execute i2cset -f -y 2 0x66 0x04 0xff before start streaming. The system requires this preparation only once after its boot.

When i2cget -f -y 2 0x66 0x01 returns 0x21

No preparation is required.

1.1.5.2. Input FSYNC frequency for C1

You can input any frequency that is slower than 30fps.

1.1.5.3. Input FSYNC frequency for C2

For C2, the allowed FSYNC frequency depends on the drive modes. Please refer to the below list.

• In the case of 30fps mode (trigger_mode=5): 15 < f <= 30 fps

• In the case of 20fps mode (trigger_mode=3): 10 < f <= 20 fps

• In the case of 10fps mode (trigger_mode=1): 5 < f <= 10 fps

1.1.5.4. FPS check

To check the FPS setting with a visualized video, please execute the following commands.

• The camera is connected to only port 1

  gst-launch-1.0 v4l2src io-mode=0 device=/dev/video0 do-timestamp=true ! 'video/x-raw, width=1920, height=1280, framerate=30/1, format=UYVY' ! fpsdisplaysink video-sink=xvimagesink sync=false
  

• Cameras are connected to ports 1 and 2

  gst-launch-1.0 v4l2src io-mode=0 device=/dev/video0 do-timestamp=true ! 'video/x-raw, width=1920, height=1280, framerate=30/1, format=UYVY' ! xvimagesink sync=false v4l2src io-mode=0 device=/dev/video1 do-timestamp=true ! 'video/x-raw, width=1920, height=1280, framerate=30/1, format=UYVY' ! fpsdisplaysink video-sink=xvimagesink sync=false