1.1. TIER IV Cameras Getting Started Guide for ADLINK ROSCube RQX-58G#
Attention
This document is for the users of ADLINK RQX-58G
1.1.1. Preparation#
1.1.1.1. Required items#
ADLINK ROSCube-X RQX-58G
JetPack4.5 (L4T 32.5.1) or
JetPack4.6 (L4T 32.6.1) or
JetPack5.1.2 (L4T 35.4.1)
GMSL2 coaxial cable (FAKRA - mini FAKRA, 1:4)
TIER IV Automotive HDR Camera C1, C2 or C3
1.1.1.2. Camera connection#
Firstly, make sure that the ROSCube-X is turned off.
Connect cameras to the FAKRA connectors of the cable(single FAKRA connector side). Then insert the mini FAKRA connectors of the cable to the ROSCube-X’s GMSL2 ports.
Please make sure the direction of the lock clown on FAKRA connectors and mini FAKRA connectors is correct (Fig.1~3).
Fig. 1.1 RosCube#
Fig. 1.2 Connector#
Fig. 1.3 Fakra cable insertion direction#
1.1.2. Power on and check the camera image stream output#
1.1.2.1. Power on#
Power on the ROSCube-X and confirm that the power-on LED is turned on blue.
1.1.2.2. Login#
On the startup window, type the password to log in.
Default settings are shown below.
user |
|
password |
|
1.1.2.3. Camera driver installation#
New in version v1.4.1: C2 camera support has beed added
New in version v1.5.0: C3 camera support has beed added
Get the camera driver deb package from Github. The latest release has been confirmed to work with RQX-58G.
Copy the provided driver package file (tier4-camera-gmsl_*.*.*_arm64.deb) to any directory in ROSCube-X (e.g. ~/c1_driver). Then follow the command line instructions below:
Updating the apt package. It requires an internet connection. Then, install the driver by using the
apt installcommand. Please make sure*.dtbofiles are generated at/boot.# Install sudo apt update sudo apt install make debhelper dkms sudo apt install ~/c1_driver/tier4-camera-gmsl_*.*.*_arm64.deb # Confirm /boot/tier4-*.dtbo exists ls /boot/*.dtbo
The command for the device tree overlay differs for the L4T version. To check the L4T version, execute
$ cat /etc/nv_tegra_releaseand check the result. For example, If it returns# R32 (release), REVISION: 5.1...,the installed L4T version is32.5.1.Choose appropriate instructions for the installed L4T version. Also, to assign the camera for each port, please refer to the camera driver’s README page. The user needs to specify an appropriate overlay command for each user’s configuration.
# For L4T 32.5.1 sudo /opt/nvidia/jetson-io/config-by-hardware.py -n "TIERIV ISX021 GMSL2 Camera Device Tree Overlay" # Confirm /boot/kernel_tegra194-rqx-58g-tier4-isx021-gmsl2-camera-device-tree-overlay-roscube-r32_x.dtb has been generated ls /boot/kernel_tegra194-rqx-58g-tier4-isx021-gmsl2-camera-device-tree-overlay*.dtb # Then, shutdown the system sudo shutdown -h now
# For L4T 32.6.1 sudo /opt/nvidia/jetson-io/config-by-hardware.py -n 2="TIERIV ISX021 GMSL2 Camera Device Tree Overlay" # Confirm /boot/kernel_tegra194-rqx-58g-user-custom.dtb has been generated ls /boot/kernel_tegra194-rqx-58g-user-custom.dtb # Then, shutdown the system sudo shutdown -h now
# For L4T 35.4.1 sudo /opt/nvidia/jetson-io/config-by-hardware.py -n 1="TIERIV ISX021 GMSL2 Camera Device Tree Overlay" # Confirm /boot/kernel_tegra194-p2888-0001-rqx580-nosuspend-user-custom.dtb has been generated ls /boot/kernel_tegra194-p2888-0001-rqx580-nosuspend-user-custom.dtb # Then, shutdown the system sudo shutdown -h now
You can run /opt/nvidia/jetson-io/config-by-hardware.py -l to check available overlay options.
$ sudo /opt/nvidia/jetson-io/config-by-hardware.py -l
[sudo] password for ros:
Header 1 [default]: Jetson 40pin Header
No hardware configurations found!
Header 2: Jetson AGX Xavier CSI Connector
Available hardware modules:
1. TIERIV IMX490 GMSL2 Camera Device Tree Overlay
2. TIERIV IMX728 GMSL2 Camera Device Tree Overlay
3. TIERIV ISX021 GMSL2 Camera Device Tree Overlay
4. TIERIV ISX021 IMX490 GMSL2 Camera Device Tree Overlay
5. TIERIV ISX021 IMX490 IMX728 GMSL2 Camera Device Tree Overlay
For instance, to assign all GMSL ports to the C2 camera, the overlay command should be
sudo /opt/nvidia/jetson-io/config-by-hardware.py -n 2="TIERIV IMX490 GMSL2 Camera Device Tree Overlay"
To Assign GMSL ports for both C1 and C2, the overlay command should be
sudo /opt/nvidia/jetson-io/config-by-hardware.py -n 2="TIERIV ISX021 IMX490 GMSL2 Camera Device Tree Overlay"
In this case, ports 1, 2, 5, and 6 will be assigned to C1, and ports 3, 4, 7, and 8 will be assigned to C2.
To Assign GMSL ports for C1, C2 and C3, the overlay command should be
sudo /opt/nvidia/jetson-io/config-by-hardware.py -n 2="TIERIV ISX021 IMX490 IMX728 GMSL2 Camera Device Tree Overlay"
Please refer to the device driver GitHub repository for the details of the device overlay.
1.1.2.4. Check if the ROSCube-X recognizes cameras#
Open a terminal window and type the following command. If it returns /dev/videoX, cameras are recognized properly as video devices.
ls /dev/video*
/dev/video0
/dev/video1
.
.
.
/dev/video7 # When 8 cameras are connected
1.1.3. Visualize camera output using GStreamer#
Open a terminal window and type the following command. Gstreamer will launch and the camera image stream will appear on a new window（Fig. 4, Fig.5）
1.1.3.1. Case 1: 1 C1 camera is connected#
# If cameras are running in slave mode, execute this i2cset command first
i2cset -f -y 2 0x66 0x04 0xff
# Start streaming
gst-launch-1.0 v4l2src io-mode=0 device=/dev/video0 do-timestamp=true ! 'video/x-raw, width=1920, height=1280, framerate=30/1, format=UYVY' ! videoscale ! xvimagesink sync=false
1.1.3.2. Case 2: 8 C1 cameras are connected#
# If cameras are running in slave mode, execute this i2cset command first
i2cset -f -y 2 0x66 0x04 0xff
# Start streaming
gst-launch-1.0 v4l2src io-mode=0 device=/dev/video0 do-timestamp=true ! \
'video/x-raw, width=1920, height=1280, framerate=30/1, format=UYVY' ! \
xvimagesink sync=false v4l2src io-mode=0 device=/dev/video1 do-timestamp=true ! \
'video/x-raw, width=1920, height=1280, framerate=30/1, format=UYVY' ! xvimagesink sync=false \
v4l2src io-mode=0 device=/dev/video2 do-timestamp=true ! \
'video/x-raw, width=1920, height=1280, framerate=30/1, format=UYVY' ! xvimagesink sync=false \
v4l2src io-mode=0 device=/dev/video3 do-timestamp=true ! \
'video/x-raw, width=1920, height=1280, framerate=30/1, format=UYVY' ! xvimagesink sync=false \
v4l2src io-mode=0 device=/dev/video4 do-timestamp=true ! \
'video/x-raw, width=1920, height=1280, framerate=30/1, format=UYVY' ! xvimagesink sync=false \
v4l2src io-mode=0 device=/dev/video5 do-timestamp=true ! \
'video/x-raw, width=1920, height=1280, framerate=30/1, format=UYVY' ! xvimagesink sync=false \
v4l2src io-mode=0 device=/dev/video6 do-timestamp=true ! \
'video/x-raw, width=1920, height=1280, framerate=30/1, format=UYVY' ! xvimagesink sync=false \
v4l2src io-mode=0 device=/dev/video7 do-timestamp=true ! \
'video/x-raw, width=1920, height=1280, framerate=30/1, format=UYVY' ! xvimagesink sync=false \
See also
If there is a problem concerning unstable camera activation when you connect multiple cameras, please refer to this page.
1.1.3.3. Case 3: 1 C2 camera is connected#
gst-launch-1.0 v4l2src io-mode=0 device=/dev/video0 do-timestamp=true ! 'video/x-raw, width=2880, height=1860, framerate=30/1, format=UYVY' ! videoscale ! xvimagesink sync=false
1.1.3.4. Case 4: 1 C3 camera is connected#
gst-launch-1.0 v4l2src io-mode=0 device=/dev/video0 do-timestamp=true ! 'video/x-raw, width=3840, height=2160, framerate=20/1, format=UYVY' ! videoscale ! xvimagesink sync=false
Restriction regarding C2 and C3 camera
Currently, with ROSCube RQX-58G, two C2 / C3 cameras cannot be connected to single deserializer board.
For example:
Port 1 and 2 … NG
Port 3 and 4 … NG
Port 1 and port 3 … OK
Port 1 and 5 … OK
Fig. 1.4 Visualized camera output#
Fig. 1.5 Visualized camera output (x8 cameras)#
1.1.4. Record video data as rosbag (ROS format log data)#
1.1.4.1. Install ROS melodic and gscam (ROS camera driver for GStreamer)#
Install ROS melodic and gscam by following command line instructions.
Caution
The installation command may change. If the following commands do not work, please refer to the ROS official documentation.
sudo sh -c 'echo "deb http://packages.ros.org/ros/ubuntu $(lsb_release -sc) main" > /etc/apt/sources.list.d/ros-latest.list'
curl -s https://raw.githubusercontent.com/ros/rosdistro/master/ros.asc | sudo apt-key add -
sudo apt update
sudo apt install ros-melodic-desktop
echo "source /opt/ros/melodic/setup.bash" >> ~/.bashrc
sudo apt install -y ros-melodic-gscam
1.1.4.2. Run gscam and visualize the image topic#
Run the gscam by following instructions.
Open a new terminal and execute the following command to start roscore
roscoreOpen another terminal and execute the following commands
export GSCAM_CONFIG="v4l2src device=/dev/video0 ! video/x-raw,framerate=30/1 ! videoconvert" rosrun gscam gscam
Now that a topic (ROS format message data) camera/image_raw is published, visualize the topic by rqt_image_view. To launch rqt_image_view, open another terminal and execute the following command.
rosrun rqt_image_view rqt_image_view
This command opens a window to visualize image topics. Select camera/image_raw by clicking the pull-down menu on the upper left of the window (Fig. 6)
Fig. 1.6 Select a published image topic#
To run gscam with multiple cameras, create a text file with the following XML message and save it as ~/2cameras.launch. This .launch file is for streaming with 2 cameras.
<launch>
<arg name="frame_rate" default="30"/>
<node name="gscam_driver_v4l_port_1" pkg="gscam" type="gscam" output="screen" ns="port_1">
<param name="camera_name" value="port_1"/>
<param name="frame_id" value="port_1"/>
<param name="camera_info_url" value="package://gscam/examples/uncalibrated_parameters.ini"/>
<param name="gscam_config" value="v4l2src device=/dev/video0 ! video/x-raw,format=UYVY,width=1920,height=1280,framerate=30/1 ! videoconvert"/>
<param name="sync_sink" value="true"/>
</node>
<node name="gscam_driver_v4l_port_2" pkg="gscam" type="gscam" output="screen" ns="port_2">
<param name="camera_name" value="port_2"/>
<param name="frame_id" value="port_2"/>
<param name="camera_info_url" value="package://gscam/examples/uncalibrated_parameters.ini"/>
<param name="gscam_config" value="v4l2src device=/dev/video1 ! video/x-raw,format=UYVY,width=1920,height=1280,framerate=30/1 ! videoconvert"/>
<param name="sync_sink" value="true"/>
</node>
</launch>
Caution
This is the setting for C1. For C2 and C3, please change the width, height and framerate if necessary.
Assuming that you have created /home/ros/2cameras.launch, execute the following command to run gscam with multiple cameras.
roslaunch /home/ros/2cameras.launch
Note
For more cameras, add nodes with appropriate node name, namespace(ns), camera_name, frame_id, and gscam_config.
1.1.4.3. Record a rosbag#
While running gscam, you may record a rosbag (Log file in ROS format) by
rosbag record -a
Now you may also check the information of rosbag by rosbag info
rosbag info <recorded rosbag file name>
# example output
path: 2022-03-29-17-42-07.bag
version: 2.0
duration: 3.1s
start: Mar 29 2022 17:42:07.97 (1648543327.97)
end: Mar 29 2022 17:42:11.07 (1648543331.07)
size: 365.7 MB
messages: 113
compression: none [53/53 chunks]
types: rosgraph_msgs/Log [acffd30cd6b6de30f120938c17c593fb]
sensor_msgs/CameraInfo [c9a58c1b0b154e0e6da7578cb991d214]
sensor_msgs/Image [060021388200f6f0f447d0fcd9c64743]
topics: /port_1/camera/camera_info 26 msgs : sensor_msgs/CameraInfo
/port_1/camera/image_raw 26 msgs : sensor_msgs/Image
/port_2/camera/camera_info 26 msgs : sensor_msgs/CameraInfo
/port_2/camera/image_raw 26 msgs : sensor_msgs/Image
/rosout 8 msgs : rosgraph_msgs/Log (3 connections)
/rosout_agg 1 msg : rosgraph_msgs/Log
1.1.5. Configuration#
To change driver configuration, edit the configuration file /etc/modprobe.d/tier4-*.conf. (See following sections).
The configuration file includes the line by default. (This is the case of C1)
options tier4_isx021 trigger_mode=0 enable_auto_exposure=1 enable_distortion_correction=1
To apply the configurations, reboot ROSCube-X after editing.
1.1.5.1. Switch drive mode#
Caution
To run the camera in slave mode, you need to input frame synchronization signal from ROSCube-X. please check the Shutter triggering over GMSL2 to drive cameras in slave mode
1.1.5.1.1. The case of C1#
Camera drive mode can be switched from Master mode to Slave mode(and vice versa) by editing the configuration file.
Master mode: Free running mode at 30fps (Default)
Slave mode: Shutter triggering mode, Shutter timing, and the frame rate can be adjusted by the FSYNC signal frequency (configurable with ROSCube-X)
To switch modes, edit trigger_mode in /etc/modprobe.d/tier4-isx021.conf. (See following cases)
Drive mode |
Frame rate |
trigger_mode= |
Master |
30fps |
0 |
Slave |
Depends on the FSYNC input frequency |
1 |
1.1.5.1.2. The case of C2#
In the case of C2, the following modes can be selected by editing /etc/modprobe.d/tier4-imx490.conf respectively.
The following table shows the available settings.
Drive mode |
Frame rate |
trigger_mode= |
Master |
10fps |
0 |
Slave |
10fps |
1 |
Master |
20fps |
2 |
Slave |
20fps |
3 |
Master |
30fps |
4 |
Slave |
30fps |
5 |
1.1.5.2. Enable/Disable Lens Distortion Correction (LDC)#
Note
This setting is common for C1, C2 and C3.
To enable LDC, set the flag enable_distortion_correction=1 (default).
To disable LDC, set the flag enable_distortion_correction=0.
1.1.5.3. Enable/Disable auto exposure (AE)#
Note
This setting is common for C1, C2 and C3.
To enable AE, set the flag enable_auto_exposure=1 (default).
To disable AE, set the flag enable_auto_exposure=0.
1.1.5.4. Setting exposure time#
1.1.5.4.1. The case of C1#
In the case of C1, users can specify three types of exposure time via variables named shutter_time_min, shutter_time_mid, and shutter_time_max.
The actual exposure time of C1 transits these three (and linearly interpolated values) corresponding to the illuminance.
The values for the variables should be specified in microseconds. Users who would like to fix exposure time under arbitral illuminance conditions can set the exact same value for these variables.
For example, the following configuration in /etc/modprobe.d/tier4-isx021.conf fixes exposure time to 11 ms:
shutter_time_min=11000 shutter_time_mid=11000 shutter_time_max=11000
1.1.5.4.2. The case of C2#
Likewise C1, C2 also has capability to set two types of exposure time via variables named shutter_time_min and shutter_time_max.
The unit of the value for these variables is also microseconds.
For example, the following configuration in /etc/modprobe.d/tier4-imx490.conf fixes exposure time to 11 ms:
shutter_time_min=11000 shutter_time_max=11000
1.1.6. Shutter triggering over GMSL2#
Camera operation in slave mode requires triggering signals input (FSYNC input) over GMSL2. Please refer to ADLINK’s documentation for how to configure the FSYNC parameters and GPIO settings.
1.1.6.1. Preparation to drive cameras in slave mode#
Check the HW version of your ROSCube-X by executing i2cget -f -y 2 0x66 0x01 on a terminal and follow the instructions below.
Execute i2cset -f -y 2 0x66 0x04 0xff before start streaming.
The system requires this preparation only once after its boot.
No preparation is required.
1.1.6.2. Input FSYNC frequency for C1#
You can input any frequency that is slower than 30fps.
1.1.6.3. Input FSYNC frequency for C2#
For C2 the allowed FSYNC frequency depends on the drive modes. Please refer to the below list.
In the case of 30fps mode (
trigger_mode=5): 15 < f <= 30 fpsIn the case of 20fps mode (
trigger_mode=3): 10 < f <= 20 fpsIn the case of 10fps mode (
trigger_mode=1): 5 < f <= 10 fps
1.1.6.4. FPS check#
To check the FPS setting with a visualized video, please execute the following commands.
The camera is connected to only port 1
gst-launch-1.0 v4l2src io-mode=0 device=/dev/video0 do-timestamp=true ! 'video/x-raw, width=1920, height=1280, framerate=30/1, format=UYVY' ! fpsdisplaysink video-sink=xvimagesink sync=false
Cameras are connected to ports 1 and 2
gst-launch-1.0 v4l2src io-mode=0 device=/dev/video0 do-timestamp=true ! 'video/x-raw, width=1920, height=1280, framerate=30/1, format=UYVY' ! xvimagesink sync=false v4l2src io-mode=0 device=/dev/video1 do-timestamp=true ! 'video/x-raw, width=1920, height=1280, framerate=30/1, format=UYVY' ! fpsdisplaysink video-sink=xvimagesink sync=false