Skip to main content

Chapter 4: The Robot's Senses: Simulating Sensors and Visualizing Data

Overview: The Big Idea​

A robot is only as good as its ability to perceive the world around it. In this chapter, we will learn how to add sensors to our simulated robot in Gazebo. We will explore how to simulate common sensors like cameras and LiDAR, and then we will learn how to visualize the sensor data in ROS 2 using a powerful tool called RViz.

Key Concepts​

  • Sensor Plugins: Gazebo provides a variety of plugins for simulating different types of sensors. These plugins can be attached to a link in your URDF model and will publish sensor data to a Gazebo topic.
  • RViz: RViz is a 3D visualization tool for ROS. It allows you to view the state of your robot, the data from its sensors, and the environment around it.
  • TF (Transformations): TF is a ROS package that lets you keep track of multiple coordinate frames over time. It is essential for understanding the relationship between the different parts of your robot and the world around it.

Real-World Use Cases​

Simulating sensors and visualizing their data is a critical part of robotics development.

  • Perception Algorithm Development: New algorithms for object detection, SLAM (Simultaneous Localization and Mapping), and other perception tasks can be developed and tested using simulated sensor data.
  • Debugging: Visualizing sensor data in RViz is an invaluable tool for debugging perception and navigation algorithms.
  • Data Collection: Large datasets of simulated sensor data can be collected to train and test machine learning models.

Technical Explanations​

Adding a Sensor to a URDF​

To add a sensor to your robot, you need to add a <sensor> element to your URDF file. Inside the <sensor> element, you specify the type of sensor, its properties, and the Gazebo plugin that will be used to simulate it.

Here is an example of how to add a camera sensor to a URDF:

<gazebo reference="base_link">
  <sensor type="camera" name="camera">
    <update_rate>30.0</update_rate>
    <camera name="head">
      <horizontal_fov>1.3962634</horizontal_fov>
      <image>
        <width>800</width>
        <height>800</height>
        <format>R8G8B8</format>
      </image>
      <clip>
        <near>0.02</near>
        <far>300</far>
      </clip>
    </camera>
    <plugin name="camera_controller" filename="libgazebo_ros_camera.so">
      <alwaysOn>true</alwaysOn>
      <updateRate>0.0</updateRate>
      <cameraName>camera</cameraName>
      <imageTopicName>image_raw</imageTopicName>
      <cameraInfoTopicName>camera_info</cameraInfoTopicName>
      <frameName>camera_link</frameName>
      <hackBaseline>0.07</hackBaseline>
      <distortionK1>0.0</distortionK1>
      <distortionK2>0.0</distortionK2>
      <distortionK3>0.0</distortionK3>
      <distortionT1>0.0</distortionT1>
      <distortionT2>0.0</distortionT2>
    </plugin>
  </sensor>
</gazebo>

This XML snippet defines a camera sensor and attaches it to the base_link of the robot. It also specifies the Gazebo plugin that will be used to simulate the camera and publish the image data to a ROS 2 topic.

Visualizing Data in RViz​

Once you are publishing sensor data from Gazebo, you can use RViz to visualize it. RViz provides a variety of display types for different types of sensor data. For example, you can use the Image display type to view the data from a camera, or the LaserScan display type to view the data from a LiDAR sensor.

Diagram Description: A screenshot of RViz showing a 3D model of a robot and the data from a simulated LiDAR sensor. The LiDAR data is displayed as a series of points that show the distance to obstacles in the environment.

Code Samples​

There is no new node code for this chapter. The code is in the URDF file and the launch file. Here is an updated launch file that starts RViz and a robot_state_publisher node to publish the robot's TF tree.

import os
from ament_index_python.packages import get_package_share_directory
from launch import LaunchDescription
from launch.actions import ExecuteProcess
from launch_ros.actions import Node

def generate_launch_description():
    package_dir = get_package_share_directory('your_package_name')
    urdf = os.path.join(package_dir, 'urdf', 'two_wheeled_robot_with_camera.urdf')
    
    return LaunchDescription([
        ExecuteProcess(
            cmd=['gazebo', '--verbose', '-s', 'libgazebo_ros_init.so', '-s', 'libgazebo_ros_factory.so'],
            output='screen'
        ),
        Node(
            package='gazebo_ros',
            executable='spawn_entity.py',
            arguments=['-entity', 'two_wheeled_robot', '-file', urdf],
            output='screen'
        ),
        Node(
            package='robot_state_publisher',
            executable='robot_state_publisher',
            name='robot_state_publisher',
            output='screen',
            parameters=[{'robot_description': open(urdf).read()}]
        ),
        Node(
            package='rviz2',
            executable='rviz2',
            name='rviz2',
            output='screen'
        )
    ])

Summary​

In this chapter, we have learned how to add sensors to our simulated robot and how to visualize the sensor data in RViz. We have seen how to use Gazebo sensor plugins to simulate cameras and other sensors, and we have learned about the importance of TF for understanding the spatial relationship between different parts of our system.

In the next chapter, we will explore the Unity simulator and see how it can be used to create high-fidelity simulations for robotics.