Isaac ROS Freespace Segmentation

概述

Isaac ROS Freespace Segmentation是一个专为ROS 2设计的软件包，它能够创建用于导航的占用网格。这个软件包通过分析机器人相对于地面的姿态和自由空间分割掩码，利用Bi3D Freespace技术为Nav2生成占用网格，帮助机器人在导航时避开障碍物。得益于GPU加速，该软件包能够实现实时且响应迅速的导航效果，非常适合机器人应用场景。此外，Bi3D Freespace还为地面移动机器人提供了一种新的占用网格数据源。

在isaac_ros_bi3d的节点图中，它能够将同步的立体图像对作为输入，输出自由空间分割掩码。这个掩码随后被isaac_ros_bi3d_freespace使用，结合相机相对于地面的位置信息，将平面自由空间转换成Nav2所需的占用网格。

虽然存在多种预测占用网格以供导航使用的方法，但没有任何方法是完美无缺的。每种方法在从传感器获取的测量数据中都存在一定的准确性限制。每种传感器都有其特定的视场和测量范围，以及它们无法测量的区域。Bi3D Freespace通过立体摄像机输入，提供了一种识别障碍物的多样化方法。与激光雷达相比，立体摄像机在垂直视场方面表现更佳，能够检测到激光雷达可能忽略的低矮障碍物。因此，Bi3D Freespace为基于激光雷达的占用网格扫描提供了一种稳健的、基于视觉的补充方案。

isaac_ros_bi3d_freespace

下载快速入门资源

从NGC下载快速启动数据：

#确保已安装所需的库
sudo apt-get install -y curl tar     

NGC_ORG="nvidia"
NGC_TEAM="isaac"
NGC_RESOURCE="isaac_ros_assets"
NGC_VERSION="isaac_ros_bi3d_freespace"
NGC_FILENAME="quickstart.tar.gz"

REQ_URL="https://api.ngc.nvidia.com/v2/resources/$NGC_ORG/$NGC_TEAM/$NGC_RESOURCE/versions/$NGC_VERSION/files/$NGC_FILENAME"

mkdir -p ${ISAAC_ROS_WS}/isaac_ros_assets/${NGC_VERSION} && \
    curl -LO --request GET "${REQ_URL}" && \
    tar -xf ${NGC_FILENAME} -C ${ISAAC_ROS_WS}/isaac_ros_assets/${NGC_VERSION} && \
    rm ${NGC_FILENAME}

下载预先训练的Bi3D模型：

mkdir -p ${ISAAC_ROS_WS}/isaac_ros_assets/models/bi3d_proximity_segmentation && \
cd ${ISAAC_RO
S_WS}/isaac_ros_assets/models/bi3d_proximity_segmentation && \
wget 'https://api.ngc.nvidia.com/v2/models/nvidia/isaac/bi3d_proximity_segmentation/versions/2.0.0/files/featnet.onnx' &&
wget 'https://api.ngc.nvidia.com/v2/models/nvidia/isaac/bi3d_proximity_segmentation/versions/2.0.0/files/segnet.onnx'

将 .onnx 模型文件转换为 TensorRT 引擎计划文件

/usr/src/tensorrt/bin/trtexec \
--saveEngine=${ISAAC_ROS_WS}/isaac_ros_assets/models/bi3d_proximity_segmentation/featnet.plan \
--onnx=${ISAAC_ROS_WS}/isaac_ros_assets/models/bi3d_proximity_segmentation/featnet.onnx \
--int8 --useDLACore=0 --allowGPUFallback &&
/usr/src/tensorrt/bin/trtexec \
--saveEngine=${ISAAC_ROS_WS}/isaac_ros_assets/models/bi3d_proximity_segmentation/segnet.plan \
--onnx=${ISAAC_ROS_WS}/isaac_ros_assets/models/bi3d_proximity_segmentation/segnet.onnx \
--int8 --useDLACore=0 --allowGPUFallback

运行启动文件

rosbag

ros2 launch isaac_ros_examples isaac_ros_examples.launch.py \
launch_fragments:=bi3d,bi3d_freespace \
interface_specs_file:=${ISAAC_ROS_WS}/isaac_ros_assets/isaac_ros_bi3d_freespace/rosbag_quickstart_interface_specs.json \
featnet_engine_file_path:=${ISAAC_ROS_WS}/isaac_ros_assets/models/bi3d_proximity_segmentation/featnet.plan \
segnet_engine_file_path:=${ISAAC_ROS_WS}/isaac_ros_assets/models/bi3d_proximity_segmentation/segnet.plan \
max_disparity_values:=10

启动另一个docker终端

ros2 bag play -l ${ISAAC_ROS_WS}/isaac_ros_assets/isaac_ros_bi3d_freespace/quickstart.bag

RealSense Camera

ros2 launch isaac_ros_examples isaac_ros_examples.launch.py \
launch_fragments:=realsense_stereo_rect,bi3d,bi3d_freespace \
interface_specs_file:=${ISAAC_ROS_WS}/isaac_ros_assets/isaac_ros_bi3d_freespace/realsense_quickstart_interface_specs.json \
featnet_engine_file_path:=${ISAAC_ROS_WS}/isaac_ros_assets/models/bi3d_proximity_segmentation/featnet.plan \
segnet_engine_file_path:=${ISAAC_ROS_WS}/isaac_ros_assets/models/bi3d_proximity_segmentation/segnet.plan \
max_disparity_values:=10 publish_default_tf:=true

Hawk Camera

ros2 launch isaac_ros_examples isaac_ros_examples.launch.py \
launch_fragments:=argus_stereo,resize_rectify_stereo,bi3d,bi3d_freespace \
interface_specs_file:=${ISAAC_ROS_WS}/isaac_ros_assets/isaac_ros_bi3d_freespace/hawk_quickstart_interface_specs.json \
featnet_engine_file_path:=${ISAAC_ROS_WS}/isaac_ros_assets/models/bi3d_proximity_segmentation/featnet.plan \
segnet_engine_file_path:=${ISAAC_ROS_WS}/isaac_ros_assets/models/bi3d_proximity_segmentation/segnet.plan \
max_disparity_values:=10 publish_default_tf:=true

可视化结果

打开一个新的docker终端

rviz2

#将Fixed Frame设置为base_link

ROS参数

ROS 参数	类型	默认	描述
base_link_frame	std::string	base_link	base link frame
camera_frame	std::string	camera	camera frame
f_x	double	0.0	x轴上的焦距
f_y	double	0.0	y上的焦距
grid_width	int	100	输出占用网格的宽度
grid_height	int	100	输出占用网格的高度
grid_resolution	double	0.01	输出占用网格的分辨率

ROS 主题	消息格式	描述
freespace_segmentation/mask_in	stereo_msgs/DisparityImage	输入视差图像，其中与地面对应的像素标记为 0

ROS 主题	消息格式	描述
freespace_segmentation/occupancy_grid	nav_msgs/OccupancyGrid	输出占用网格，其中单元格标记为 0（如果空闲）