Autoware Full Stack to achieve autonomous driving

Velodyne – BaseLink TF

roslaunch runtime_manager setup_tf.launch x:=1.2 y:=0.0 z:=2.0 yaw:=0.0 pitch:=0.0 roll:=0.0 frame_id:=/base_link child_frame_id:=/velodyne period_in_ms:=10

Robot Model

roslaunch model_publisher vehicle_model.launch

PCD Map

rosrun map_file points_map_loader noupdate PCD_FILES_SEPARATED_BY_SPACES

VectorMap

rosrun map_file vector_map_loader CSV_FILES_SEPARATED_BY_SPACES

World-Map TF

roslaunch world_map_tf.launch

Example launch file (From Moriyama data)

<launch>
  <!-- world map tf -->
  <node pkg="tf" type="static_transform_publisher" name="world_to_map" args="14771 84757 -39 0 0 0 /world /map 10" />
</launch>

roslaunch PATH/TO/MAP_WORLF_TF.launch

Voxel Grid Filter

roslaunch points_downsampler points_downsample.launch node_name:=voxel_grid_filter

Ground Filter

roslaunch points_preprocessor ring_ground_filter.launch node_name:=ring_ground_filter point_topic:=/points_raw

NMEATOPOSE (IF GNSS available)

roslaunch gnss_localizer nmea2tfpose.launch plane:=7

NDT Matching

roslaunch ndt_localizer ndt_matching.launch use_openmp:=False use_gpu:=False get_height:=False

Mission Planning

rosrun lane_planner lane_rule

rosrun lane_planner lane_stop

roslaunch lane_planner lane_select.launch enablePlannerDynamicSwitch:=False

roslaunch astar_planner obstacle_avoid.launch avoidance:=False avoid_distance:=13 avoid_velocity_limit_mps:=4

roslaunch autoware_connector vel_pose_connect.launch topic_pose_stamped:=/ndt_pose topic_twist_stamped:=/estimate_twist sim_mode:=False

roslaunch astar_planner velocity_set.launch use_crosswalk_detection:=False enable_multiple_crosswalk_detection:=False points_topic:=points_no_ground enablePlannerDynamicSwitch:=False

roslaunch waypoint_maker waypoint_loader.launch multi_lane_csv:=/path/to/saved_waypoints.csv decelerate:=1

roslaunch waypoint_follower pure_pursuit.launch is_linear_interpolation:=True publishes_for_steering_robot:=False

roslaunch waypoint_follower twist_filter.launch

How to setup IMU XSense MTI

Check the assigned USB device to the IMU using dmesg

[ 8808.219908] usb 3-3: new full-speed USB device number 28 using xhci_hcd
[ 8808.237513] usb 3-3: New USB device found, idVendor=2639, idProduct=0013
[ 8808.237522] usb 3-3: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[ 8808.237527] usb 3-3: Product: MTi-300 AHRS
[ 8808.237531] usb 3-3: Manufacturer: Xsens
[ 8808.237534] usb 3-3: SerialNumber: 03700715
[ 8808.265957] usbcore: registered new interface driver usbserial
[ 8808.265982] usbcore: registered new interface driver usbserial_generic
[ 8808.265999] usbserial: USB Serial support registered for generic
[ 8808.268037] usbcore: registered new interface driver xsens_mt
[ 8808.268048] usbserial: USB Serial support registered for xsens_mt
[ 8808.268063] xsens_mt 3-3:1.1: xsens_mt converter detected
[ 8808.268112] usb 3-3: xsens_mt converter now attached to ttyUSB0
  1. Change permissions of the device chmod a+rw /dev/ttyUSB0 Probably is USB0, change it accordingly to your setup
  2. In an Autoware sourced terminal execute: rosrun xsens_driver mtdevice.py -m 2 -f 100
    (this configures the IMU to publish raw data from the sensor at 100Hz)
    To publish data execute (in a sourced terminal):
    rosrun xsens_driver mtnode.py _device:=/dev/ttyUSB0 _baudrate:=115200
  3. Confirm data is actually coming using rostopic echo /imu_raw in a different terminal

How to launch NDT Localization

Requisites

Before starting make sure you have

  1. PCD Map (.pcd)
  2. Vector Map (.csv)
  3. TF File (.launch)

How to start localization

  1. Launch Autoware’s Runtime Manager
  2. Go to Setup Tab
  3. Click on TF and Vehicle Model buttons

    This will create the transformation between the localizer (Velodyne) to the base_link frame (car’s tires)
  4. Go to Map tab
  5. Click the ref button on the PointCloud section
  6. Select ALL the PCD Files that form the map, then click Open
  7. Click the Point Cloud button to the left. A bar below will show the progress of the load. Wait until it’s complete.

    Do the same for Vector Map, but this time select all the csv files
    Finally load the TF for the map
  8. Go to Simulation Tab
  9. Click the Ref Button and load a ROSBAG.
  10. Click Play and once the ROSBAG started to play, immediatly press Pause

    This step is required to set the Time to simulation instead of real.
  11. IF your rosbag contains /velodyne_packets instead of /points_raw, go to Sensing tab and Launch the LiDAR node corresponding to your sensor, to decode the packets into points_raw

    The corresponding calibration YML files are located in ${AUTOWARE_PATH}/ros/src/sensing/drivers/lidar/packages/velodyne/velodyne_pointcloud/params/
    Select the correct one depending on the sensor.
  12. In the Sensing Tab, Inside the *Points Downsampler** section, click on voxel_grid_filter
  13. Go to Computing tab and click on the [app] button next to ndt_matching inside the Localization section. Make sure the Initial Pos is selected.
  14. Click on the ndt_matching checkbox.
  15. Launch RVIZ using the button below Runtime Manager, and load the default.rviz configuration file located in ${AUTOWARE_PATH}/ros/src/.config/rviz.
  16. In RVIZ click on the 2D Pose Estimate button, located in the top bar
  17. Click on the initial position to start localization AND drag to give an initial pose.
  18. If the initial position and pose are correct, the car model should now be seen in the correct position.
    If the model starts spinning, try to give a new initial position and pose.

Building Apollo outside of Docker

WARNING – this will probably mess up your system.
At some point I had an unrelated crash so I restarted my computer and now it is stuck in a boot loop. So don’t attempt this until I have updated with a safe version!

I will probably try this again in a VM, assuming graphics drivers will work. Apollo is extremely fussy about library versions so be warned!

Current status: Bricked my OS, but at last attempt built until hitting some ROS dependency issues, and some linker problems probably caused by Apollo wanting some archaic glibc version..

You must not have Boost or PCL installed before attempting this, as Apollo requires specific versions. You’ll also need to make sure you have the correct version of GCC (4.8) and probably glibc (TODO).

sudo apt-get install gcc-4.8 g++-4.8 g++-4.8-multilib gcc-4.8-multilib
sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-4.8 50
sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-4.8 50

Now you need to build Boost.

wget http://downloads.sourceforge.net/project/boost/boost/1.54.0/boost_1_54_0.tar.gz
tar -zxvf boost_1_54_0.tar.gz
cd boost_1_54_0

You will also need to modify the Boost source code to get the threading library to compile with the old GCC version – see here:
Then you can build boost as normal.

./bootstrap.sh --prefix=/usr/local
sudo ./b2 --with=all -j 4

Now get some dependencies:

sudo apt-get install -y build-essential apt-utils curl debconf-utils doxygen lcov libcurl4-openssl-dev libfreetype6-dev lsof python-pip python-matplotlib python-scipy python-software-properties realpath software-properties-common unzip vim nano wget zip cppcheck libgtest-dev git bc apt-transport-https shellcheck cmake gdb psmisc python-empy librosconsole0d librosconsole-dev libtf-conversions0d
sudo add-apt-repository ppa:webupd8team/java
sudo apt-get install -y oracle-java8-installer

Install Bazel (again, very fussy with which version):

wget https://github.com/bazelbuild/bazel/releases/download/0.5.3/bazel_0.5.3-linux-x86_64.deb
dpkg -i bazel_0.5.3-linux-x86_64.deb
sudo apt-get install -f

At this point I did some cleanup.

sudo apt-get clean autoclean
sudo apt-get autoremove -y
sudo rm -fr /var/lib/apt/lists/*

You have to build a specific version of protobuf. Get yourself a coffee ready:

wget https://github.com/google/protobuf/releases/download/v3.3.0/protobuf-cpp-3.3.0.tar.gz
tar xzf protobuf-cpp-3.3.0.tar.gz
cd protobuf-3.3.0/
./configure --prefix=/usr
make
sudo make install
sudo chmod 755 /usr/bin/protoc

Moving on to install node JS (again, a specific version required here):

wget https://github.com/tj/n/archive/v2.1.0.tar.gz
tar xzf v2.1.0.tar.gz
cd n-2.1.0/
sudo make install
sudo n 8.0.0

Make up an empty file called py27_requirements.txt and insert the following:

glog
protobuf == 3.1
python-gflags
flask
flask-socketio
gevent
requests >= 2.18
simplejson
PyYAML
catkin_pkg
rospkg

Now install the python corresponding bits and pieces for Python:

pip install -r py27_requirements.txt

I think the following might have to be done earlier, and might also get the wrong version of ROS given current build errors. Investigation needed (TODO):

curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add -
echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.list
sudo apt-get update && sudo apt-get install -y yarn

sudo sh -c 'echo "deb http://packages.ros.org/ros/ubuntu $(lsb_release -sc) main" > /etc/apt/sources.list.d/ros-latest.list'
sudo apt-key adv --keyserver hkp://ha.pool.sks-keyservers.net:80 --recv-key 421C365BD9FF1F717815A3895523BAEEB01FA116

And some more pesky libraries:

sudo apt-get install -y libbz2-dev libconsole-bridge-dev liblog4cxx10-dev libeigen3-dev liblz4-dev libpoco-dev libproj-dev libtinyxml-dev libyaml-cpp-dev sip-dev uuid-dev zlib1g-dev

sudo apt-get install -y libatlas-base-dev libflann-dev libhdf5-serial-dev libicu-dev libleveldb-dev liblmdb-dev libopencv-dev libopenni-dev libqhull-dev libsnappy-dev libvtk5-dev libvtk5-qt4-dev mpi-default-dev

sudo apt-get update && apt-get install -y --force-yes libglfw3 libglfw3-dev freeglut3-dev

You have to build glew to get the right version:

wget https://github.com/nigels-com/glew/releases/download/glew-2.0.0/glew-2.0.0.zip
unzip glew-2.0.0.zip
cd glew-2.0.0/
make
sudo make install

Same kind of thing for PCL:

wget https://github.com/PointCloudLibrary/pcl/archive/pcl-1.7.2.tar.gz
tar xzf pcl-1.7.2.tar.gz
cd pcl-pcl-1.7.2 && mkdir build && cd build
cmake ..
make
sudo make install

And the big bad CUDA (you need CUDA 8.0):

sudo apt-get install -y libgflags-dev libgoogle-glog-dev
wget https://developer.nvidia.com/compute/cuda/8.0/Prod2/local_installers/cuda-repo-ubuntu1604-8-0-local-ga2_8.0.61-1_amd64-deb
sudo dpkg -i cuda-repo-ubuntu1604-9-0-local_9.0.176-1_amd64-deb
sudo apt-key add /var/cuda-repo-9-0-local/7fa2af80.pub
sudo apt-get update
sudo apt-get install -y cuda-8-0

Next up is Caffe:

wget https://github.com/BVLC/caffe/archive/rc5.zip
unzip rc5.zip
cd caffe-rcg/
cp Makefile.config.example Makefile.config

In the file Makefile.config you have to uncomment the line WITH_PYTHON_LAYER := 1 before continuing.

sudo ln -s libhdf5_serial.so.10.1.0 libhdf5.so
sudo ln -s libhdf5_serial_hl.so.10.0.2 libhdf5_hl.so
cd python
for req in $(cat requirements.txt); do pip install --user $req; done
cd..
make all
make test
make runtest
make distribute

Now you can get to the actual Apollo part.

wget https://github.com/ApolloAuto/apollo/archive/master.zip
unzip master.zip
cd apollo-master
./apollo.sh build-gpu #will fail
source bazel-apollo/external/ros/setup.bash
./apollo.sh build-gpu

You might need to grab the docker image before doing the Apollo build – I will test if it works without. I think this is because it grabs a special, Apollo-specific version of ROS. You can see why they recommend you keep within the Docker!

How to create a map using NDT Mapping

  1. Go to Autoware/ros directory
  2. Run Autoware using “./run” command
  3. Go to Simulation tab and Load a ROSBAG
  4. Click Play and immediately PAUSE
  5. Click Computing tab and select ndt_mapping
  6. Click RViz button at the bottom
  7. In Rviz click File menu and then click Open Config to select visualization template for ndt_mapping.rviz located in Autoware/src/.config/rviz
  8. ndt_mapping will read from /points_raw
    IF the pointcloud is being published in a different topic, use the relay tool in a new terminal window
    rosrun topic_tools relay /front/velodyne_points /points_raw
    This will forward the topic /front/velodyne_points to /points_raw
  9. Go back to Simulation tab and click Pause to start mapping
  10. Mapping process can be seen from Rviz
  11. Once the desired area is mapped. Click [app] button next to ndt_mapping
  12. Select the desired path specified using the Ref button
  13. Press the PCD OUTPUT to generate the file.
  14. Uncheck the ndt_mapping node to stop.

DONE

How to verify Map

  1. Select Map tab in runtime Manager and click on Ref button
  2. Select the recently created file
  3. Click on the PointCloud button and wait until the progress bar reaches Loading… 100%
  4. Open RVIZ, Click the ADD button
  5. Select the By Topic Tab
  6. Double Click on /points_map PointCloud2
  7. The map will be displayed (remember to set the frame to map)

How to create waypoints for path following

Method 1: using Rosbag, PCD Map and NDT Matching

  1. Load all the nodes required by NDT (map, nmea*, tf, voxel, ndt_matching) and verify ndt is working
  2. Launch vel_pose_connect

current_pose: /ndt_pose
current_velocity/estimate_twist

  1. Launch waypoint_saver

[X] Check Save /current_velocity
Interval 1 (store a waypoint every meter)

  1. In Rviz add the topic waypoint_saver_marker
  2. Play the rosbag, the path followed by NDT will be stored as a series of waypoints in the specified CSV file, in the same coordinates as the PCD map’s frame

Method 2: Using Vector Map, TF, Lane and RViz’s Publish Point

  1. Launch waypoint_clicker

velocity
output file

  1. In Rviz add the topic /waypoint_guide

Small points belonging to the lanes will be displayed.

  1. Use the Published Point in Rviz to click the desired points in the lane to create a path. (Points can only be added following the lane direction)

Load the path to confirm the file

The file can be loaded using a text editor.

  1. Select the csv file and launch waypoint_loader

  1. In Rviz add the /global_waypoints_marker

Run Autoware full stack

Nodes required to run the autonomous car

RuntimeManager Tab Path Following (PF) Nodes + Traffic Signal
Setup TF
Map PointCloud,VectorMap,TF VectorMap, TF
Sensing LiDAR, VoxelGrid filter Camera, Calibration Publisher
Computing Euclidean Cluster or Ground Filtering (Ray or Ring)
Computing/Localization NDT Matching, vel_pose_connect
Computing/Mission Planning lane_rule, lane_stop, lane_select, astar_navi
Computing/Motion Planning obstacle_avoid, velocity_set, way_point_loader, pure_pursuit, twist_filter
Computing/Detection feat_proj, region_tlr
Interface,(if ZMP) vehicle_gateway
Interface,(if AStuff) pac_mod

* (if lane change is required)