Advanced Robotics: Master ROS 2, Perception & AI-Driven Systems (2026)

Home  ›  Robotic Courses  › Advanced Robotics: Mastering Perception, Kinematics & AI

Move beyond the basics. Master industry-standard (Robot Operating System 2) ROS 2 Jazzy, robot kinematics, sensor fusion, and AI-driven computer vision — the complete stack used in production autonomous systems worldwide.

TL;DR — Quick Insights

• ROS 2 is the industry standard: Every serious robotics company — Boston Dynamics, Agility Robotics, NVIDIA, Waymo — uses ROS 2. This course teaches ROS 2 Jazzy Jalisco (LTS until 2029) with working code examples.
• Kinematics unlocks robot arms: Forward kinematics answers “where is my robot arm?” Inverse kinematics answers “how do I reach that position?” Master both and you can program any robotic manipulator.
• Perception is the hardest and most valuable skill: Building robots that can see, interpret depth, and classify objects is the highest-paid specialisation in robotics. This course covers the complete perception stack.
• From this course to production: The skills in this course are the exact prerequisites for the Expert Robotics course — Physical AI, VLA models, and sim-to-real pipelines.

Prerequisites

Before starting this course, complete Robotics for Beginners or confirm you already understand:

• What sensors and actuators are and how they connect
• Basic Python: variables, conditions, loops, functions
• What a microcontroller does

If you are unsure, the Beginners course takes 8–12 hours and requires no prior knowledge.

Course Overview

The Advanced Robotics course bridges the gap between foundational knowledge and real engineering capability. It covers the three technical layers that every professional robotics engineer must master: the mathematics of robot motion (kinematics), the perception stack that lets robots see and understand their environment, and ROS 2 — the middleware that connects every subsystem into a coordinated, production-ready robotic system.

• Focus: ROS 2, robot kinematics, sensor fusion, computer vision
• Approach: Theory with working code, real system examples, guided exercises
• Outcome: Build complete ROS 2 robotic systems; qualify for the Expert course

This course maps directly onto Route A and Route B of UDHY’s How to Become a Robotics Engineer career guide — the fastest paths for CS/EE and mechanical engineering backgrounds.

Who This Course Is For

• 🎓 Graduates of UDHY’s Beginner Robotics course ready to go deeper
• 💻 Computer science and engineering students who want industry-grade robotics skills
• 🚗 Developers transitioning into autonomous vehicles who need the robotics foundation
• 🔬 Researchers who want practical implementation skills to complement theoretical knowledge
• 🛠️ Engineers already working in adjacent fields who want to add robotics to their skill set

What You Will Build in This Course

By the end of this course, you will have built or simulated three complete systems:

1. A ROS 2 publisher-subscriber network — multiple nodes communicating sensor data and motor commands across a simulated robot
2. A 3-DOF robot arm kinematic solver — given a target position, compute the joint angles required to reach it (inverse kinematics in Python)
3. A real-time object detection pipeline — using OpenCV and a pre-trained YOLO model to detect and classify objects from a live camera feed in ROS 2

Module 4: Practical Robotics Mathematics

Robotics mathematics is the language robots use to understand position, movement, and space. You do not need a mathematics degree. You need four concepts and how to apply them in code.

4.1 Linear Algebra for Robotics: Rotation Matrices and Vectors

A robot arm operating in 3D space needs to know: where am I? which way am I facing? how do I get from here to there?

Position is described by a vector — a set of three numbers (x, y, z) representing a point in space relative to a reference frame.

Orientation is described by a rotation matrix — a 3×3 grid of numbers that encodes how much the robot has rotated around each axis. This sounds complex but in practice reduces to: “my arm is rotated 45 degrees around the vertical axis.”

In Python using NumPy:

import numpy as np

# Define a 45-degree rotation around the Z axis
theta = np.radians(45)
Rz = np.array([
    [np.cos(theta), -np.sin(theta), 0],
    [np.sin(theta),  np.cos(theta), 0],
    [0,              0,             1]
])

# Position of end-effector in local frame
p_local = np.array([0.5, 0, 0])  # 50 cm along X

# Position in world frame
p_world = Rz @ p_local
print(f"End-effector world position: {p_world}")

4.2 Forward Kinematics: “Where is my robot arm?”

Forward kinematics (FK) answers: given the angles of all my joints, where is my end-effector in 3D space?

For a 2-link planar arm with link lengths L1=0.3m and L2=0.25m:

import numpy as np

def forward_kinematics_2dof(theta1_deg, theta2_deg, L1=0.3, L2=0.25):
    """
    Compute end-effector (x, y) position for a 2-DOF planar arm.
    theta1: angle of first joint (degrees)
    theta2: angle of second joint relative to first link (degrees)
    """
    t1 = np.radians(theta1_deg)
    t2 = np.radians(theta2_deg)

    x = L1 * np.cos(t1) + L2 * np.cos(t1 + t2)
    y = L1 * np.sin(t1) + L2 * np.sin(t1 + t2)

    return round(x, 4), round(y, 4)

# Example: joint 1 at 30°, joint 2 at 45°
x, y = forward_kinematics_2dof(30, 45)
print(f"End-effector position: x={x}m, y={y}m")
# Output: End-effector position: x=0.2933m, y=0.3384m

4.3 Inverse Kinematics: “How do I reach that position?”

Inverse kinematics (IK) answers the opposite question: given a target position in space, what joint angles do I need to get there?

import numpy as np

def inverse_kinematics_2dof(x_target, y_target, L1=0.3, L2=0.25):
    """
    Compute joint angles for a 2-DOF planar arm to reach target (x, y).
    Returns (theta1, theta2) in degrees, or None if unreachable.
    """
    r = np.sqrt(x_target**2 + y_target**2)

    # Check reachability
    if r > (L1 + L2) or r < abs(L1 - L2):
        return None  # Target outside workspace

    # Compute elbow angle (theta2)
    cos_t2 = (r**2 - L1**2 - L2**2) / (2 * L1 * L2)
    sin_t2 = np.sqrt(1 - cos_t2**2)   # elbow-up solution
    theta2 = np.degrees(np.arctan2(sin_t2, cos_t2))

    # Compute shoulder angle (theta1)
    k1 = L1 + L2 * cos_t2
    k2 = L2 * sin_t2
    theta1 = np.degrees(np.arctan2(y_target, x_target) - np.arctan2(k2, k1))

    return round(theta1, 2), round(theta2, 2)

angles = inverse_kinematics_2dof(0.35, 0.25)
print(f"Required joint angles: θ1={angles[0]}°, θ2={angles[1]}°")

Module 5: Robot Perception & Computer Vision

Perception is the robot’s ability to extract meaningful information from raw sensor data. A camera returns 8 million pixel values per frame — meaningless numbers. A perception pipeline transforms those numbers into “there is a pedestrian 3 metres ahead, moving left at 1.2 m/s.”

This is the highest-valued skill in robotics in 2026. Perception engineers at Waymo, Aurora, and NVIDIA earn between $185,000 and $300,000. See the AI & Robotics Engineer Salary Guide 2026 for full compensation data.

5.1 Image Processing Fundamentals with OpenCV

import cv2
import numpy as np

# Load and process a robot camera frame
frame = cv2.imread('robot_camera_frame.jpg')

# Convert to grayscale for edge detection
gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)

# Apply Gaussian blur to reduce noise
blurred = cv2.GaussianBlur(gray, (5, 5), 0)

# Canny edge detection — finds object boundaries
edges = cv2.Canny(blurred, threshold1=50, threshold2=150)

# Show result
cv2.imshow('Detected Edges', edges)
cv2.waitKey(0)
cv2.destroyAllWindows()

5.2 Object Detection with YOLO

YOLO (You Only Look Once) is the standard real-time object detection model used in production robotics. Running YOLOv8 on a robot camera:

from ultralytics import YOLO
import cv2

model = YOLO('yolov8n.pt')  # nano model — fast for edge deployment

cap = cv2.VideoCapture(0)   # robot camera

while True:
    ret, frame = cap.read()
    results = model(frame, verbose=False)

    for box in results[0].boxes:
        cls = int(box.cls[0])
        label = model.names[cls]
        conf = float(box.conf[0])
        x1, y1, x2, y2 = map(int, box.xyxy[0])
        cv2.rectangle(frame, (x1, y1), (x2, y2), (0, 255, 0), 2)
        cv2.putText(frame, f"{label} {conf:.2f}",
                    (x1, y1-10), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0,255,0), 1)

    cv2.imshow('Robot Perception', frame)
    if cv2.waitKey(1) == ord('q'):
        break

5.3 Sensor Fusion: Combining LiDAR and Camera

A camera provides rich visual information but no depth. LiDAR provides precise depth but no colour or texture. Sensor fusion combines both, giving the robot rich visual context with metric depth — the foundation of every modern autonomous vehicle.

The architectures for production-grade sensor fusion — including BEV (Bird’s-Eye-View) perception — are covered in our deep-dive post: BEV Sensor Fusion with Spatiotemporal Transformers.

The Autonomous Navigation and SLAM course covers how fused sensor data feeds into map building and navigation.

Module 6: Mastering ROS 2 Jazzy — Industry-Standard Robotics Middleware

ROS 2 is the middleware that connects every component of a robotic system: sensors, perception pipelines, motion planners, and actuator controllers communicate through a standardised, real-time, distributed network of nodes and topics.

Every major robotics company — Boston Dynamics, Agility Robotics, NVIDIA Isaac, Waymo, Moovita — builds on ROS 2. Learning ROS 2 is not optional for a professional robotics engineer. It is the lingua franca of the field.

The current LTS distribution is ROS 2 Jazzy Jalisco (supported until May 2029), running on Ubuntu 24.04 LTS.

6.1 Installing ROS 2 Jazzy

# Set up locale
sudo apt update && sudo apt install locales
sudo locale-gen en_US en_US.UTF-8
export LANG=en_US.UTF-8

# Add ROS 2 repository
sudo apt install software-properties-common
sudo add-apt-repository universe
sudo curl -sSL https://raw.githubusercontent.com/ros/rosdistro/master/ros.key \
  -o /usr/share/keyrings/ros-archive-keyring.gpg
echo "deb [arch=$(dpkg --print-architecture) \
  signed-by=/usr/share/keyrings/ros-archive-keyring.gpg] \
  http://packages.ros.org/ros2/ubuntu \
  $(. /etc/os-release && echo $UBUNTU_CODENAME) main" | \
  sudo tee /etc/apt/sources.list.d/ros2.list > /dev/null

# Install ROS 2 Jazzy Desktop (includes RViz2, Gazebo, rqt)
sudo apt update && sudo apt install ros-jazzy-desktop

# Add to ~/.bashrc for automatic sourcing
echo "source /opt/ros/jazzy/setup.bash" >> ~/.bashrc
source ~/.bashrc

6.2 Your First ROS 2 Publisher-Subscriber System

# publisher_node.py — publishes robot sensor data
import rclpy
from rclpy.node import Node
from std_msgs.msg import Float32

class SensorPublisher(Node):
    def __init__(self):
        super().__init__('sensor_publisher')
        self.pub = self.create_publisher(Float32, 'distance', 10)
        self.timer = self.create_timer(0.1, self.publish_distance)

    def publish_distance(self):
        msg = Float32()
        msg.data = 42.5   # replace with real sensor read
        self.pub.publish(msg)
        self.get_logger().info(f'Distance: {msg.data} cm')

def main():
    rclpy.init()
    node = SensorPublisher()
    rclpy.spin(node)
    rclpy.shutdown()

# subscriber_node.py — receives and acts on sensor data
import rclpy
from rclpy.node import Node
from std_msgs.msg import Float32

class MotorController(Node):
    def __init__(self):
        super().__init__('motor_controller')
        self.sub = self.create_subscription(
            Float32, 'distance', self.react_to_distance, 10)

    def react_to_distance(self, msg):
        if msg.data < 20.0:
            self.get_logger().warn(f'Obstacle at {msg.data}cm — stopping!')
            # command: stop motors
        else:
            self.get_logger().info(f'Clear at {msg.data}cm — moving forward')
            # command: run motors

def main():
    rclpy.init()
    node = MotorController()
    rclpy.spin(node)
    rclpy.shutdown()

For a complete step-by-step ROS 2 Jazzy tutorial with Gazebo simulation, see: ROS 2 Jazzy Tutorial for Beginners.

6.3 Running Gazebo Harmonic Simulation

# Install TurtleBot3 simulation packages
sudo apt install ros-jazzy-turtlebot3*
export TURTLEBOT3_MODEL=waffle

# Launch simulation world
ros2 launch turtlebot3_gazebo turtlebot3_world.launch.py

# Drive with keyboard in a new terminal
ros2 run turtlebot3_teleop teleop_keyboard

# Visualise sensor data in RViz2
ros2 launch turtlebot3_bringup rviz2.launch.py

Are you a seasoned engineer looking to master modern foundation architectures and embodied intelligence? Take our flagship [Expert Physical AI and VLA Models Masterclass].

Your Next Step

After completing this course you are ready for:

• Expert Robotics → — Physical AI, VLA models, production ROS 2, sim-to-real
• Deep Learning for Robotics → — Neural networks for perception and control
• Autonomous Navigation and SLAM → — Map building, localisation, path planning

💡 Running the AI track in parallel?

This course pairs perfectly with Deep Learning for Robotics — Module 5 here (Perception) directly complements that course’s CNN and YOLO content. Running both simultaneously gives you the complete picture: the AI side and the robotics engineering side of the same perception pipeline.

Read alongside this course:

• Sensor Fusion Explained: Cameras, LiDAR & Radar — the multi-sensor stack this course implements
• Why Self-Driving Cars Still Fail — the real-world perception challenges your YOLO model must overcome
• The Complete Guide to AV Teleoperation — human-in-the-loop control that pairs with your ROS 2 learning
• How to Choose the Right LiDAR — hardware guide for the point cloud work in Module 5

Designed by Dr. Dilip Kumar Limbu — Former Principal Research Scientist, A*STAR · Co-Founder, Moovita, Singapore’s first autonomous vehicle company · 30 years building real-world autonomous systems. UDHY.com.