2023

Machine Learning Robot

Lab Manual

ADVANCED ROBOTICS AND MECHATRONICS LAB

DREXEL UNIVERSITY | Engineering and Technology

 Table of Contents

I. Introduction…………………………………………………………………………………………………………………2
II. Objectives……………………………………………………………….…………………………………………………..2
III. Prerequisites……………………………………………………………..………………………………………………..2
IV. Software and Hardware Requirements……………………….………………………………………………2
V. Robot Platform……………………………………………………………………………………………………………2

1. Lab Sessions……………………………………………………………………………………………………………….……….3
1.1 Lab 1: Setting Up the Robot Environment………………………………………………………………………….3
1.2 Lab 2: Basic Robot Movement and Control……………………………………………………………………....5
1.3 Lab 3: Introduction to Machine Learning Algorithms………………………………………………………..6
1.4 Lab 4: Implementing Supervised Learning on the Robot…………………………………………………...8
1.5 Lab 5: Implementing Unsupervised Learning on the Robot………………………………………………10
1.6 Lab 6: Implementing Reinforcement Learning on the Robot…………………………………………….12
1.7 Lab 7: Designing and Evaluating a Custom Machine Learning Model……………………………….14

2. Final Project………………………………………………………………………………………………………………………16
3. Additional Resources…………………………………………………………………………………………………………17

1|Page
 I. Introduction

This lab manual for machine learning robots aims to provide a comprehensive guide for students who
wish to explore the fascinating intersection of robotics and machine learning. As machine learning
continues to revolutionize various industries, its applications in robotics have led to the development of
intelligent systems that can autonomously navigate, manipulate objects, and learn from their
environments. Throughout this course, students will gain hands-on experience in implementing and
evaluating machine learning algorithms on a robot platform. The course is designed to cover essential
topics such as basic robot movement and control, supervised learning, unsupervised learning, and
reinforcement learning, providing students with a strong foundation in both robotics and machine
learning.

II. Objectives

The primary objectives of this lab manual are to provide students with a solid understanding of machine
learning algorithms as applied to robotics. Enable students to implement and evaluate machine learning
models on a robot platform. Develop critical problem-solving and analytical skills in the context of
robotics and machine learning.

III. Prerequisites

a. Basic knowledge of programming (Python is preferred)

b. Basic understanding of linear algebra, calculus, and probability theory

IV. Software and Hardware Requirements

a) Python (version 3.7 or higher)

b) Robot Operating System (ROS)
c) TensorFlow or PyTorch
d) A robot platform with sensors and actuators (e.g., TurtleBot or similar)
e) Gazebo

V. Robot Platform

This course will use ROS for lab sessions. The robot is equipped with sensors such as cameras, lidar, and
encoders and actuators like motors and servos in addition students will be taught how to use Gazebo.

2|Page
 LAB SESSIONS

LAB 1: SETTING UP THE ROBOT ENVIRONMENT

In this lab, you will learn to install and configure the necessary software and set up the robot platform.
This setup is crucial to creating a functional robot environment that enables the development, testing,
and deployment of robotics applications.

1. Install and configure the necessary software:

a. Install Robot Operating System (ROS):

ROS is a popular open-source robotics middleware framework that provides tools, libraries, and
conventions for developing robot applications. Choose the ROS distribution suitable for your system
(e.g., ROS Noetic for Ubuntu 20.04) and follow the installation instructions on the ROS Wiki
(http://wiki.ros.org/).

b. Install Gazebo (optional):

Gazebo is a 3D robot simulator that can be used alongside ROS for simulating robot systems in realistic
environments. If you plan to use Gazebo, install it by following the instructions on the Gazebo website
(http://gazebosim.org/).

c. Install additional ROS packages:

Depending on your robot platform and project requirements, you may need to install additional ROS
packages (e.g., sensor drivers, navigation stack, etc.). Use the 'apt' package manager or 'rosdep' to install
the required packages.

d. Set up a development environment:

Choose a development environment (e.g., Visual Studio Code, Atom, or Sublime Text) and install the
necessary plugins for ROS and your preferred programming language (e.g., Python or C++).

2. Connect and set up the robot platform:

a. Assemble the robot hardware:

Assemble the robot platform according to the manufacturer's instructions, ensuring that all mechanical
and electrical components are correctly connected.

b. Connect sensors and actuators:

Attach any sensors (e.g., cameras, LIDAR, IMU) and actuators (e.g., motors, servos) to the robot's control
board or microcontroller. Ensure that the wiring is correct and secure.

c. Install the robot's ROS driver (if applicable):

Some robots come with ROS drivers that facilitate communication between the robot's hardware and
ROS software. Install the ROS driver for your robot by following the manufacturer's instructions or by
searching for the driver in the ROS Wiki or GitHub.

3|Page
d. Configure the robot's launch file:

Create or modify the robot's launch file to include the required nodes, parameters, and topics for your
robot platform. This file is used to start the robot's software components when the robot is powered on.

e. Test the robot's basic functionality:

Run the launch file and verify that the robot's sensors and actuators are functioning correctly. Use ROS
tools like 'rostopic', 'roslaunch', and 'rviz' to visualize sensor data and confirm that the robot responds to
commands.

After completing Lab 1, you will have set up a functional robot environment, allowing you to proceed
with further development, such as implementing algorithms, control strategies, and perception systems.

4|Page
 LAB 2: BASIC ROBOT MOVEMENT AND CONTROL

Objective: The objective of this lab is to familiarize you with robot kinematics and dynamics, as well as
the practical implementation of basic robot movement using Python and the Robot Operating System
(ROS).

Part 1: Robot Kinematics and Dynamics

In this part of the lab, you will learn about the fundamentals of robot kinematics and dynamics, which
are essential for understanding robot movement and control.

Kinematics: Kinematics deals with the study of motion without considering the forces causing it. In
robotics, kinematics focuses on the geometrical relationships between the robot's components and how
they move in relation to each other. Forward kinematics and inverse kinematics are the two main types
of kinematic analysis.

Forward Kinematics: Given the joint angles or positions, forward kinematics computes the end-effector
(e.g., gripper) position in the robot's workspace.
Inverse Kinematics: Inverse kinematics is the reverse process, determining the joint angles or positions
required to achieve a desired end-effector position.
Dynamics: Dynamics, on the other hand, is the study of motion considering the forces and torques
involved. Robot dynamics includes the analysis of forces, torques, and accelerations in a robotic system,
often used to develop control strategies for actuating robot joints.

Part 2: Implementing Basic Robot Movement with Python and ROS

In this part of the lab, you will implement basic robot movement using Python and ROS. This will involve
the following steps:

Setup: Install the necessary ROS packages, create a workspace, and set up a basic robot simulation
environment.

Python Script: Write a Python script to control the robot's movement. This script will leverage ROS
libraries and interfaces to send commands to the robot's actuators.

Subscribing to Sensor Data: Implement a ROS subscriber in the Python script to receive sensor data from
the robot, such as position, velocity, or force readings.

Implementing Movement: Using the robot kinematics and dynamics knowledge from Part 1, create
functions in the Python script to control the robot's movement. This could include moving in a straight
line, rotating in place, or following a predefined trajectory.

ROS Communication: Set up the necessary ROS publishers, subscribers, and services to enable
communication between the Python script and the robot's hardware or simulation environment.

Testing: Test the Python script in the simulation environment to verify the desired robot movements are
being executed correctly.

By the end of Lab 2, you should have a better understanding of robot kinematics and dynamics and be
able to implement basic robot movement and control using Python and ROS.

5|Page
 LAB 3: INTRODUCTION TO MACHINE LEARNING ALGORITHMS

This lab aims to introduce you to the basics of machine learning (ML) algorithms, with a focus on their
application in robotics. The lab covers three primary learning paradigms: supervised learning,
unsupervised learning, and reinforcement learning. Additionally, the lab discusses some common
machine-learning algorithms used in robotics.

Supervised Learning:

Supervised learning is a type of ML where the algorithm learns to map input data to output labels based
on a set of labeled training examples. The goal is to generalize from the provided examples to make
accurate predictions on unseen data. Supervised learning is commonly used for tasks such as
classification and regression.

Common supervised learning algorithms in robotics include:

a. Linear Regression: A technique used to model the relationship between a dependent variable and one
or more independent variables. It is often used for predicting continuous outcomes.

b. Logistic Regression: A classification algorithm used to predict the probability of an instance belonging
to a particular class. It is particularly useful for binary classification tasks.

c. Support Vector Machines (SVM): SVM is a classification and regression algorithm that seeks to find the
optimal hyperplane that maximizes the margin between different classes.

d. Artificial Neural Networks (ANN): ANNs are computing systems inspired by the neural networks
present in the human brain. They consist of interconnected nodes called neurons and are used for a
wide range of tasks, including image recognition, natural language processing, and control tasks in
robotics.

Unsupervised Learning:

Unsupervised learning is a type of ML where the algorithm learns patterns and structures within the
input data without relying on labeled examples. It is commonly used for tasks such as clustering,
dimensionality reduction, and anomaly detection.

Common unsupervised learning algorithms in robotics include:

a. K-means Clustering: An algorithm that groups data points into 'K' clusters based on similarity.

b. Principal Component Analysis (PCA): A technique for dimensionality reduction that transforms data
into a new coordinate system by projecting it onto orthogonal axes that maximize variance.

6|Page
Reinforcement Learning:

Reinforcement learning is a type of ML where an agent learns to make decisions by interacting with an
environment. The agent learns to maximize cumulative rewards by exploring and exploiting different
actions in various states. Reinforcement learning is widely used in robotics for tasks such as navigation,
manipulation, and control.

Common reinforcement learning algorithms in robotics include:

a. Q-Learning: A model-free, value-based reinforcement learning algorithm that learns the optimal
action-value function using a tabular approach or function approximators like neural networks.

b. Deep Q-Network (DQN): An extension of Q-learning that combines the power of deep neural
networks to generalize complex, high-dimensional state spaces.

c. Proximal Policy Optimization (PPO): A policy-based reinforcement learning algorithm that optimizes a
policy directly using a surrogate objective function and iteratively updates the policy.

By the end of this lab, you should have a solid understanding of the basics of supervised, unsupervised,
and reinforcement learning, as well as their application in robotics. This knowledge will be helpful in
designing and implementing ML algorithms for a variety of robotic tasks.

7|Page
 LAB 4: IMPLEMENTING SUPERVISED LEARNING ON THE ROBOT

Objective: Train a robot to perform a simple task using supervised learning techniques and evaluate the
performance of the trained model.

Overview: In this lab, you will implement supervised learning techniques to train a robot to perform a
specific task. You will collect a dataset of input-output pairs, train a machine learning model using this
dataset, and evaluate the model's performance on unseen data.

Task definition and data collection:

Choose a simple task for the robot to perform, such as object recognition, navigation, or grasping.
Define the inputs and outputs for the task. For example, if you choose object recognition, the input
could be images of objects, and the output could be the class labels of the objects.

Collect a dataset of input-output pairs by manually performing the task with the robot or by using a
simulator like Gazebo. Ensure that the dataset is diverse and representative of various situations the
robot might encounter.

Data preprocessing:

Process the collected data to prepare it for training. This may involve normalizing the data, converting it
into appropriate formats, and splitting it into training and validation sets. For image-based tasks, you
may need to resize or augment the images to improve model performance.

Model selection and training:

Choose a machine learning model suitable for your task. Popular choices include convolutional neural
networks (CNNs) for image-based tasks, recurrent neural networks (RNNs) for sequential data, or other
supervised learning algorithms like decision trees, SVMs, or k-NN. You can use libraries like TensorFlow
or PyTorch to implement and train your model.

Train the model using the training set and validate its performance using the validation set. Tune the
model's hyperparameters to improve its performance, such as learning rate, batch size, or the number
of layers.

Model evaluation:

After training, evaluate the performance of the model on a test set of input-output pairs not used during
training. Use performance metrics like accuracy, precision, recall, F1-score, or mean squared error,
depending on your task. Analyze the results to identify areas where the model performs well and areas
that need improvement.

Model deployment and real-world testing:

Deploy the trained model on the robot and test it in a real-world environment. Monitor the robot's
performance and gather insights into its behavior. Identify any discrepancies between simulation and
real-world performance and address them by improving the model or data collection process.

8|Page
Conclusion: By the end of this lab, you should have successfully trained a robot to perform a simple task
using supervised learning techniques and evaluated its performance in both simulated and real-world
environments. The skills and concepts learned can be applied to more complex tasks and other types of
learning algorithms, such as reinforcement learning or unsupervised learning.

9|Page
 LAB 5: IMPLEMENTING UNSUPERVISED LEARNING ON THE ROBOT

Objective:

In this lab, the goal is to implement an unsupervised learning algorithm to cluster or classify sensor data
collected from a robot. You will analyze the results and discuss potential applications of the algorithm in
robotics or other fields.

Overview:

Unsupervised learning algorithms are a class of machine learning techniques that can identify patterns
or structures in data without labeled training examples. In the context of robotics, unsupervised learning
can be useful for processing sensor data, understanding the environment, and making decisions based
on observed patterns.

Steps:

a. Data Collection:

Begin by collecting sensor data from your robot. This may include data from various sensors such as
cameras, LIDAR, sonar, or IMU (Inertial Measurement Unit). Ensure that you have a diverse dataset
representing different scenarios or environments the robot may encounter.

b. Data Preprocessing:

Preprocess the collected data to remove noise, normalize values, and transform it into a suitable format
for the unsupervised learning algorithm. This may involve cleaning the data, converting it into numerical
representations, and scaling the features.

c. Algorithm Selection:

Choose an appropriate unsupervised learning algorithm for your task. Popular algorithms for clustering
or classification include K-Means, DBSCAN, and hierarchical clustering. Consider the nature of your data,
the desired output, and the computational complexity of the algorithm when making your selection.

d. Model Training:

Train the chosen unsupervised learning algorithm on your preprocessed sensor data. Depending on the
algorithm, you may need to tune hyperparameters (such as the number of clusters in K-Means) or set
distance metrics (e.g., Euclidean or Manhattan distance). Monitor the training process to ensure
convergence or to avoid overfitting.

e. Model Evaluation:

Evaluate the performance of your unsupervised learning model by visualizing the clusters or classes
formed and analyzing the distribution of sensor data points within them. Use relevant evaluation
metrics, such as silhouette score or adjusted Rand index, to quantify the quality of the clustering or
classification.

10 | P a g e
 f. Results Analysis & Applications:

Analyze the results of your unsupervised learning algorithm to identify patterns or structures in the
sensor data. Discuss potential applications of these findings in robotics or other fields, such as
environment mapping, object recognition, anomaly detection, or decision-making.

g. Conclusion:

Summarize your work, highlighting the key findings and their potential applications. Discuss any
limitations or challenges encountered during the implementation and suggest possible improvements or
future work in this area.

11 | P a g e
 LAB 6: IMPLEMENTING REINFORCEMENT LEARNING ON THE ROBOT

Objective: Train a robot to complete a task using reinforcement learning techniques, evaluate the
robot's performance, and discuss the challenges encountered.

Overview:

In this lab, you will implement a reinforcement learning (RL) algorithm to train a robot to perform a
specific task. You will utilize simulation tools to train the robot in a virtual environment before deploying
it in the real world. Finally, you will evaluate the robot's performance and discuss the challenges
encountered during the implementation.

Problem Definition:

Begin by defining the task the robot needs to perform. This could be a simple task like navigating
through a maze or a more complex task like picking and placing objects. Clearly define the objective, the
environment, and the constraints of the problem.

Choose a Reinforcement Learning Algorithm:

Select a suitable RL algorithm for the task, such as Q-learning, Deep Q-Networks (DQN), Proximal Policy
Optimization (PPO), or Soft Actor-Critic (SAC). Consider factors like the complexity of the task, the size of
the state and action spaces, and the desired level of exploration vs. exploitation.

Set up the Simulation Environment:

Using a simulator like Gazebo or ROS, create a virtual environment that closely resembles the real-world
scenario in which the robot will operate. This environment will be used for training and evaluating the
RL algorithm.

Implement the RL Algorithm:

Implement the chosen RL algorithm, integrating it with the robot's control system and the simulation
environment. This includes defining the state and action spaces, the reward function, and any other
necessary components.

Train the Robot:

Train the robot in the simulation environment using the implemented RL algorithm. Monitor the robot's
progress and adjust the algorithm's hyperparameters as needed to achieve the desired level of
performance.

Evaluate Performance:

Once the robot has been trained, evaluate its performance in the simulation environment by measuring
metrics such as success rate, completion time, and cumulative reward. Additionally, test the robot's
performance in the real world to ensure that the learned behavior generalizes well.

12 | P a g e
Discussion of Challenges:

Discuss the challenges encountered during the implementation of the RL algorithm, including issues
related to exploration, sample efficiency, stability of learning, and sim-to-real transfer. Identify areas for
improvement and propose potential solutions to overcome these challenges.

By the end of this lab, you will have gained hands-on experience implementing reinforcement learning
techniques to train a robot for a specific task, evaluating its performance, and discussing the challenges
encountered during the process.

13 | P a g e
 LAB 7: DESIGNING AND EVALUATING A CUSTOM MACHINE LEARNING MODEL

Objective:

The objective of this lab is to design, implement, and evaluate a custom machine-learning model for a
specific robotic application. This process involves understanding the problem, collecting and processing
data, selecting an appropriate algorithm, training and tuning the model, and evaluating its performance
on the robot platform.

Steps

Problem definition:

Identify the robotic application for which you want to develop a custom machine-learning model. It
could be a robotic arm performing pick-and-place tasks, a mobile robot navigating through an
environment, or an autonomous drone following a predetermined path. Clearly define the problem,
goals, and constraints of the application.

Data collection and processing:

Gather the required data for training and testing the machine learning model. This data could include
sensor readings, robot states, or even images and videos. Make sure to preprocess the data by cleaning
it, normalizing it, and splitting it into training and testing sets.

Algorithm selection:

Choose an appropriate machine learning algorithm based on the problem and data. For instance, if the
task involves image recognition, consider using convolutional neural networks (CNNs); if the task
involves predicting continuous values, use regression algorithms like linear regression or support vector
regression.

Model implementation:

Implement the selected algorithm using a machine learning library like TensorFlow or PyTorch. Define
the model architecture, loss function, and optimization method. Initialize the model with the
appropriate hyperparameters and train it using the training dataset.

Model evaluation:

Evaluate the performance of the trained model using the testing dataset. Calculate relevant evaluation
metrics such as accuracy, precision, recall, and F1 score for classification tasks, or mean squared error
(MSE) and R2 score for regression tasks. Analyze the results and identify any overfitting or underfitting
issues.

Model tuning:

Tune the model by adjusting its hyperparameters, changing its architecture, or incorporating
regularization techniques. Retrain the model and evaluate its performance again, iterating until
satisfactory results are obtained.

14 | P a g e
Deployment on the robot platform:

Integrate the trained machine learning model into the robot's software stack. This could involve
developing a ROS node or implementing the model in the robot's control system. Test the model's
performance in the real-world robotic application and evaluate its effectiveness.

Conclusion:

Analyze the results of the implemented machine learning model on the robot platform. Discuss the
model's strengths, limitations, and potential improvements. Finally, consider how the model could be
generalized to other similar robotic applications or adapted to different robot platforms.

15 | P a g e
 FINAL PROJECT

In the final project, students will apply the knowledge and skills acquired throughout the lab sessions to
develop a machine learning-based robotic application. The project should involve:

Defining a problem statement and the scope of the project

Designing and implementing a machine-learning model

Integrating the model with the robot platform

Evaluating the performance of the model and discussing the results

Preparing a project report and presentation

The final project can be completed individually or in small groups. Students are encouraged to discuss
their project ideas with the instructor or lab assistants for guidance and feedback.

16 | P a g e
Additional Resources

 Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep Learning. MIT Press.
 Murphy, K. P. (2012). Machine Learning: A Probabilistic Perspective. MIT Press.
 Quigley, M., Gerkey, B., & Smart, W. D. (2015). Programming Robots with ROS. O'Reilly Media,
Inc.
 Thrun, S., Burgard, W., & Fox, D. (2005). Probabilistic Robotics. MIT Press.

Online Resources:

 ROS Wiki (http://wiki.ros.org/):

The Robot Operating System (ROS) Wiki is a comprehensive online resource for the open-source
robotics middleware framework, ROS. It offers detailed documentation, tutorials, and guides for
using ROS in various robotics applications, including hardware interfacing, software development,
and simulation.

 TensorFlow (https://www.tensorflow.org/):

TensorFlow is an open-source machine-learning library developed by Google. It provides a flexible

platform for building, training, and deploying machine learning models. The TensorFlow website
offers extensive documentation, tutorials, and resources for users ranging from beginners to experts
in the field of deep learning and artificial intelligence.

 PyTorch (https://pytorch.org/):

PyTorch is an open-source machine learning library developed by Facebook's AI Research Lab (FAIR).
It is known for its dynamic computation graph and efficient tensor computation, making it popular
among researchers and developers. The PyTorch website contains detailed documentation,
tutorials, and community resources to help users develop and deploy deep learning models.

 Gazebo (http://gazebosim.org/)

Gazebo is a powerful open-source robot simulator that allows users to create realistic 3D environments
for simulating robot systems. It provides an extensive set of tools for modeling, rendering, and
simulating physics, making it useful for developing and testing robotics algorithms. The Gazebo website
offers comprehensive documentation, tutorials, and support resources for users.

In addition to the provided resources, students are encouraged to explore related literature, open-
source projects, and online resources to enhance their understanding of machine learning and robotics.

17 | P a g e