Ad Astra

Software Subsystem
Agenda for the
Session
OS requirements (Linux Ubuntu distribution)

Python

ROS2

UI

Yolo

Image Processing

Networking
 OS Requirements
Why Linux?
Robotics Middleware Support: Linux supports ROS (Robot Operating System), which is widely used for
robotics development, including sensor integration and motion planning.

Real-Time Processing: Linux offers low-latency performance, crucial for real-time control and navigation.

Driver Compatibility: Jetson's hardware drivers (for cameras, LIDAR, and motor controllers) are optimized
for Linux.

AI and Computer Vision Libraries: Access to NVIDIA's CUDA, cuDNN, and TensorRT libraries, essential for AI-
powered vision and decision-making.

Developer Tools and Community Support: Strong community support, along with tools like JetPack SDK
and NVIDIA Nsight, facilitate efficient development and debugging.

Flexibility: Linux based OS offers a high level of customization than any other OS in the market.
 OS Requirements
Basic Linux Commands
ls – Lists files and directories in the current directory.
ls -l # Shows detailed info like permissions, size, and modification date
ls src #Lists the contents of the src directory
ls src/sample #Lists the contents of sample directory inside src
ls arguments.
-l – Long listing format (shows details like permissions, owner, size, and modification date).
-a – Shows all files, including hidden files (those starting with a dot .)
-h – Human-readable sizes (e.g., KB, MB) when used with -l.
-1 – Lists one file per line.
-R – Recursively lists all files in subdirectories.
-S – Sorts files by size, largest first.
-r – Reverses the order of the sort (e.g., oldest first with -t).
 OS Requirements
Basic Linux Commands
cd – Changes the current directory.
cd /home/user/Documents # Navigates to the Documents folder
cd .. #Navigates to the previous directory
cd #Navigates to home directory

touch – Creates an empty file or updates the timestamp of an existing file.

touch newfile.txt # Creates an empty file named "newfile.txt"
touch file1.txt file2.txt file3.txt #Creates multiple files
touch file{1..10}.txt #Creates files in the range 1 to 10

pwd – Displays the present working directory.

pwd # Outputs the full path of the current directory
 OS Requirements
Basic Linux Commands

mkdir – Creates a new directory.

mkdir new_folder # Creates a folder named "new_folder"
mkdir -p x/y/z #If the parent directories don't exist, they will be created automatically.
mkdir -m 755 newdir #Set permission for the directory created

rm – Removes files or directories.

rm file.txt # Deletes "file.txt"
rm -r folder_name # Recursively deletes a directory and its contents
rm -f protectedfile.txt #Forcibly removes the file

cp – Copies files or directories.

cp source.txt destination.txt # Copies a file to a new location
cp -r source_dir/ target_dir/ # Recursively copies a directory
 Python
What is Python?

High-level, interpreted programming language.

Emphasizes readability and simplicity.

Interpreter based programming language.

It is both Procedural and Object Oriented.

It has the biggest developer community.

 Python
Why Python?

Simple syntax and rapid prototyping

Extensive libraries for sensors, communication, and control

Strong community support and documentation

Compatible with popular microcontrollers and single-board computers (e.g., Raspberry Pi)

Extensive Support for ROS packages.

Comprehensive library for image processing.

 Image Processing

OpenCV (Open Source Computer Vision Library) is an open-source library for computer vision, image
processing, and machine learning. It is widely used for real-time computer vision applications.

Object Detection and Recognition: Detects obstacles, targets, or landmarks for navigation and exploration.

Color Detection and Tracking: To track objects of specific colors, useful in tasks like target identification.

ArUco Detection: To detect and recognize ArUco tags.

Camera Calibration and Image Correction: Corrects distortions caused by the camera lens.
 Image Processing
OpenCV Basics
Importing OpenCV in Python

Read and Display Images

Save an Image
 Image Processing
OpenCV Basics
Stream Video from Webcam
 Image Processing
OpenCV Basics
Resize Image

Convert to Grayscale

Blur Image

Edge Detection using Canny

 YOLO
What is YOLO?

YOLO (You Only Look Once) is a state-of-the-art real-time object detection algorithm known for its speed
and accuracy. It detects objects in images or video feeds by predicting both bounding boxes and class
probabilities simultaneously.

Real-Time Performance: Capable of processing images at high frame rates, making it ideal for live video
feeds, like those on a rover.

Single Neural Network: YOLO uses a single convolutional neural network to predict multiple bounding
boxes and class probabilities in one go.

High Accuracy: Balances speed with accuracy, detecting objects with high precision.

Generalization: Performs well on a variety of objects and scenes due to its global reasoning approach.
 YOLO
Model Training

Create config file(.yaml)

 YOLO
Model Training

Load and train the model

 UI
Need for UI.

Purpose: Monitor and control the rover remotely using real-time data.

Technologies Used:
Flask: Backend framework to serve live data.
HTML/CSS: Frontend for displaying the feed with a user-friendly
interface.
JavaScript: For dynamic updates without page refresh.

Applications:
Remote navigation and control.
Surveillance in hazardous or hard-to-reach areas.
Scientific data collection and monitoring.
 UI
Architecture Overview

Rover Sensors & Camera → Capture live data.

Flask Server (On Rover) → Processes and streams data.

Image processing
Sensor data reading
IP hosting

Frontend (HTML/CSS) → Displays live feed on a web interface.

JavaScript → Ensures smooth real-time updates.

 Networking
What is Socket Programming?

Socket programming is a way to enable communication between devices over a network using IP
addresses and ports. It allows two devices to send and receive data, much like making a phone
call where one device dials (client) and the other answers (server).

Sensor Data Transmission:

Stream live feed from cameras, LIDAR, and other sensors to a remote base station.
Monitor environmental conditions or rover status (e.g., battery level).

Remote Control & Teleoperation:

Send commands (e.g., move forward, turn) from a control station to the rover.
Real-time remote steering and navigation.
 Networking
Server
 Networking
Client
 ROS
What is ROS?
ROS (Robot Operating System) is an open-source middleware framework for building and controlling
robots. It provides tools, libraries, and conventions designed to simplify the complex task of creating robust,
scalable robot software.

Modularity: Breaks down robot software into reusable packages and nodes.

Communication:
Topics: Publish/Subscribe messaging system for sensor data and commands.
Services: Request/Response communication for synchronous operations.
Actions: Asynchronous communication for long-running tasks.

Hardware Abstraction: Works with a variety of sensors, actuators, and computing platforms.

Simulation: Integration with simulators like Gazebo for testing in virtual environments.

Community and Ecosystem: Large community support and a vast collection of pre-built packages.
 ROS
Why use ROS?
Easy integration with sensors (LIDAR, GPS, cameras) and actuators (motors, servos).

Built-in libraries for navigation, path planning, and localization.

Reusability and modularity allow rapid prototyping and development.

Strong support for SLAM (Simultaneous Localization and Mapping).

Simple access to data and seamless communication over the network.

ROS abstracts hardware interfaces, making it easier to switch or upgrade components without

changing the control logic.

ROS is compatible with NVIDIA Jetson devices, leveraging CUDA and TensorRT for AI tasks like object

detection and visual navigation.

 ROS
ROS Commands
Initialize ROS Workspace

Create a New Package:

 ROS
ROS Commands
List Available Nodes

Run a Node

Run a Launch File

 ROS
ROS Commands
Visualize

Build the Workspace

Clean the Workspace

 ROS
Where is it used in the rover?

To send control signal over the network from base station to rover.

To access image feed and other sensor data from various sources

in the rover.

Path planning algorithms are implemented in ROS.

Web interface teleportation using ROS bridge.

Data transfer through antenna using UDP.

 ROS
Where is it used in the rover?

To send control signal over the network from base station to rover.

To access image feed and other sensor data from various sources

in the rover.

Path planning algorithms are implemented in ROS.

Web interface teleportation using ROS bridge.

Data transfer through antenna using UDP.