VIRTUAL TELEPRESENCE ROBOT USING

RASPBERRY Pi AND IOT

2nd Project review on 03.03.2021
Bhadri .V (18BEC008)
Gokul . P (18BEC013)
Karthick .V.M (18BEC022)
Karthik . K (18BEC023)
III B.SC (ECS)
Under the Guidance of :
Dr.O.M.SARAVANAKUMAR M.Sc., M.Phil., M.E,Ph.D.
Assistant Professor
Department of Electronics and Communication Systems
 VIRTUAL TELEPRESENCE ROBOT USING
RASPBERRY Pi and iot

 Introduction

 Literature review

 Proposed method

 Block diagram & Circuit Diagram

Raspberry pi camera interface

 Enabling camera functionality

Apache Web server

Rpi camera web interface

RPi cam movement with Android sensors

IMU sensors

Components used

 Project Budget

 Future work

 Conclusion
 INTRODUCTION
Tele-presence refers to a set of technologies which allow a person to feel as if they were
present, to give the appearance of being present, or to have an effect, via tele-robotics, at a
place other than their true location.
Tele-presence requires that the user senses be provided with such stimuli as to give the
feeling of being in that other location. Additionally, users may be given the ability to
affect the remote location.
In this case, the user's position, movements, actions, voice, etc. may be sensed,
transmitted and duplicated in the remote location to bring about this effect.
Therefore information may be travelling in both directions between the user and the
remote location.
The target of this project is to develop a system that helps people with disability to extend
their social interactions with the society. When one is sick in bed or unable to move out
of a restricted environment this system can help the user to observe and experience the
world
 LITRATURE REVIEW
I. Bluetooth based virtual telepresence robot , Shamin P. Shaji , electronicsforu

 This article describes about to build the virtual telepresence robot based on Bluetooth module HC-05
its explains to interface Rpi camera to raspberry pi 4 board .
LINK -https://www.electronicsforu.com/electronics-projects/virtual-telepresence-robot-raspberry-pi

II. Wi-Fi Controlled Robot Car Using Nodemcu and Blynk by Manitesla , instructables

This article helps to understand the working of Wi-Fi Controlled Robot using Blynk.
It's more effective while compare to Bluetooth controlled robot.
LINK- https://www.google.com/amp/s/www.instructables.com/Wifi-Controlled-Robot-Car-Using-Nodemcu

III.  Information on Accelerometers by Hobbyronics  

  This article gives us the clear knowledge about accelerometer and it's quite helpful for coding part.
LINK - https://www.hobbytronics.co.uk/accelerometer-info
 PROPOSED METHOD

 Raspberry pi 4 model B is the central component that interfaces user and the virtual
telepresence robot through Wi-Fi. Its used  video capture and 360° camera rotation.
 The video captured by camera can be viewed on a smartphone placed in the virtual reality
(VR) headset. The project lets the user experience virtual reality through VR headset. Dual-
screen mode is enabled in the smartphone for this purpose.
 The Smartphone reads the accelerometer and magnetometer data of the direction in
which the user turns his head, say, right or left. This data is sent to the modem over Wi-Fi
and to the RPi board, which in turn, provides these values as inputs to the servo motors.
 Servo motor is a rotatory actuator or linear actuator that allows for precise control of
angular or linear position, velocity and acceleration. It consists of a suitable motor
coupled to a sensor for position feedback.
 PROPOSED METHOD
 Two servo motors are used to move the camera one for the vertical movement and
the other for the horizontal movement. So when you turn your head along with VR
headset to the right side, the camera will also turn to the right direction.

 ESP8266 Board and BLYNK android application is used for controlling the robot
movement through Internet of things (IoT) .The BLYNK application's virtual keys
are configured for right left forward backward movements . These commands are
send to the ESP8266 Board through wi-fi.

 L298N is a typical motor driver module that allows DC motor to drive in either
direction. L298N can control a set of two DC motors simultaneously in any
direction.
Block diagram for Raspberry Pi camera movement
BLOCK DIAGRAM FOR WIFI ROBOT
CIRCUIT DIAGRAM FOR RPi CAMERA
MOVEMENT

MOTOR RASPBERRY PI GPIO

SERVO 1 GPIO 16
SERVO 2 GPIO 24
 CIRCUIT DIAGRAM FOR WI-FI ROBOT

NODE MCU MOTOR DRIVER

DIGITAL PINS INPUT PINS
D4 IN3
D5 IN4
D6 IN1
D7 IN2
 Blynk application
Blynk is a Platform with IOS and Android apps to control Arduino, Raspberry Pi and the
likes over the Internet.
 Blynk is the most popular IoT platform for connecting devices to the cloud, designing
apps to remotely control and monitor them, and managing thousands of deployed products.

Blynk architecture Control buttons for wi-fi car

 Raspberry pi camera interface
Pi Camera module is a camera which can be used to take pictures and high
definition video.
Raspberry Pi Board has CSI (Camera Serial Interface) interface to which we
can attach PiCamera module directly.
This Pi Camera module can attach to the Raspberry Pi’s CSI port using 15-pin
ribbon cable.
 enabling camera functionality
Start up your Raspberry Pi.
Go to the main menu and open the
Raspberry Pi Configuration tool.
 Select the Interfaces tab and ensure
that the camera is enabled
Reboot your Raspberry Pi.
Now our Camera Module is connected
and the software is enabled, using terminal
we can take pictures and videos by typing the
commands
"raspistill" and "raspivid" .
 Apache Web server
Apache is an open-source and free web server software.The official name is
Apache HTTP Server, and it’s maintained and developed by the Apache
Software Foundation.
It allows website owners to serve content on the web — hence the name “web
server.”
When someone wants to visit a website,
they enter a domain name into the address bar
of their browser.
Then, the web server delivers the requested files
by acting as a virtual delivery man.
File servers, database servers, mail servers, and web servers use different
kinds of server software. Each of these applications can access files stored
on a physical server and use them for various purposes.
The job of a web server is to serve websites on the internet. To achieve that
goal, it acts as a middleman between the server and client machines. It pulls
content from the server on each user request and delivers it to the web.
Web servers process files written in different programming languages such
as PHP, Python, Java, and others.
 Rpi camera web interface
RPi Cam Web Interface is a web interface for the Raspberry Pi Camera module.
It can be used for a wide variety of applications including surveillance, dvr
recording and time lapse photography
It is highly configurable and can be extended with the use of macro scripts. It
can be opened on any browser
Installation process:
Step 1 :- open terminal window on the raspberry Pi
Step 2 :- updating software using this commands
sudo apt-get update
sudo apt-get dist-upgrade
sudo apt-get install git
• Step 3 :- Clone the code from github and enable and run the install script
with the following commands:git clone
https://github.com/silvanmelchior/RPi_Cam_Web_Interface.gitcd
RPi_Cam_Web_Interface
• Step 4 :- entering our raspberry Pi " IP adress " one any web browser in
our case http://192.168.43.100/virtual/
• Step 5 :- now we can see a live stream from ah raspberry Pi camera
module on the Apache Web server
RPi cam movement with Android sensors
The Android smartphone raw sensor data is transmitted
by the "Wireless IMU" app to raspberry Pi via UDP on a
WiFi network
To map the real time response of the users head
movement to the video captured by the camera, we
make use of the inertial measurement unit which is in-
built in smartphones and VR headsets nowadays.
This helps is in avoiding an additional use of an IC and
helps us in the direct transfer of data between the sender
and reciever. Intertial Measurement Unit(IMU) is made of
these three sensors: Accelerometer, Gyroscope and
Magnotrometer
To mirror the rotation of the head of the user wearing the VR-headset, the data
from the IMU must be used.
This app sends the sensor readings to the target IP address and port via UDP as
Comma Separated Values (CSV).
This application can be run in the background to capture the data while one can
watch the live screening

Getting readings from IMU

 IMU sensors
• Accelerometer :-It measures linear acceleration (change of velocity) across a
single axis. Integrating acceleration once reveals an estimate for velocity, and
integrating again gives you an estimate for position.

• Gyroscope :-Gyroscopes, however, measure angular velocity about three axes:

pitch (x axis), roll (y axis) and yaw (z axis). When integrated with sensor fusion
software, a gyro can be used to determine an object’s orientation within 3D
space.

• Magnetometer : -A magnetometer, as the name suggests, measures magnetic

fields. It can detect fluctuations in Earth’s magnetic field, by measuring the air’s
magnetic flux density at the sensor’s point in space.
 COMPONTENTS USED
• Raspberry Pi 4 model B

 Raspberry Pi 4 model B
 Raspberry Pi camera module-5mp
NodeMCU
Motor driver - L298N
Lead acid battery for motors - 6v, 1.2 A
Dc motors -2
Servo motors -2
VR box
 Raspberry Pi 4 model B
DESCRIPTION :
The Raspberry pi 4 Model B is the latest product in the popular Raspberry Pi range of single
board computers. It offers ground-breaking increases in processor speed , multimedia performance, memory.

SPECIFICATION :

Broadcom BCM2711, Quad core Cortex-A72 (ARM v8) 64-bit SoC @ 1.5GHz
2GB, 4GB or 8GB LPDDR4-3200 SDRAM (depending on model)
2.4 GHz and 5.0 GHz IEEE 802.11ac wireless, Bluetooth 5.0, BLE
Gigabit Ethernet2 USB 3.0 ports; 2 USB 2.0 ports.
Raspberry Pi standard 40 pin GPIO header (fully backwards compatible with previous boards)
2 × micro-HDMI ports (up to 4kp60 supported)
2-lane MIPI DSI display port
2-lane MIPI CSI camera port
 Raspberry Pi 4 model B
H.265 (4kp60 decode), H264 (1080p60 decode, 1080p30 encode)
OpenGL ES 3.0 graphics
Micro-SD card slot for loading operating system and data storage
5V DC via USB-C connector (minimum 3A*)
5V DC via GPIO header (minimum 3A*)
Power over Ethernet (PoE) enabled (requires separate PoE HAT)
Operating temperature: 0 – 50 degrees C ambient
Raspberry Pi camera module
DESCRIPTION :
 Raspberry Pi camera module board is a high definition 5MP
camera that has the ability to capture good quality photos.

SPECIFICATION :

Interface Type: CSI(Camera Serial Interface)

Still Picture Resolution: 2592 x 1944
Video: Supports 1080p @ 30fps, 720p @ 60fps
and 640x480p 60/90 Recording
Megapixel: 5
Sensor best resolution: 1080p
Picture resolution: 2592 × 1944
Weight: 3gDimension: 25mm x 20mm x 9 (Approx.)
 NODE MCU – ESP8266
DESCRIPTION :
NodeMCU is an open source firmware for which open source prototyping board designs are
available. The name "NodeMCU" combines “NODE" and "MCU" (micro-controller unit)

SPECIFICATION :
Microcontroller: Tensilica 32-bit RISC CPU Xtensa LX106
Operating Voltage: 3.3VInput
Voltage: 7-12V
Digital I/O Pins (DIO): 16
Analog Input Pins (ADC): 1
UARTs: 1
SPIs: 1
I2Cs: 1
Flash Memory: 4 ; MBSRAM: 64 KB
LEAD ACID BATTERY
DESCRIPTION :
Lead acid rechargeable batteries are the oldest type of
rechargeable Batteries . Despite having a very low energy-to-weight
ratio and a low energy-to-volume ratio, they are able to supply high
surge currents, meaning that they are able to meet the power
requirements of applications such as storage in back-up power supplies
in mobile phone towers.

SPECIFICATION :


Weight Approx. 310g (0.68lbs)

Internal Resistance (@1kHz) Approx. 41mΩ

Maximum Discharge Current

For 5 seconds 18A

Case Material ABS
 MOTOR DRIVER – L298N
DESCRIPTION :
L298N Motor Driver Module is a high power motor driver module
for driving DC and Stepper Motors. This module consists of an L298 motor driver
IC and a 78M05 5V regulator.

SPECIFICATION :

Driver Chip: Double H Bridge L298NMotor

Supply Voltage (Maximum): 46V
Motor Supply Current (Maximum): 2A
Logic Voltage: 5V
Driver Voltage: 5-35V
Driver Current:2A
Logical Current:0-36mA
Maximum Power (W): 25W
 DC MOTOR
DESCRIPTION :
The motor is ideal for DIY enthusiasts. This motor set is
Inexpensive , small, easy to install, and ideally suited for use in a mobile robot car.
They are commonly used in our 2WD platforms.

SPECIFICATION :

Operating Voltage: 4.5V to 9V

Recommended/Rated Voltage: 6V
Current at No load: 70mA (max)
No-load Speed: 9000 rpm
Loaded current: 250mA (approx)
Rated Load: 10g*cm
Motor Size: 27.5mm x 20mm x 15mm
Weight: 17 grams
 SERVO MOTOR
DESCRIPTION :
A servo motor is a type of motor that can rotate with great
precision. Normally this type of motor consists of a control circuit that
provides feedback on the current position of the motor shaft, this
feedback allows the servo motors to rotate with great precision.

SPECIFICATION :

Operating Voltage is +5V typically

Torque: 2.5kg/cm
Operating speed is 0.1s/60°
Gear Type: Plastic Rotation : 0°-180°
Weight of motor : 9gm
Package includes gear horns and screws
VIRTUAL REALITY BOX
DESCRIPTION :
VR BOX 3D Glasses bring you wonderful experience of
watching movies and playing games.3D Virtual Reality (VR) Headset that
works with your smartphone. It allows you to view in true 3D and explore the
Virtual 3D world with head tracking

SPECIFICATION :

Strap Material: Nylon (flexible)

Lens: Augmented HD Optical Resin Lens
Diameter: 42mm
FOV: 70-80 degrees
Focal Length: 70-75mm
Viewing Angle: 115 degrees
Dimensions: 200mm x 100mm x 140mm
 PROJECT BUDGET
S.NO COMPONENT RATE
1. Raspberry Pi 4 model B 3899
2. Raspberry Pi camera module-5mp 366
3. NodeMCU 345
4. Motor driver - L293D 249
5. Lead acid battery for motors - 6v, 1.2 A 1493
6. Dc motors -2 179
7. Servo motors -2 279
8. VR box 399

Approximately the total is cost around – 7200

 CONCLUSION
• These robots can change dynamics of countless domains: they can provide homecare assistance to
the elderly and even facilitate virtual attendance for physically challenged students.
• In businesses, these robots can significantly reduce the need for traveling for meetings.
• Thus, the virtual telepresence robot is a simple, cost-effective and efficient solution to multiple
real world problems.

FUTURE WORK
• Now the robot can look around to different places successfully . But it lacks in something well that
is sound . We thought of adding a microphone and a speaker , to hear the atmosphere and to
respond or talk to people who is near to the robot .
• We also thought of adding a robotic arm on the top of it . To pick or drop the object which user
watch through virtual reality . That robotic arm is controlled with the MPU6050 gyroscope and
accelerometer sensor which is place on the user hand. Robotic arm acts according to the users
hand gestures .
THANK YOU ..!