Centre for Product Design and Manufacturing

IISc Bangalore

Project Title : Drawing Robot

Difficulty : Intermediate
Prerequisites : MATLAB Basics and Coordinate Systems.
Materials/Equipment needed to carry out the experiment: Arduino Engineering Kit.

Introduction
This project covers drawing robot's assembly, electronic hardware integration and MATLAB
programming.

Usability

This robot is capable of drawing 2D text / figures in the XY plane with provision of choosing
between red and black markers.

Learnings

MATLAB Programming
N20 Polulu DC Motor control
SG-90 Servo Motor Duty-Cycle Control
Understanding Coordinate Control
Basic Image Processing

Components Used

I. N20 Pololu DC motor with Encoder

II. SG-90 Tower Motor
III. MKR 1000
IV. MKR Motor Carrier
V. Li-Po Battery
VI. Robot Chassis
VII. Miscellaneous : Tools, Jumper wire and others.
Assembled Model & Its dimensions
 EXERCISES

EXERCISE 1 : Connecting To The Robot

EXERCISE 2 : MATLAB Function

EXERCISE 3 : Move to Specified Point

EXERCISE 4 : Identify Position Limit

EXERCISE 5 : Draw preprocessed Image

EXERCISE 6 : Draw any image

EXERCISE 7 : Drawing Live Image

 EXERCISE 1

Connecting To The Robot

For establishing connection to the robot with the system by running Arduino setup. Follow
the below steps for setting up connection :

STEP 1] >> arduinosetup

STEP 2] On the following screen, include the Arduino/MKRMotorCarrier library as well as

I2C, SPI, and Servo. From the Port dropdown list, choose the port that the Arduino is
connected to (COM## for Windows, /dev/ttyACM## for Linux, /dev/tty.usbmodem## for
OSX). Click Program to upload the program to the board.
Note : This may take several minutes. If an error message received related to source not
being found for the Arduino/MKRMotorCarrier library, then type edit
ArduinoKitHardwareSupportReadMe.txt in MATLAB Command Window and follow the
instructions provided on this text file.

Once the upload is complete, test the connection on the next screen using the Test Connection
button. Close the setup interface.

STEP 3] CONNECT TO THE HARDWARE

>> edit Task1

Then, run the code in to connect to the hardware section.

STEP 4 : CONTROL OF SERVOS

The drawing robot has two different color markers that can be raised and lowered by means
of a servo motor. In this section, software limitations for drawing robots given the physical
limitation will be estimated.
The Task1.mlx live script contains code to connect to the servo motor and command it to
move. First, connect to the motor by running the sections under the heading. Control the
servo in the live script Task1.mlx.
Using numeric slider control, vary the pos variable and observe the change. Note the
required servo positional values for LeftMarker, RightMarker and NoMarker Position.

Note : Run this section of code to save these values to a MAT-file so you can load them for
use in future exercises.
STEP 5 : CONTROL THE DC MOTORS

The next step is to figure out how to control the two DC motors that make the drawing robot
move over the whiteboard.

First, identify and read the sections of Task1.mlx under the heading Control the DC motors.
In this code, Vset is a target voltage to supply to the motor, and Vmax is the battery voltage.
These parameters are used to compute a Speed value for the motors between 0 and 1,
where 1 is maximum speed and 0 supplies no voltage to the motor. Run each of the sections
of the live script under the heading Control the DC motors.
STEP 6] READ THE ENCODERS

DC motors require external control mechanisms to ensure that they are behaving as
expected. After displacing the motors from their initial positions, the count value from the
encoders to determine how far each motor has turned in units of encoder counts. The
conversion from counts to physical distances will be calculated in a later exercise. Run the
code in the Read the encoders section of the live script Task1.mlx.

Note :

Create a new MATLAB script. Write a series of commands that will run the left motor for
three seconds, then stop and run the right motor for three seconds, then run the left motor in
reverse for three seconds, then run the right motor in reverse for three seconds, and finally
stop the motors. Run your script and check that it performs as expected. Read the counts
from each of the encoders. Reset the encoder counts and run the script again to see how
consistent this result is.

Reset the encoder counts and run the script again to see how consistent this result is. Notice
the change in behavior of drawing robots.
 EXERCISE 2

STEP 1] DEFINE DISTANCES ON THE WHITEBOARD

Imagine hanging the robot up on the whiteboard. Define the values y, Base, Z1, and Z2 as
shown in the following diagram.

The black circles at both sides of the diagram represent the pulleys hanging from the robot,
while the robot is in the center of the image, hanging from two strings connected to the robot
arms, where the motors are located. Note how the strings go from the robot arm to the pulley
and come back to the motor. Also observe that x and y positions on the whiteboard are
measured with respect to the left-most pulley, with x being the horizontal distance to the right
of the pulley and y being the vertical distance down from the pulley.
STEP 2] APPLY THE PYTHAGOREAN THEOREM TO COMPUTE POSITION

The sides of these triangles have the following relationships.

Z1^2 = x^2 + y^2

Z2^2 = (Base -x)^2 + y^2

the first equation can be rewritten as

Z1^2 = x^2 + [Z2^2 - (Base - x)^2]

Expanding this equation and solving for x, we end up with the following equations to
solve for x and y sequentially.

x = (Base^2 + Z1^2 - Z2^2)/2*Base

y = (Z1^2 - x^2)^0.5
STEP 3] UNDERSTAND INPUT DIALOGS

Dialog boxes are simple apps that can report messages, gather input, or allow the user to
interact with the file system. See the MATLAB documentation on Dialog Boxes for a list of
dialog boxes available in MATLAB. To request input from a user, use an input dialog.

Open the documentation page directly from the MATLAB command prompt by typing the
following:

>> doc inputdlg

Read the Syntax and Description sections of this documentation page to understand all the
allowed input arguments for this function.

Highlight the code from Example 1, then right-click and select Evaluate Selection.
When the dialog launches, enter new values for matrix size and colormap name. Click OK.

Return to the MATLAB Command Window. Also, note there is a new answer variable in the
Workspace. Check its value.
STEP 4] FIND A STARTING POSITION

Open the live script Task2.mlx by typing in the Command Window:

>> edit task2

Then follow the instructions of the first four sections of Task2.mlx under the heading Find
starting position, entering these values where appropriate. Please note that all
measurements should be written in meters.
STEP 5] CONNECT TO THE HARDWARE AND DRAW ON THE WHITEBOARD

MATLAB apps typically have a graphical user interface and code to perform a specific task.
The drawing robot project provides an app called SimplePlotterApp that can be used to
interactively control the robot's motors. Run the code in the Connect to the hardware and
Draw on the whiteboard sections in the live script Task2.mlx.
STEP 6] COMPUTE THE NEW ROBOT POSITION

For calculating the new robot positions, follow the below steps

The amount of string spooled or unspooled by the motors is related to the angle it has
rotated through and the radius of the spool.

△StringLength = Rspool.θ

On the robot, the string loops over the pulley and back to the robot body. Because the
string is doubled, the resulting change in distance from the pulleys (Z1 or Z2) is equal
to half the change in total string length.

△ Z = △ StringLength / 2

Z = △ Z + Zi

The new robot position is fully defined by the Base, Z1, and Z2 lengths.

x = (Base^2 + Z1^2 - Z2^2) / 2*Base

y = (Z1^2 - x^2)^0.5

To perform these calculations in code, run the remaining sections of Task2.mlx under
the heading Calculate new robot position.
STEP 7] ENCAPSULATE CODE IN FUNCTIONS

First write a MATLAB function for requesting the initial string length measurements from the
user. In the MATLAB Desktop, select New>Script . This will create an empty MATLAB file.
Enter the following code and save the file as initialPosition.m.

function Z_i = initialPosition()

str = inputdlg({'L1 (meters)','L2 (meters)'},'Enter initial string lengths.',[1 50]);

L1_i = str2double(str{1}); %meters

L2_i = str2double(str{2}); %meters

L_arm = 0.075; %meters Z_i = [L1_i L2_i] + L_arm; %meters

Initialize Z_i with following command

>> Z_i = initialPosition

Run the following line at the MATLAB Command prompt to create the counts variable.

>> counts = [counts1 counts2]

Create a new function called countsToXY.m and include the following code

funcfunction xy = countsToXY(counts,Z_i,Base) % Define constants

countsPerdRadian = countsPerRevolution/(2\*pi);

r_spool = 0.0045; % Convert counts to angle

phi = counts/countsPerRadian; % Convert angle to change in string length

dStringLength = r_spool\*phi; % Convert change in string length to change in Z

dZ = dStringLength/2; %Add change in Z to initial Z to get current Z

z = z_i + dz; % Compute x and y from Z1 and Z2

x = (Base^2 + z(1)^2 - z(2)^2)/(2\*Base);

y= sqrt(z(1)^2-x^2); xy = [x y];

end

Run the following at the command prompt, and check that the x-y position is correct

>> xy = countsToXY(counts,Z_i,Base)
 EXERCISE 3

STEP 1] CHOOSE A POSITION TO MOVE TO

Open the live script Task3.mlx

>> edit Task3

In the first section, called Choose position to move to in x-y, replace the values of x and y.
Make sure that all measurements of the coordinates are in meters. Run this section.
STEP 2] CONVERT TARGET POSITION TO ENCODER COUNTS

Compute the encoder counts for that position. To do this, use the same equations from the
previous exercise, but reverse the order. Starting with the x-y coordinate, the Pythagorean
Theorem can be used to calculate the Z1 and Z2 lengths.

Z1 = (x^2 + y^2)^0.5

Z2 = ((Base-x)^2 + y^2)^0.5

The change in Z can be computed and converted to a change in string length

△ Z = Z - Zi

△ StringLength = 2 * △ Z

Finally, use the motor encoder counts per m constant i.e. 498000 for calculating required
count

Cr (Required Counts) = △ StringLength * 498000

To perform these calculations in code, Task3.mlx

STEP 3] WRITE A FUNCTION FOR CONVERTING POSITION TO MOTOR
COUNTS

% Get Target Coordinates

dx = x2-x1;
dy = y2-y1;
d = [dx dy];
disp('Delta Coordinates acquired !')

%Get String Length

StringL1 = abs(hypot(x1, y1) - hypot(x2, y2));

StringL2 = (hypot(Base-x2, y2)- hypot(Base-x1, y1));
%disp('String Length acquired !');
String = [StringL1 StringL2];
%disp(String);

if x2>x1
t=-1;
else
t=1;
end

resetCount(eL);
resetCount(eR);
n = readCount(eL);
m = readCount(eR);

%Estimating required counts

Cl = t*StringL1*493000;
Cr = StringL2*493000;
Counts = [Cl Cr];

STEP 4] MOVE ROBOT TO TARGET POSITION

Run the function moveToCoordinate1 to move the robot to target position, more
details about the function is provided in upcoming sections.
STEP 5] MOVE TO A SEQUENCE OF POSITIONS

The functions xyToCounts and moveToCounts are also able to take lists of points. If you
define your target x-y points as an Nx2 array where the first column is the x -coordinates and
the second column holds the y -coordinates, you can direct the robot to move to a series of
points. The Move to a sequence of positions section of Task3.mlx has an example of
coordinates that form a rectangle. Check whether these points fall in the drawable area of
your whiteboard and change them if they do not. Notice that this code also includes a
sequence of actions required for drawing shapes. It raises the marker, moves to the first
point, lowers the marker, moves through all the points, and raises the marker again.
 EXERCISE 4

STEP 1] UNDERSTAND MOTOR EQUATIONS AND STALL CONDITIONS

For a DC motor, a mathematical equation can be used to describe the relationship between
motor load (torque), supply voltage, and rotational speed. This is sometimes referred to as
the DC motor torque equation:

τ = (V - ω * k) * k / R

In this equation, is the motor torque or motor load, V is the supply voltage, and is the angular
speed. (Note that in some versions of this equation, the variable T is used for torque. R is
the resistance in the motor windings, and k is the motor constant. Both R and k are constant
for a given motor.

τ = ωfree * k

The second special case is when the motor reaches stall conditions. This is when the load is
so high that the motor cannot spin. In this case, = 0, and the equation simplifies to the
following:

τ = Vk / R

STEP 2] CALCULATE MAXIMUM ALLOWABLE LOAD

First, check out the specifications provided for the motor. They are available here or here.
The motor is rated at 12 V. Record the specified voltage, free-run speed, and stall torque in
code. Convert them to SI units. Symbolic Math Toolbox provides a useful function called
unitConversionFactor. The code for these steps is contained in the live script Task4.mlx.
Open this file. Then run the section of code under the heading Define motor specifications.
First use the motor equation in free-run conditions where τ is 0 to calculate k. Then use the
motor equation in stall conditions where ω is 0 to calculate R. Run the section of code under
the heading Calculate motor constants from specs in Task4.mlx.

It's good practice not to run the motor at more than 30% of the stall torque , more
information here or here. Use the motor equation under stall conditions to calculate the stall
torque given your battery, then choose 30% of this value as the maximum allowed torque on
the robot's motors. To compute this value, run the section of code in Task4.mlx under the
heading Calculate torque limit for available voltage.
STEP 3] UNDERSTAND THE FREE-BODY DIAGRAM

In physics and engineering, it's often useful to be able to calculate the forces on an object.
For the whiteboard robot, these forces can tell how much load is on each of the motors. This
is done by creating what is known as a free-body diagram, which is a diagram showing a
body with forces acting on it in different directions. On the robot, there are two strings pulling
toward the left pulley. Finally, there is a weight pulling downward.
On a free-body diagram, it is also useful to break the forces down into their x and y
components. First, define θ1 as the angle of F1 from vertical and θ2 as the angle of F2 from
vertical, as shown in the following diagram. The relationship between F1,F2, and Weight will
be explained in the subsequent sections; for now, let's focus on how to express the values of
those forces in terms of the angles of the strings, which depend on the robot's location on
the whiteboard.

Then use trigonometry to break each force F1 and F2 down into its x-component and
y-component, as shown.

F1,x = -F1 * sin θ1 F1,y = F1 * cos θ1

F2,y = F2 * sin θ2 F2,y = F2 * cos θ2

It is fairly easy to answer whether the motors will suffer the same forces at different parts of
the whiteboard, at least under steady-state. The next section will cover this case.
STEP 4 ] UNDERSTAND FORCE BALANCE ON A STATIC BODY

Newton's third law of motion tells how an object behaves with forces acting on it. The sum of
the forces acting on the object is equal to the mass of the object times the acceleration of the
object.

ΣF=m*a

This equation can be broken into its component dimensions, in this case x and y.

Σ Fx = m * ax

Σ Fy = m * ay

When the robot is not moving, the acceleration in every direction is zero. This means the left
side of each equation is the sum of the forces in that direction, and the right side of the
equation is 0. For the whiteboard robot, this gives the following two equations.

-F1 * sinθ1 + F2 * sinθ2 = 0

F1 * cosθ1 + F2 * cosθ2 -Weight = 0

Solving this system of equations algebraically for the unknown forces F1 and F2 gives the
following:

F1 = Weight * sinθ2 / (sinθ1 * cosθ2 + sinθ2 * cosθ1)

F1 = Weight * sinθ2 / (sinθ1 * cosθ2 + sinθ2 * cosθ1)

However, the motor's equations are expressed in terms of torque, not force. This will allow
determining how much load the motors are under at every point on the whiteboard.
STEP 5] DERIVE EQUATIONS TO COMPUTE TORQUE FROM X-Y POSITION

For any x, y position on the whiteboard, it is possible to compute the torque on the motor.
First, use the x, y position to compute the hypotenuses of the triangles formed by the robot's
hanging position,Z1,Z2 . Use these values to compute the angles of the strings,θ1, θ2. Then
compute F1, F2 and T1, T2. Finally, use T1, T2 to compute τ1, τ2. Here is the set of
equations to use. To compute Z1, Z2 from x-y, simply use the Pythagorean Theorem as
have done previously.

Z1 = (x^2 + y^2)^0.5

Z2 = ((Base - x)^2 + y^2)^0.5

To compute θ1, θ2 from x, y, Z1, and Z2, use the law of sines. This law states the following
for any triangle:

a / sinA = b / sinB = c / sinC

In this case, we can use the relationship between the right angle of each triangle and the
unknown angle θ of each triangle. This gives us the following equations:

x / sinθ1 = Z1 / sin 90

(Base - x) / sin θ2 = Z2 / sin 90

Since sin 90 is equal to 1, these equations simplify to:

Θ1 = sin^-1 (x/Z1)

Θ2 = sin^-1 ((Base - x)/Z2)

To compute F1, F2 from θ1, θ2, use the equations derived in the previous section:

F1 = Weight * sinθ2 / (sinθ1 * cosθ2 + sinθ2 * cosθ1)

F2 = Weight * sinθ2 / (sinθ1 * cosθ2 + sinθ2 * cosθ1)

To compute T1, T2 from F1, F2, recall that the total force pulling the robot in the direction of
a pulley is equal to twice the tension on the string over that pulley, because there are two
points where the string pulls on the robot.

T1 = F1 / 2

T2 = F2 / 2

Finally, compute torque from the tension on the string. The string pulls the motor in a
direction perpendicular to the line from the center of the spool to the point where the string
touches the motor, as shown in the following diagram.

In this situation, the magnitude of the torque is given as the product of the force and the
distance between the axis of rotation and the point where the force is applied. In this case,
that distance is the radius of the spool, so the following equations:

τ1 = T1 * Rspool
τ2 = T2 * Rspool

Given all the equations introduced in this section, it is possible to compute angles in terms of
location, forces in terms of angles, tension in terms of forces, and torque in terms of tension,
which means there is a way to relate torque to location on the whiteboard.
STEP 6] CREATE A GRID OF ALL POSSIBLE BOARD POSITIONS

Using a meter stick or tape measure, measure the height of the whiteboard, from the pulleys
down to the bottom of the board. Find the section of code in Task4.mlx under the heading
Define whiteboard dimensions. On the first line of code, change the value of H_board so that
it is equal to your measured height, defined in meters.

A data grid of coordinate positions can be created in MATLAB using the meshgrid function.
An easy way to call this function is by defining arrays of the x -values and y -values to be
contained in the grid. The next section of code defines these arrays over the whole range of
x and y values at an interval of 1 mm. Run the section of code under the heading Create a
grid of all possible board positions in Task4.mlx.

Open the variables Xand Y in the Variable Editor by double-clicking them in the Workspace.
Note that they are the same size, the columns of X are all the same, and the columns of Y
are all the same. Therefore, each pair of positions X(i,j) , Y(i,j) defines a unique location on
the whiteboard.
STEP 7] COMPUTE TORQUE AT EVERY POSITION AND PLOT SURFACE

To calculate the tension on each string, the robot's weight is required. To convert the tension
to torque, the radius of the spool is required. These are both defined for the given robot kit.
To define these in the MATLAB code, run the section of code under the heading Define robot
constants for calculating torque.

Next, use MATLAB to apply the derived equations for converting x-y position to torque. First,
compute Z1 and Z2 for every position on the whiteboard. Then apply the trigonometric
equations to calculate the angles θ1 and θ2 based on the formed triangles. Use the
force-balance equations to calculate the force in each direction and the tension on each
string. Then compute the torque load on each motor at every position. Finally, define a single
variable Tau that contains the larger of the two torque values at every position. You must
never move the robot to a location where this value is greater than TauMax. Run the section
of code under the heading Compute torque at every position in Task4.mlx.
Now that the motor load Tau is defined at every position on the whiteboard, you can visualize
it in MATLAB. Create a simple surface plot using the surf function in MATLAB. To do so, run
the section of code in Task4.mlx under the heading Plot torque at every position.

Explore this plot. Click the icon marked Rotate. Then click and drag the plot to examine the
surface from all sides and get a better sense of what the torque is like at different positions
on the whiteboard.
STEP 8] ELIMINATE BAD REGIONS AND PLOT IMAGE

Now make a plot that removes the regions the robot should not move to. This includes two
types of regions. First, remove the regions where the torque is too high. Then remove the
regions where the robot cannot go because it is too close to the pulleys or bottom of the
whiteboard.

To "remove" these regions from our plot, a transparency mask for them is defined. The plot
will be visible at regions the robot can reach and transparent at regions the robot cannot
reach. In MATLAB, the data that defines these regions is called AlphaData. To create the
opacity variable and remove the unreachable regions, run the section of code under the
heading Eliminate bad regions.
STEP 9] DEFINE AND SAVE DRAWING LIMITS FOR YOUR WHITEBOARD

Next, define the part of the whiteboard that will allow the robot to draw on. These are likely to
fall somewhere along the curve that defines how close the robot can get to the left pulley.
Use the Data Cursor tool in the live script to select a point and check its x - y coordinates.

Record the x and y values from the plot in the code. Find the next section of Task4.mlx
under the heading Choose drawable region limits and visualize. Replace the pickedX and
pickedY variable values with the values selected in the plot.

Use the rectangle function to draw the rectangle. To do this and verify that the drawable
region looks correct, run the code under the heading Choose drawable region limits and
visualize in Task4.mlx.

Run the section of code under the heading Save chosen drawing limits in Task4.mlx so that
you can use those values in future exercises.
 EXERCISE 5

STEP 1] UNDERSTANDING HOW TO READ IMAGES IN MATLAB

MATLAB can work with images in common image file formats. The general workflow is first
to read the image into a MATLAB variable. To read an image into MATLAB, use the imread
function. An image that ships with MATLAB is in the file peppers.png. Read this by entering
the following at the MATLAB command line.

>> RGB = imread('peppers.png');

Examine the RGB variable in your Workspace. Note that the variable is three-dimensional
and of type uint8. Each one of the components (R, G, B) representing one of the basic colors
gets 8 bits to store its value in digital form. This means that you can work with images of
24-bit color depth (8 bits or 1 byte for each one of them).

View the image using the imshow function. Execute the following statement at the command
prompt.

>> imshow(RGB)
The three components of the variable represent the red (R), green (G), and blue (B)
information of the image respectively. . To see a representation of the red plane data, use
the following command to view the image regions that have the most red:

>> imshow(RGB(:,:,1))

Run the following command to do this with the peppers image. The result is the same as
when we displayed the full color matrix.

>> imshow('peppers.png')
STEP 2] PLOT AN IMAGE OF A DRAWING TO REPLICATE

The robot is not able to draw full RGB planes, and not even grayscale images. The current
design needs vector graphics as an input. The Images folder contains both the original
images and their corresponding MAT-files containing the path to draw. Find the file
MathWorksLogo.jpg.

Right click it in the Current Folder and select Open Outside MATLAB to see what it looks like
in the system's image viewer. Note that different operating systems have different image
viewers.

Next step is checking whether the processed file looks similar to the original image. For that
open the live script Task5.mlx :

>> edit Task5

STEP 3] LOAD AND PLOT COORDINATES FOR AN EXISTING IMAGE

The coordinates of the points to plot are stored in a cell array. Run the section Load
pathways for sample images in Task5.mlx to bring the data into MATLAB.

Double-click the segmentsPix variable in Workspace to open it in the Variable Editor. Notice
that it is a cell array with three elements, each of which contains a two-column array of
doubles with a different number of rows each.

In other words, there are three different line traces in this image file. Double-click one of
these elements to see the data contained in that cell. These values represent the pixel
coordinates in the image for the selected line trace.
The nth element of the segmentsPix cell array variable can be accessed using curly braces,
as in the following command.

>> data = segmentsPix{n};

Find the section in Task5.mlx under the heading Plot pixels chosen from sample image. Note
that this section of code contains a for loop with loop index ii . In each iteration of the loop,
the ith element of segmentsPix is extracted and plotted. Run this section to see the
pathways stored in this variable.
STEP 4] UNDERSTAND SCALING TO PHYSICAL UNITS

To decide where to draw an image on the whiteboard, pixels to metric units (meters) are
needed to be converted. The whiteboard dimensions will determine the maximum and
minimum whiteboard positions in meters. . There will be two limits in the x axis, and two in
the y axis (a maximum and minimum position for moving on each one of the axes).

Define the xRangePix and yRangePix as the difference between these maximum and
minimum values as shown in the following diagram.

That scaling factor will be one of the following two quantities:

xScaleFactor = xRange / xRangePix

yScaleFact = yRange / yRangePix

The choice of which scaling factor to use depends on which one is smaller. This will be
related to the aspect ratio of the whiteboard drawing area versus the image to draw.
Consider a relatively square drawing area and a short, wide image.
STEP 5] UNDERSTAND THE TRANSFORMPIXELSTOMETERS FUNCTION, LINE
BY LINE

The function also allows the user to specify what percent of the available space should be
used to draw the image. This is specified by the fraction input argument.

This function computes the scale factor for converting pixels to meters, then converts
all the segments in segmentsPix to meters.

function segmentsMeters =
transformPixelsToMeters(segmentsPix,xLim,yLim,xLimPix,yLimPix,fraction)

%First, create variables to represent the ranges xRange, yRange, xRangePix, and
yRangePix.
xMinM = xLim(1);
yMinM = yLim(1);
xRangeM = diff(xLim);
yRangeM = diff(yLim);

% Determine the range of the coordinates to draw

xMinPix = xLimPix(1);
yMinPix = yLimPix(1);
xRangePix = diff(xLimPix);
yRangePix = diff(yLimPix);

% Scale from pixels to real world units (meters)

xScaleFactor = fraction*xRangeM/xRangePix;
yScaleFactor = fraction*yRangeM/xRangePix;
Choose the smaller of the two scale factors.

% Pick the smaller scale factor. If both are NaN, pick

pix2M = min(xScaleFactor,yScaleFactor);
if isnan(pix2M)
pix2M = 0;
end

% Identify origin position of scaled drawing

centerMeters = [xMinM yMinM] + [xRangeM yRangeM]/2;
drawingOriginM = centerMeters - pix2M*[xRangePix yRangePix]/2;
segmentsMeters = cell(size(segmentsPix));
nSegments = length(segmentsPix);
for ii = 1:nSegments
% Scale all segments by the computed scaling factor
coordsPix = segmentsPix{ii};
coordsPix = fliplr(coordsPix); %Convert from row,col to x,y
coordsMeters = pix2M*(coordsPix-[xMinPix yMinPix]) + drawingOriginM;
segmentsMeters{ii} = coordsMeters; end;
STEP 6] CONVERT PIXELS TO PHYSICAL DISTANCES AND PLOT

TransformPixelsToMeters function to use on the data you loaded from the sample image.
You'll use the whiteboard x-limits and y-limits that you determined in the previous exercise.
Then, for each segment, you'll convert the pixels from that segment to meters and store the
new segment data in a cell array variable called segmentsMeters. To do this, run the
sections of code under the heading Convert pixel coordinates to physical distances.

The resulting plot should look the same but use units of meters and a range within the limits
of the whiteboard. Run the code in the Plot paths in meters section of Task5.mlx.
STEP 7] UNDERSTAND THE REDUCE SEGMENT FUNCTION, LINE BY LINE

The reduceSegment function reduces the number of points in a segment by removing points
within a specified radius. In other words, it filters out points, which reduces the amount of
subsegments to draw.

This function computes the scale factor for converting pixels to meters, then converts
all the segments in segmentsPix to meters.

function segmentsMeters =
transformPixelsToMeters(segmentsPix,xLim,yLim,xLimPix,yLimPix,fraction)

xMinM = xLim(1);
yMinM = yLim(1);
xRangeM = diff(xLim);
yRangeM = diff(yLim);

% Determine the range of the coordinates to draw

xMinPix = xLimPix(1);
yMinPix = yLimPix(1);
xRangePix = diff(xLimPix);
yRangePix = diff(yLimPix);

Calculate the two possible scale factors. The fraction factor indicates what
percentage of the available space should be used.

% Scale from pixels to real world units (meters)

xScaleFactor = fraction*xRangeM/xRangePix;
yScaleFactor = fraction*yRangeM/xRangePix;

Choose the smaller of the two scale factors.

pix2M = min(xScaleFactor,yScaleFactor);
if isnan(pix2M)
pix2M = 0;
end

% Identify origin position of scaled drawing

centerMeters = [xMinM yMinM] + [xRangeM yRangeM]/2;
drawingOriginM = centerMeters - pix2M*[xRangePix yRangePix]/2;
segmentsMeters = cell(size(segmentsPix));
nSegments = length(segmentsPix);
for ii = 1:nSegments
% Scale all segments by the computed scaling factor
coordsPix = segmentsPix{ii};
coordsPix = fliplr(coordsPix); %Convert from row,col to x,y
coordsMeters = pix2M*(coordsPix-[xMinPix yMinPix]) + drawingOriginM;
segmentsMeters{ii} = coordsMeters;
end
STEP 8] REDUCE SEGMENT SIZES AND PLOT

Let’s execute the reduceSegment function in a for loop to reduce the number of points in
each segment of this image. Run the section of code under the heading Reduce size of
each segment. Then run the section of code under the heading Plot reduced path in
meters to visualize the updated paths.

STEP 9] CONVERT DESIRED POSITIONS TO ENCODER COUNTS

The xyToCounts function converts the target positions in meters to encoder counts. Then run
the section of code under the heading Identify initial position in Z and enter these lengths
in the dialog box.
Store the resulting encoder count targets in a cell array called segmentsCounts. Run the
code under the heading Convert distances to encoder counts in Task5.mlx.
STEP 10] DRAW LINE SEGMENTS ON WHITEBOARD

Run the code sections under the heading Connect to hardware in Task5.mlx.

Look at the code in the section Draw image on whiteboard. It uses a for loop to step through
the segmentsCounts variable. Think about what's happening in the loop. Each iteration of
the loop applies to one of the line traces in the image. It extracts the encoder counts for that
line trace, moves the robot to the first position, lowers the marker, then moves the robot to all
positions in that line trace, and finally raises the marker. This is repeated for each of the
three traces in the image. Run this section of code to draw the image on the whiteboard.
STEP 11] WRITE A FUNCTION FOR DRAWING AN IMAGE FROM PIXELS

The function will take pixel data and whiteboard starting position as inputs and then draw the
image. Include the following code in the function and save it.

function drawImageFromPix(segmentsPix,xLimPix,yLimPix,Z_i)
nSegments = length(segmentsPix)

% Define whiteboard limits

load WhiteboardLimits.mat xLim yLim

% Convert pixel coordinates to physical distances and then to encoder counts

fraction = 0.7;
segmentsMeters =
transformPixelsToMeters(segmentsPix,xLim,yLim,xLimPix,yLimPix,fraction);

% Reduce size of each segment

radius = 0.002; %Max distance between points to draw (meters)
for ii = 1:nSegments
segmentsMeters{ii} = reduceSegment(segmentsMeters{ii},radius);
End

load RobotGeometry.mat Base

segmentsCounts = cell(size(segmentsMeters));
for ii = 1:nSegments
segmentsCounts{ii} = xyToCounts(segmentsMeters{ii},Z_i,Base);
end

% Define up and down positions for servo motor

load ServoPositions.mat LeftMarker NoMarker

% Draw image on whiteboard

writePosition(s,NoMarker)
for ii = 1:nSegments

% Get counts for current segment

countList = segmentsCounts{ii};

% ADD CODE MOVE TO COORDINATE

 EXERCISE 6

STEP 1] UNDERSTAND IMAGE CONVERSION, FILTERING, AND ANALYSIS

Image Processing Toolbox provides a large amount of functionality for image processing,
analysis, visualization, and algorithm development.

Here are some other forms an image can take in MATLAB. A color image can also be
represented as a 2-dimensional MxN array with a corresponding colormap. We won't be
working with this type of color image. A grayscale image can be represented simply as an
MxN array. This is useful for image processing functions that operate on a 2-D grid of data,
such as filtering. A binary image can be represented as an MxN array of logical (true or
false) values. This is useful for extracting characteristics of particular regions in an image.

Let's examine an image and convert it to grayscale. Run the following code at the MATLAB
command prompt to load and view the peppers image in MATLAB:

>> RGB = imread('peppers.png');

>> imshow(RGB);
Binary images are images where each pixel is either on or off. These are visualized in pure
black-and-white, which can be useful for describing specific regions or objects in an image.
A grayscale image can be converted to binary using the imbinarize function. This function
replaces all values above a certain threshold with 1 and those below that threshold with 0.
The threshold can be global or regional. Convert the peppers image to binary and view it.
Also, note the data type of the binary image.

>> BW = imbinarize(I);

>> imshow(BW)
Test this out on the peppers image to see what line traces represent the thinnest version of
the objects shown in this image.

>> BW2 = bwmorph(BW,'thin',inf);

>> imshow(BW2)

See the following MATLAB documentation pages for more information about these and other
image processing functions:

I. Image Types in the Toolbox

II. Image Type Conversion
III. Morphological Operations
IV. imbinarize
V. bwmorph

In the following sections, we'll use the same techniques to extract line traces from raster
images.
STEP 2] LOAD EXISTING IMAGE FROM FILE

Start by loading an image into MATLAB. We'll use a line drawing of the MathWorks logo.
Open Task6.mlx in MATLAB.

>> edit Task6

Run the first section of the live script under the heading Load image from file.

Start by loading an image into MATLAB. We'll use a line drawing of the MathWorks logo.
Open Task6.mlx in MATLAB.

>> edit Task6

Run the first section of the live script under the heading Load image from file.
STEP 3] EXTRACT LINE TRACES FROM IMAGE

Process the image so that it contains only the thin traces of lines. Use image processing
functions to convert the image to grayscale, convert it to binary black and white, remove
isolated pixels, and thin objects to lines. Do this by running the section of Task6.mlx under
the heading Extract line traces from image. The output of the live script shows what the
image looks like after each step.

STEP 4] UNDERSTANDING THE GETCOORDS FUNCTIONS, LINE BY LINE

To extract the traces from an image in the form of pairs of coordinate values that compose
curves.It is a recursive function that continues to call itself while there are still pixels
available on a binarized image and concatenates them into a single array.

This function calls bwboundaries recursively, stripping off the outer pixel boundaries from the
image each time and concatenating them.
function curvePoints = getCoords(shapeImage)

% Detect boundary points

[curves,~,N] = bwboundaries(shapeImage);

curves = curves(1:N); % Ignore hole boundaries

Get the pixels from the boundaries.

% Get the points from the boundary detected

curvePoints = cell2mat(curves);

% Remove all duplicate points from the curve

curvePoints = unique(curvePoints,'rows','stable');

% Remove curves from the image

curveInd = sub2ind(size(shapeImage),curvePoints(:,1), curvePoints(:,2));

shapeImage(curveInd) = 0;

% Call getCoords recursively if there are other curves remaining

if any(shapeImage(:))

curvePoints = [curvePoints; getCoords(shapeImage)];

end
BREAK COORDINATES LIST INTO CONTINUOUS SEGMENTS

Plot each group of pixels to see which ones are organized together. Run the section of the
live script Task6.mlx under the heading Break coordinates list into contiguous segments.

Notice how the red and blue traces share a common endpoint at the leftmost end.

UNDERSTANDING THE CONNECT SEGMENTS FUNCTION, LINE BY LINE

Given the results from the previous function, there might be segments with adjacent
endpoints that could be merged with one another. This could reduce the total number of
segments in the image.

The connectSegments function does two things. First, it closes any segments that
intersect themselves.Then it merges any segments whose endpoints are adjacent.

function segments = connectSegments(segments)

If a segment intersects itself, such as in the letter "O," it will end at a point that is adjacent to
another pixel in that segment. This will create a small gap when drawn on the whiteboard. To
correct this, add the final adjacent pixel to the end of that segment so the line is closed. This
section of code looks at each segment, finds any self-intersections, and closes them. p1 is
the first point in a segment. pN is the last point in the segment. Add any points near p1 to the
beginning of the segment. Add any points near pN to the end of the segment.

% Close any segments that self-intersect (like an "O" or a "P")

for ii = 1:length(segments)
points = segments{ii};
p1 = points(1,:);
pN = points(end,:);

% Add any points near p1 to beginning of segment ii

nearP1 = isadjacent(p1,points(4:end,:)); %Don't check first 3 points
if any(nearP1)
idx = find(nearP1,1)+3;
points = [points(idx,:); points]; %#ok<*AGROW>
End

% Add any points near pN to end of segment ii

nearPN = isadjacent(pN,points(1:end-3,:)); %Don't check last 3 points
if any(nearPN)
idx = find(nearPN,1);
points = [points; points(idx,:)];
end
segments{ii} = points;
End

This code merges any segments with endpoints that are adjacent. It looks at every pair of
segments ii and jj. Take the first and last point of each segment in the pair (four points total),
and check whether any are adjacent. If any adjacencies are found, replace the two
segments with a single segment describing a continuous path through both segments.
% Merge segments with adjacent endpoints
for ii = 1:length(segments)-1
jj = ii + 1;

% Check all combinations of 2 segments ii and jj

while jj <= length(segments)
points_i = segments{ii};
points_j = segments{jj};
pi1 = points_i(1,:);
piN = points_i(end,:);
pj1 = points_j(1,:);
pjN = points_j(end,:);

% Compare points 1 and N from segments ii and jj

if isadjacent(pi1,pj1)
segments{ii} = [flipud(points_j); points_i];
segments(jj) = [];
elseif isadjacent(pi1,pjN)
segments{ii} = [points_j; points_i];
segments(jj) = [];
elseif isadjacent(piN,pj1)
segments{ii} = [points_i; points_j];
segments(jj) = [];
elseif isadjacent(piN,pjN)
segments{ii} = [points_i; flipud(points_j)];
segments(jj) = [];
end
jj = jj + 1;
end
end

End
The sub function is adjacent and creates a convenient way to check whether two points are
adjacent. As a subfunction, it can only be called from inside the connectSegments function.

% Subfunction to define if points are adjacent

function tf = isadjacent(p1,p2)
tf = all(abs(p1-p2) <= [1 1],2);
end

MERGE CONNECTED SEGMENTS AND GET LIMITS

Now call connectSegments on the cell array of pixels groups created in the previous step.
This will return a new cell array of pixel groups. In this new cell array, some of the groups
may be merged if they can be connected to form one path. Each group of pixels in this cell
array is a path that will be drawn by the robot, which will lift the marker before drawing the
next path. Plot each group of pixels to visualize all the paths that the robot will traverse. Run
the section of the live script Task6.mlx under the heading Merge connected segments
.
Also create new variables xLimPix and yLimPix as a quick way to store the maximum and
minimum pixel positions for use when drawing the image on the whiteboard. Remember they
are needed when estimating how to scale the image to fit on the whiteboard. Run the section
of code with the heading Store x and y pixel limits .

WRITE A FUNCTION TO CONVERT IMAGE TO PIXEL SEGMENTS

The function will return the segmentsPix , xLimPix , and yLimPix variables that you have
computed. Create a new MATLAB function and call it imageToPixelSegments.m. Include the
following code in the function and save it.

function [segmentsPix,xLimPix,yLimPix] = imageToPixelSegments(img)

% Extract line traces from image
img2 = ~imbinarize(rgb2gray(img), 'adaptive' , 'ForegroundPolarity' , 'dark' );
img3 = bwmorph(img2, 'clean' );
img4 = bwmorph(img3, 'thin' ,inf);

% Extract pixels in order

coordsPix = getCoords(img4);
% Break coordinates list into contiguous segments
segmentsPix = coords2segments(coordsPix);

% Clean data and merge connected segments

segmentsPix = connectSegments(segmentsPix);

% Store x and y pixel limits

xLimPix = [min(coordsPix(:,2)) max(coordsPix(:,2))];
yLimPix = [min(coordsPix(:,1)) max(coordsPix(:,1))];

Now check that the code in the imageToPixelSegments function behaves in the same way
as the code you've executed so far. To do this, run the function and compare the result to the
segmentsPix , xLimPix and yLimPix variables in your workspace. Run the section Test
function imageToPixelSegments in Task6.mlx.
STEP 5 ] DRAWING IMAGE USING MOVETOCOORDINATE1 FUNCTION

In the previous exercise, functions for getting the processed coordinate of the required
image are already explained. This section will explain the workflow in moveToCoordinate1.m
function.

This flowchart illustrates the steps involved in this function. The steps are explained
below.

Step 1] Input imaged is preprocessed into rgb array with imread function

Step 2] The pixels are converted to metric form (meters)

Step 3] These dimensions are scaled in accordance to the whiteboard workable space
restrictions.

Step 4] This step converts the scaled metric data to coordinates.

Step 5] As an unfeasible number of coordinates are generated we need to reduce them

taking drawing robot's capabilities into consideration.

Step 6] In this step, the drawing loop processes the reduced coordinates with segment loop,
Marker position and MoveToCoordinate1.

Step 6.1] In the segment loop, depending on the number of segments this step will send
commands to Marker position and moveToCoordinate1 block.

Step 7] After Completing the drawing loop & coordinates drawing robot reaches its final
position.
Unless all the coordinates of each segment are reached the function will repeat itself. The
function code is given as follow :

function [Cl,Cr] = moveToCoordinate1(x1,y1,x2,y2,eL,eR,mL,mR)

% Get Target Coordinates

dx = x2-x1;
dy = y2-y1;
d = [dx dy];

%Get String Length

StringL1 = abs(hypot(x1, y1) - hypot(x2, y2));
StringL2 = (hypot(Base-x2, y2)- hypot(Base-x1, y1));
%disp('String Length acquired !');
String = [StringL1 StringL2];
%disp(String);

if x2>x1
t=-1;
else
t=1;
End

resetCount(eL);
resetCount(eR);
n = readCount(eL);
m = readCount(eR);

%Estimating required counts

Cl = t*StringL1*493000;
Cr = StringL2*493000;
Counts = [Cl Cr];

% Count satisfying loop

while true

if Cl>0 && Cr>0

mL.Speed=-1;

mR.Speed=1;
disp(mL.Speed);
if Cl-n>1000 && Cr-m > 1000
start(mL); start(mR);
pause(0.1);
stop(mL); stop(mR);
n = readCount(eL);
m = readCount(eR);

end
if Cl-n>1000 && Cr-m <= 1000
start(mL);
pause(0.1);
 stop(mL);
n = readCount(eL);
m = readCount(eR);
end
if Cl-n<=1000 && Cr-m > 1000
start(mR);
pause(0.1);
stop(mR);
n = readCount(eL);
m = readCount(eR);

end
if Cl-n<=1000 && Cr-m<=1000
%disp('Reached');
n = readCount(eL);
m = readCount(eR);

break;
end

end

if Cl<0 && Cr>0

mL.Speed=1;

mR.Speed=1;
disp(mL.Speed);
if Cl-n<-1000 && Cr-m > 1000
start(mL); start(mR);
pause(0.1);
stop(mL); stop(mR);
n = readCount(eL);
m = readCount(eR);

end
if Cl-n<-1000 && Cr-m <= 1000
start(mL);
pause(0.1);
stop(mL);
n = readCount(eL);
m = readCount(eR);

end
if Cl-n>=-1000 && Cr-m > 1000
start(mR);
pause(0.1);
stop(mR);
n = readCount(eL);
m = readCount(eR);

end
if Cl-n>=-1000 && Cr-m<=1000
%disp('Reached');
n = readCount(eL);
m = readCount(eR);
 break;
end

end

if Cl>0 && Cr<0

mL.Speed=-1;

mR.Speed=-1;
disp(mL.Speed);
if Cl-n>1000 && Cr-m<=-1000
start(mL); start(mR);
pause(0.1);
stop(mL); stop(mR);
n = readCount(eL);
m = readCount(eR);

end
if Cl-n>1000 && Cr-m>=-1000
start(mL);
pause(0.1);
stop(mL);
n = readCount(eL);
m = readCount(eR);

end
if Cl-n<=1000 && Cr-m<=-1000
start(mR);
pause(0.1);
stop(mR);
n = readCount(eL);
m = readCount(eR);

end
if Cl-n<=1000 && Cr-m>=-1000
%disp('Reached');
n = readCount(eL);
m = readCount(eR);

break;
end
end

if Cl<0 && Cr<0

mL.Speed=1;

mR.Speed=-1;
disp(mL.Speed);
if Cl-n<-1000 && Cr-m<-1000
start(mL); start(mR);
pause(0.1);
stop(mL); stop(mR);
n = readCount(eL);
m = readCount(eR);
 end
if Cl-n<-1000 && Cr-m>=-1000
start(mL);
pause(0.1);
stop(mL);
n = readCount(eL);
m = readCount(eR);

end
if Cl-n>=-1000 && Cr-m<-1000
start(mR);
pause(0.1);
stop(mR);
n = readCount(eL);
m = readCount(eR);

end
if Cl-n>=-1000 && Cr-m>=-1000
%disp('Reached');
n = readCount(eL);
m = readCount(eR);

break;
end
end

end

Correction code : In each script such as MATLAB logo drawing or custom rectangle
drawing, some point due to some limitation correction part is required in order to smoothen
or complete drawing properly.

ADDING PD CONTROLLER

With addition of a PD controller, the fluctuation in drawing is reduced to great extent thus enabling
smooth drawing. Here as Cl is required, the count for the left motor is already the difference of current
and required counts. Thus can be considered an error in our controller. We have added one extreme
condition loop which keeps constant value in between 0 - 1.

Where Kp = 150 and Kd = 10.

function [constant,previous_error,d] = PID(Cl,P,D,previous_error)

error = abs(Cl/493000);
delta = error - previous_error;
constant = P*error + D*delta;
d = constant
previous_error = error;
if constant > 1
constant = 0.9;
elseif constant < 0
constant = 0.3;
end
end
 EXERCISE 7

By now, drawing robots are pretty capable. It can take any image file and reproduce the
image on a whiteboard. All of this is automated in code. But there are still some manual
steps that involve taking the image, transferring the file to the computer, and specifying that
file required to draw.

STEP 1] CONNECT TO WEBCAM

In addition to Arduino devices, MATLAB can talk to many other types of hardware. To
acquire images directly into MATLAB a USB webcam is used. MATLAB's webcam interface
can connect to webcams, preview what they see, and take snapshots that bring the current
image directly into MATLAB. This functionality requires the MATLAB Support Package for
USB Webcams , which should have been installed when setting up all MATLAB and Simulink
software. Find the USB webcam that is included in the kit and connect it to the computer.
Open the live script Task7.mlx .

>> edit Task7

To identify the webcam in MATLAB and connect to it, run the section of code under the
heading Connect to webcam.
STEP 2] Preview the webcam image in MATLAB by running the section of code with
the heading Preview webcam image.

Using a black marker, create a simple line drawing on your whiteboard.Capture an image of
the drawing. Run the section of code in Task7.mlx with the heading Capture the current
image.
STEP 3] PROCESS AND VERIFY IMAGE

To make sure that image will draw correctly with the robot, run image processing function
imageToPixelSegments , and check that the extracted paths in the image are as expected.
Run the section of Task7.mlx under the heading Process and verify the image. If the
processed version does not look correct, take a new image and try again.
UPDATE MAIN SCRIPT FOR LIVE IMAGES

Delete the first line of code from main6.m, which loads the image from the file. Replace it
with code that will connect to the webcam, preview the image, and wait until a key before
taking a snapshot of the current image and continuing with the script. The full main7.m script
should look as follows:

% Capture image from webcam

w = webcam;

preview(w)

pause

img = snapshot(w);

clear w

% Convert image to pixel segments

[segmentsPix,xLimPix,yLimPix] = imageToPixelSegments(img);

% Identify initial position on whiteboard

Z_i = initialPosition();

% Draw image

drawImageFromPix(segmentsPix,xLimPix,yLimPix,Z_i)

RUN MAIN SCRIPT TO DRAW AN IMAGE CAPTURED LIVE

At the MATLAB command prompt, run the main script.

>> main7

Make sure nothing else is in the frame of the image and there is no glare or color variation
across the image. Strike any key in MATLAB to continue. The robot will scale the captured
image to the full whiteboard and attempt to reproduce it.