Automation Of Object Sorting System Using Pick & Place Robotic Arm & Image Processing

AUTOMATION OF OBJECT SORTING SYSTEM USING PICK &

PLACE ROBOTIC ARM & IMAGE PROCESSING
1
VISHNU R. KALE, 2V. A. KULKARNI

PG Student [EC], Dept. of E&TC, J.N.E.C., Aurangabad, Asso. Prof., Dept. of E&TC, J.N.E.C., Aurangabad
Email: [email protected], [email protected]

Abstract: The paper presents a smart approach for a real time inspection and selection of objects in continuous flow. Image
processing in today’s world grabs massive attentions as it leads to possibilities of broaden application in many fields of high
technology. The real challenge is how to improve existing sorting system in the modular processing system which consists of
four integrated stations of identification, processing, selection and sorting with a new image processing feature. Existing
sorting method uses a set of inductive, capacitive and optical sensors do differentiate object color. This paper presents a
mechatronics color sorting system solution with the application of image processing. Image processing procedure senses the
objects in an image captured in real-time by a webcam and then identifies color and information out of it. This information is
processed by image processing for pick-and-place mechanism.The Project deals with an automated material handling
system. It aims in classifying the colored objects by colour, size, which are coming on the conveyor by picking and placing
the objects in its respective pre-programmed place. Thereby eliminating the monotonous work done by human, achieving
accuracy and speed in the work. The project involve sensors that senses the object’s colour, size and sends the signal to the
microcontroller. The microcontroller sends signal to circuit which drives the various motors of the robotic arm to grip the
object and place it in the specified location. Based upon the detection, the robotic arm moves to the specified location,
releases the object and comes back to the original position.

Keywords: Camera, Conveyor belt system ,Image Processing, Micro-controller, Robotic System, Servomotor.

I. INTRODUCTION upon the colour detected, the robotic arm moves to

the specified location, releases the object and comes
Determining real time and highly accurate back to the original position [1][2].
characteristics of small objects in a fast flowing
stream would open new directions for industrial II. SYSTEM MODEL AND ASSUMPTIONS
sorting processes. The present paper relates to an
apparatus and method for classify in and sorting The fig. shows block diagram of a system. The basic
small-sized objects, using elect ronic systems and theme of this project is object flowing on conveyor
advanced sensors operating on the basis of a physical are sensed, selected and sorted depending on their
and geometric characterization of each element. colour and size. For this, camera is used as input
Recent advances in electronics and printed circuit sensor, camera is overhead camera which will be
board technology open new perspectives for mounted on PC, and will be connected to PC by USB.
industrial application in this field. The proposed The camera will take a snap and it will feed to PC for
selection process is based on a multi sensorial colour processing. In PC matlab is used for
characterization, and more specifically on crossed processing on colour, depending on this signal will be
optical and impedimetric analysis of the objects to be given to microcontroller Atmega 328. The
sorted. Parallel guides, also called channels, are microcontroller in turn will control the servomotors
created on a slanted plant support. The objects to be by PWM signals. These servomotors will control the
sorted are immersed in a continuous, free-falling flow movement of robotic arm, by controlling their angular
along said guides [1] [2]. By another way this project movement. Thus the robotic arm will be fully
can be treated an automated material handling system controlled by servomotors. The gripper of robotic arm
& can be designed by following way. It synchronizes will pick the object place it depending on its size.
the movement of robotic arm to pick the objects This is full automatic process no manual support is
moving on a conveyor belt. It aims in classifying the needed. The microcontroller used here is with the
coloured objects which are coming on the conveyor support of Arduino kit. The Arduino is good platform
by picking and placing the objects in its respective for robotics application. It is the software and
pre-programmed place. Thereby eliminating the hardware also, using both the above system is
monotonous work done by human, achieving developed. Thus the real time, continuous object
accuracy and speed in the work. The project involves sorting can be done.
colour sensors that senses the object’s colour and
sends the signal to the microcontroller. The
microcontroller sends signal to circuit which drives
the various motors of the robotic arm to grip the
object and place it in the specified location. Based

Proceedings of 3rd IRAJ International Conference, 5th January 2014, Mumbai, India. ISBN: 978-93-82702-51-1

56
 Automation Of Object Sorting System Using Pick & Place Robotic Arm & Image Processing

The camera used in this case is Logitech PN 960-

000748 whose technical specifications are:
• Video calling (640 x 480 pixels)
• Video capture: Up to 1024 x 768 pixels
• Fluid Crystal Technology
• Photos: Up to 1.3 megapixels (software enhanced)
• Built-in mic with noise reduction
• Hi-Speed USB 2.0 certified (recommended)
• Universal clip fits laptops, LCD or CRT monitors

Fig. 1 Block Diagram

Fig. 2 Camera
A. Microcontroller
The ATmega328 is a low-power CMOS 8-bit C. Matlab and Image Processing
microcontroller based on the AVR enhanced RISC The name MATLAB stands for Matrix Laboratory.
architecture. By executing powerful instructions in a MATLAB was written originally to provide easy
single clock cycle, the Atmega 328 achieves access to matrix software developed by the
throughputs approaching 1 MIPS per MHz allowing LINPACK (linear system package) and EISPACK
the system designer to optimize power consumption (Eigen system package) projects. MATLAB is a high-
versus processing speed.The AVR core combines a performance language for technical computing. It
rich instruction set with 32 general purpose working integrates computation, visualization, and
registers. All the 32 registers are directly connected to programming environment. Furthermore, MATLAB
the Arithmetic Logic Unit (ALU), allowing two is a modern programming language environment: it
independent registers to be accessed in one single has sophisticated data structures, contains built-in
instruction executed in one clock cycle. The resulting editing and debugging tools, and supports object-
architecture is more code efficient while achieving oriented programming. These factors make
throughputs up to ten times faster than conventional MATLAB an excellent tool for teaching and research.
CISC microcontrollers. The Atmega 328 provides the MATLAB has many advantages compared to
following features: 4K/8Kbytes of In-System conventional computer languages (e.g., FORTRAN)
Programmable Flash with Read-While-Write for solving technical problems. MATLAB is an
capabilities, 256/512/512/1Kbytes EEPROM, interactive system whose basic data element is an
512/1K/1K/2Kbytes SRAM, 23 general purpose I/O array that does not require dimensioning. It has
lines, 32 general purpose working registers, three powerful built-in routines that enable a very wide
flexible Timer/Counters with compare modes, variety of computations. It also has easy to use
internal and external interrupts, a serial graphics commands that make the visualization of
programmable USART, a byte-oriented 2-wire Serial results immediately available. Applications are
Interface, an SPI serial port, a 6-channel 10-bit ADC collected in packages referred to as toolbox. There
(8 channels in TQFP and QFN/MLF packages), a are tool boxes for signal processing, symbolic
programmable Watchdog Timer with internal computation, control theory, simulation, optimization,
Oscillator, and five software selectable power saving and several otherof applied science and engineering
modes. This allows very fast start-up combined with [17]. Image can be assumed as the visualization of
low power consumption.The 16 MHz Crystal what vision senses that is captured by camera. Image
Oscillator module is designed to handle off-chip is considered as a two dimensional function with
crystals that have a frequency of 16 MHz. The crystal variables that represent the spatial coordinate. It holds
oscillator output is fed to the System. As an information about color as well as shapes. In color
alternative to using a crystal, you can use an image, RGB color model mixes those three prime
externally generated 16 MHz clock source as input color components, red, green and blue, to produce
tothe on-chip 16 MHz oscillator. another color. Image capturing and processing have
been used widely in diverse applications, such in
B. Camera medical and surveillance applications.
The camera used in this case will be overhead
camera, it will take the snapshot of the object for D. Arduino
colour sensing purpuse. The image captured by the Arduino is a tool for making computers that can sense
camera will be processed by image processing using and control more of the physical world than your
matlab. desktop computer. It's an open-source physical
computing platform based on a simple

Proceedings of 3rd IRAJ International Conference, 5th January 2014, Mumbai, India. ISBN: 978-93-82702-51-1

57
 Automation Of Object Sorting System Using Pick & Place Robotic Arm & Image Processing

microcontroller board, and a development Others are electrically equivalent but change the form
environment for writing software for the board. factor, sometimes permitting the continued use of
Arduino can be used to develop interactive objects, Shields, sometimes not. Some variants even use
taking inputs from a variety of switches or sensors, completely different processors, with varying levels
and controlling a variety of lights, motors, and other of compatibility.
physical outputs. Arduino projects can be stand-
alone, or they can be communicating with software E. Robotic Arms & Servomotors
running on your computer (e.g. Flash, Processing, Arms are types of jointed robot manipulator that
MaxMSP.) The boards can be assembled by hand or allow robots to interact with their environment. Many
purchased preassembled; the open-source IDE can be have onboard controllers or translators to simplify
downloaded for free. The Arduino programming communication, though they may be controlled
language is an implementation of Wiring, a similar directly or in any number of ways. Due to this fact,
physical computing platform, which is based on the standalone arms are often classified as full robots.
Processing multimedia programming environment. The robot used in this project is 4 Axis Robotic Arm.
An Arduino board consists of an 8-bit Atmel AVR 4 Axis Robotic Arm is designed for small mobile
microcontroller with complementary components to robots. It can grip objects with the size up to 60mm
facilitate programming and incorporation into other with the force up to 250gms. Arm has reach of 23cm.
circuits. An important aspect of the Arduino is the It can lift the payload up to 400gms. Robotic Arm
standard way that connectors are exposed, allowing comes fully assembled and ready to use. First two
the CPU board to be connected to a variety of axis of the arm are made up of NRS-995 dual bearing
interchangeable add-on modules (known as shields). heavy duty metal gear motors and remaining 2 axis
Most boards include a 5 volt linear regulator and a 16 and gripper uses NRS-585 dual bearing plastic gear
MHz crystal oscillator. The Arduino board exposes servo motors. Axis 2 and 3 enables gripper to
most of the microcontroller's I/O pins for use by other maintain its angle constant with the surface while
circuits. There are many other microcontrollers and moving up and down. Robotic arm can do Left-Right,
microcontroller platforms available for physical Up-Down while keeping gripper parallel to surface,
computing. Arduino also simplifies the process of Twist motions and Gripping action. Robotic Arm will
working with microcontrollers, but it offers some require current up to 5Amps. Make sure that your
advantage for teachers, students, and interested robot can supply that much amount of current for
amateurs over other systems: proper operation of the arm. The robotic arm has
 Inexpensive -The least expensive version of following specifications.
the Arduino module can be assembled by  Number of Axis: 4 + Gripper
hand.  Gripping force: 250gms (Maximum)
 Cross-platform - The Arduino software runs  Gripping jaw length: 43mm
on Windows, Macintosh OSX, and Linux  Gripping jaw width: 60mm
operating systems.  Weight: 541gms (Including 2 NRS-995 and
 Simple, clear programming. 3 NRS-585 servo motors)
 Open source and extensible software- The  Operating voltage: 5V to 6V
Arduino software is published as open  Reach: 23cm
source tools, available for extension by Servos are DC motors with built in gearing and
experienced programmer. feedback control loop circuitry. And no motor drivers
 Open source and extensible hardware - The required. A servomotor is a rotary actuator that
Arduino is based on Atmel's ATMEGA8 and allows for precise control of angular position. They
ATMEGA168 microcontrollers. consist of a motor coupled to a sensor for position
There are a great many Arduino-compatible and feedback, through a reduction gearbox. They also
Arduino-derived boards. require a relatively sophisticated controller, often a
dedicated module designed specifically for use with
servomotors. Servomotors are used in applications
such as robotics, CNC machinery or automated
manufacturing. The servo motor has some control
circuits and a potentiometer (a variable resistor) that
is connected to the output shaft. This pot allows the
Fig. 3 Arduino Kit control circuitry to monitor the current angle of the
servo motor. If the shaft is at the correct angle, then
Some are functionally equivalent to an Arduino and the motor shuts off. If the circuit finds that the angle
may be used interchangeably. Many are the basic is not correct, it will turn the motor the correct
Arduino with the addition of commonplace output direction until the angle is correct. The output shaft of
drivers, often for use in school-level education to the servo is capableof traveling somewhere around
simplify the construction of buggies and small robots. 180 degrees.Usually, its somewhere in the 210 degree

Proceedings of 3rd IRAJ International Conference, 5th January 2014, Mumbai, India. ISBN: 978-93-82702-51-1

58
 Automation Of Object Sorting System Using Pick & Place Robotic Arm & Image Processing

range, but it varies by manufacturer. A normal servo G. Conveyor Belt

is used to control an angular motion of between 0 and The conveyor motor receives power from battery. A
180 degrees. conveyor belt consists of two or more pulleys, with a
Table I. Axis Capabilities continuous loop of material - the conveyor belt - that
rotates about them. One or both of the pulleys
are powered, moving the belt and the material on the
belt forward. The powered pulley is called the drive
pulley while the unpowered pulley is called the idler.
Conveyor frames are supplied with either butting
plate (standard) or hook and bar attachments to secure
A normal servo is mechanically not capable of each segment together. Heavy duty rollers are
turning any farther due to a mechanical stop built on supplied with shafts.
to the main output gear. The amount of power applied
to the motor is proportional to the distance it needs to
travel. So, if the shaft needs to turn a large distance,
the motor will run at full speed. If it needs to turn
only a small amount, the motor will run at a slower
speed [14] [15] [16].
Fig. 5Conveyor Belt

III. Result
We can assume objects in circular, rectangular shape
in different colours so the result is
Table II. Result

Fig. 4 Four Axis Robot

The motor is paired with some type of encoder to

provide position and speed feedback. In the simplest
case, only the position is measured. The measured CONCLUSION
position of the output is compared to the command
position, the external input to the controller. If the Fully functional sorter machine can be implemented
output position differs from that required, an error by using a structure of parallel and independent
signal is generated which then causes the motor to channels in order to increase the overall throughput
rotate in either direction, as needed to bring the which results with a forecasted performance. The
output shaft to the appropriate position. As the project can work successfully. There are two main
positions approach, the error signal reduces to zero steps in sensing part, objects detection and
and the motor stops.More sophisticated servomotors recognition. The system can successfully perform
measure both the position and also the speed of the handling station task, namely pick and place
output shaft. They may also control the speed of their mechanism with help of sensor. Thus a cost effective
motor, rather than always running at full speed. Both Mechatronics system can be designed using the
of these enhancements, usually in combination with a simplest concepts and efficient result can be
PID control algorithm, allow the servomotor to be observed.
brought to its commanded position more quickly and
more precisely, with less overshooting. The servo REFERENCES
turn rate, or transit time, is used for determining servo
rotational velocity. This is the amount of time it takes [1] AlessandroGolfarelli, RossanoCodeluppi and Marco
Tartagni, “A Self-Learning Multi-Sensing Selection
for the servo to move a set amount, usually 60 Process: Measuring Objects One by One by”,ARCES –
degrees. For example, suppose you have a servo with LYRAS LAB University of Bologna, Campus of Forlì, 1-
a transit time of 0.17sec/60 degrees at no load, this 4244-1262-5/07/$25.00 ©2007 IEEE, IEEE SENSORS
means it would take nearly half a second to rotate an 2007 Conference.
[2] Sahu, S., Lenka, P.; Kumari, S.; Sahu, K.B.; Mallick, B.;
entire 180 degrees “Design a colour sensor: Application to robot handling
radiation work”, Vol. 56, No. 10, pp. 365- 368, 2007,
Industrial. Engineering.
[3] www.google.com/microepsilon.com/catcolorsensor—e
[4] www.shortcourse.com/www.sensors.com/optical/imagesens
ors
[5] wwwgoogle.com/wisegeek.com/whatis optical
sensors.htmwww.google.com/osa.org/sensors
[6] www.pdfgenicom/compacsort.com/sorters
Fig.5 Servomotor Rotation [7] www.pdfgenicom/indiamart.com/sorting machine

Proceedings of 3rd IRAJ International Conference, 5th January 2014, Mumbai, India. ISBN: 978-93-82702-51-1

59
 Automation Of Object Sorting System Using Pick & Place Robotic Arm & Image Processing
[8] www.google.com/ATMEGA 48A/PA/328/P/DATASHEET [12] www.google.com/eyantra.org/home/project
SUMMARY [13] Project on“ Pick n Place Robot”, made by Bharat Jain and
www.google.com/zimbio.com/clip-base cam Dinesh Rajput, under guidance of Prof.KaviArya, IIT
[9] [www.google.com/pdfsb.com/smps Mumbai,2010.
[10] www.google.com/pdfgeni.com/vermnlstweb.nl [14] www.pdfgeni.com/servo-tutorial
[11] www.google.com/arduino.cc/main/software [15] www.pdfgeni.com/matlab- mathematical lab/pdf



Proceedings of 3rd IRAJ International Conference, 5th January 2014, Mumbai, India. ISBN: 978-93-82702-51-1

60