Artificial Intelligence in Agriculture 4 (2020) 172–183

Contents lists available at ScienceDirect

Artificial Intelligence in Agriculture

journal homepage: http://www.keaipublishing.com/en/journals/artificial-
intelligence-in-agriculture/

Design of a 4 DOF parallel robot arm and the firmware implementation

on embedded system to transplant pot seedlings
Rahul K., Hifjur Raheman ⁎, Vikas Paradkar
Agricultural & Food Engineering Department, Indian Institute of Technology Kharagpur, India

a r t i c l e i n f o a b s t r a c t

Article history: This paper presents a firmware design and its implementation on a real time embedded system for driving a 4
Received 17 July 2020 DOF parallel robot arm. The firmware primarily comprised of two components to produce motion of the robot
Received in revised form 14 September 2020 arm: a) generation of continuous position coordinates and b) generation of actuating signals. These two compo-
Accepted 14 September 2020
nents were processed in two different microcontrollers with a common communication bus. The position gener-
Available online 21 September 2020
ation algorithm produced and transmitted continuous position data to the motion generation algorithm in the
Keywords:
form of G-code strings by reading the input positions which were previously stored by the user in EEPROM mem-
Robot firmware ory of the microcontroller. The receipt of a handshake signal synchronized the data transmission between these
Parallel robot arm components through a communication bus. An LCD display and keypad were used as human-machine interface
Automation (HMI) to communicate with the user to set the robot target coordinates. The mechanical structure of the robot
SoC microcontrollers, pot seedlings arm comprised of multiple links which were actuated by stepper motors. The workspace boundary were sensed
by limit switches. The kinematic equations represented the gripper position for the corresponding input joint an-
gles. A microcontroller was used to compute kinematic equations of the robot arm with the help of motion gen-
eration algorithm to generate actuation signals for the simultaneous movement of robot joints. The kinematic
equations were solved with the dual-core capability of the microcontroller using real time operating system
(RTOS), which made the computation faster with an average computation time of 198 μs per step. The developed
firmware was implemented and tested on a 4 DOF parallel manipulator using embedded microcontrollers for
continuous pickup and place of the paper pot seedlings for automating the metering of pot seedlings. The cycle
time taken for pickup and dropping of each seedling was 3.5 s with a success rate of 93.3%.
© 2020 The Authors. Publishing services by Elsevier B.V. on behalf of KeAi Communications Co. Ltd. This is an open
access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/).

1. Introduction robot firmware implemented on PC or embedded microcontroller

plays a crucial role in developing the necessary actuation signals by
A robotic system generally consists of a motion generating mechan- making decisions through task allocation, algorithm computation and
ical hardware module, an electronic control module, a software module data transmission. Among the various modules of the robotic system,
that run a decision support system to drive the electromechanical actu- the firmware implemented on the embedded microcontroller decides
ators and a communication protocol for data transfer within and outside the control and action of each module to perform the predefined
the system (Angeles, 1997). The electronic module would consist of em- tasks. The typical function of a robot firmware is shown in Fig. 1.
bedded microcontrollers, PLC's, or other advanced hardware such as The firmware can fetch and modify data from storage memory to per-
FPGA's to generate the actuating signals. These signals are then received form the tasks according to the instructions provided by the user. It drives
by the motor drivers to supply driving power to the actuators. Power the actuators by reading the sensors connected with the robotic system. It
regulators are used to provide different operating power levels for decides to channel the data transfer either internally or externally to read,
each component according to its working range. Also, multiple sensors write or display data through the required data bus.
are used to monitor the environment (Joseph Raj et al., 2019) and the
robot operating parameters (position measuring devices such as en- 1.1. Literature review
coders, limit switches and potentiometers) (Chen et al., 2018). The elec-
tromechanical actuators such as stepper and servo motors provide Robot operating system (ROS) is an open-source firmware architec-
driving torque to the robot joints through power transmission mecha- ture, mainly designed for the development of robotic tools. It consists of
nisms like gears, timing belts and pulleys with minimal backlash. The several libraries which are capable of performing complex tasks such as
navigation robots, multiple robot coordination assisted with machine
⁎ Corresponding author. vision and sensor integration. Araujo et al. (2015) developed a mobile
E-mail address: [email protected] (H. Raheman). robot platform using an Arduino board controlled by ROS. The control

https://doi.org/10.1016/j.aiia.2020.09.003
2589-7217/© 2020 The Authors. Publishing services by Elsevier B.V. on behalf of KeAi Communications Co. Ltd. This is an open access article under the CC BY license (http://
creativecommons.org/licenses/by/4.0/).
R. K., H. Raheman and V. Paradkar Artificial Intelligence in Agriculture 4 (2020) 172–183

Nomenclature

θa Active arm joint angle

θp Passive arm joint angle
θtr Transmission angle
α Passive arm angle with respect to active-passive joint
orientation
β Active arm linear angle with horizontal
γ Inner angle of active arm
C(x,y) End effector position in Cartesian plane
COM bus Communication bus
EEPROM Electrically Erasable Programmable Read Only Memory
HMI Human Machine Interface
k and s Linear distance between origin of active arms and end
effector position
L1 Length of active arm Fig. 1. Functions of a robot firmware.
L2 Length of passive arm
LCD Liquid crystal display
Marlin software requires an additional system to input the positional
m Horizontal distance between the active-passive joints
coordinates. Generally, the robots run by Marlin firmware uses an exter-
Mdep Elbow position memory address for dropping position
nal PC to generate the positional G-code strings. Also, the amount of
Mdp Dropping point memory address
memory requirement of Marlin firmware is high because of additional
Mpep Elbow position memory address for pickup point
modules written for the functioning of 3D printer components. The op-
Mpp Memory address of pickup position value
eration speed cannot be improved significantly due to its core design
n Vertical distance between the active passive joints
(3D printers does not require high operation speed) and it requires an
O Origin of the robot axis
external code generation module to direct the actuators.
PGF Position Generation Firmware
Many a researchers have been using Marlin firmware to develop a ro-
SGF Signal Generation firmware
botic system to perform several useful tasks. This external software sup-
plies the necessary codes to manipulate the robotic system. It creates
difficulty in operating the robotic system in a hostile environment and
logic was developed in a PC and the command signals were sent to with space constraints. This paper presents a firmware architecture and
Arduino through a USB communication. This robot was built for educa- implementation algorithms to run a 4 DOF parallel robot, which could
tional purposes to demonstrate swarm robotics, multi-robot surveil- be deployed in an embedded system that comprised of SoC
lance and rescue tasks. The robot used an onboard camera to navigate microcontrollers, HMI and electronic components which eliminated the
and send the environment data wirelessly. Kamel et al. (2017) devel- need of an external PC to generate the positional coordinates. To imple-
oped a model in ROS for trajectory tracking technique for unmanned ae- ment and validate the developed firmware, it was implemented on a
rial vehicles (UAV). Though ROS firmware architecture is designed for real time embedded hardware, a robot arm based vegetable transplanter
solving complicated robotic operations, advanced hardware with high was selected to meter the seedlings of vegetables raised in paper pots.
computation power is required to deploy the code blocks developed
in ROS firmware. Therefore, still there is a requirement for a simple 1.2. Automation of vegetable transplantation
and efficient firmware structure that could be implemented on a com-
mercially available system on chip (SoC) microcontrollers. Vegetable transplanters are used in agriculture to simplify the man-
The most notable open-source firmware available for driving a ro- ual process of planting already grown seedlings in agricultural field to
botic system using SoC is ‘Marlin’ firmware (Wijnen et al., 2016; improve the yield and reduce drudgery of the human labor (Kumar
Kruger et al., 2018). Marlin is an open-source software constructed and Raheman, 2011; Dihingia et al., 2018). Pot seedling transplanters
using C++ language, dedicatedly developed for operating 3D printers generally consist of a seedlings pickup mechanism (extracts seedlings
using embedded microcontrollers. Some typical functions of Marlin from a feeding conveyor and place it into a drop tube one by one), con-
firmware are: a) positioning the printing nozzle in x, y and z coordinates veying unit (to continuously transfer seedlings towards the pickup unit)
according to the commanded locations by a slicing software, b) driving and a soil opening-closing unit. Researchers have been working on au-
the extruder for continuous feeding of the printing filament, c) heating tomating this mechanical transplanting process by including robots as
the filament and printing bed by PID control with the feedback of tem- a picking mechanism to improve its efficiency and to reduce the input
perature sensors. Marlin takes the input of NC (G-codes and M-codes) power requirement (Hwang and Sistler, 1986; Hu et al., 2014; Rahul
codes (Arroyo et al., 2004) to operate the 3D printer, which produces et al., 2019). The robot used in these transplanters picked up seedlings
real 3D components by melting an input filament at high temperature one by one from multiple numbers of fixed pickup locations and
through an extruder. A list of essential G-codes for positioning the print- dropped it into a fixed dropping point (Fig. 2).
ing nozzle of a 3D printer is given in Table 1. To maintain a delicate picking without damaging the seedlings and
Tiansong et al. (2019) developed a 3D printed 3 DOF serial robot to drop the picked seedlings without collisions with other components
using open source Marlin firmware controlled by ATMEGA 1284P mi-
crocontroller. The robot was actuated by geometric codes generated
Table 1
by Pronterface software installed on a desktop personal computer
List of essential G-codes for the nozzle spatial manipulation of 3D printer.
(Surange and Gharat, 2016). Nejatimoharrami et al. (2017) developed
a liquid handling robot equipped with machine vision for automating Code Function
chemical handling. It is a gantry type robot controlled by Marlin firm- G00 X Y Z Rapid linear movement
ware installed on an Arduino Mega microcontroller board. Although G01 X Y Z Linear motion of nozzle at a specified speed
Marlin architecture could be modified for operating a robotic system G28 Homing of X, Y and Z coordinates
G02 and G03 CW and CCW arc at a specified radius
using commercial microcontrollers, it has several drawbacks such as:

173
R. K., H. Raheman and V. Paradkar Artificial Intelligence in Agriculture 4 (2020) 172–183

3) Display of the operating parameters (i.e., number of droppings and

execution code received) through HMI
In this study, a robot control firmware was constructed and de-
ployed on a real time embedded system to control a 4 DOF parallel ma-
nipulator robotic system used for continuous pick and place operations
of paper pot seedlings in an agricultural field. The details of its develop-
ments and its laboratory evaluation are given in the following sections.

2. Materials and methods

2.1. Firmware architecture of the robotic system

Fig. 2. Robot pickup and drop positions with a via point trajectory.
The proposed firmware architecture of the robot system is shown in
Fig. 3. It consisted of two components: 1) signal generation firmware
of the transplanter, the manipulator needs to move from dropping point
(SGF) and 2) position generation firmware (PGF).
(initial coordinates) to each of the pickup locations through a via point.
A user interactive HMI was used to store or modify the data for pick
After picking, the manipulator again reaches the drop point through the
and place task such as the number of positions, approach point, pickup
same via point. The via point is known as the approach point for each
point and dropping point in the EEPROM memory at their specified mem-
pickup location. A systematic motion requirement of a robot for seed-
ory addresses. Also, an external communication bus was used to monitor
ling handling was reported by Rahul et al. (2019). The optimal robotic
and control the robotic system by a remote computer. During the pick and
path requires an approach point, gripping point and a dropping point.
place operation, PGF read the position data (X, Y, Z location) from
The operational speed of picking should be lesser than the speed of ap-
EEPROM memory and compiled it into a G-code string (Ex. G01 X d1,
proach and dropping. This entire problem for the robotic task is consid-
Yd2, Zd3; where dk is the kth position data). The generated G-code was
ered as a pick and place application with multiple fixed pickup positions
then sent to the SGF and waited for the handshake signal to indicate the
and one drop point at variable operating speed. Hence, the hardware
completion of the assigned task. The SGF read the input string and
and software embedded control system for the robot in an automated
interpreted the X, Y and Z position coordinates. The coordinates were
transplanter requires the following features:
used to compute the trajectory points by Bresenham 3D line algorithm.
1) A user interface system to modify the robot motion coordinates and For each trajectory point, the algorithm calculated the inverse kinematics
its parameters without the aid of an external computer of the robotic system (in this case, a planar parallel robot was selected) to
2) A fault diagnosis system to detect the status of sensors and actuators compute the joint angular positions at that instance. The required joint
whenever required motor actuation signals were generated and supplied to follow the

Fig. 3. Generalized firmware architecture of the robotic system.

174
R. K., H. Raheman and V. Paradkar Artificial Intelligence in Agriculture 4 (2020) 172–183

trajectory at the specified linear speed of the manipulator. Further, accel- coupled with the platform. Thus, the rotation of the stepper motor was
eration at the beginning and deceleration at the end of the trajectory were converted into a linear motion in Z direction. A parallel jaw gripper was
provided to maintain smooth actuation. Once, the manipulator reached attached to the robot arm at the connecting point of the two passive links.
the target position, SGF sent a handshake signal to the PGF through serial The gripper was made to open and close using a servo motor. An-
bus to continue the cycle for the next position. other servo motor mounted on the passive link was attached to the grip-
per to provide the rotary motion of the gripper in planner coordinates.
2.2. Design of robot arm Four limit switches were provided at the robot arm frames to act as
sensing boundary of the robot arm. Two limit switches (limit switches
The validation of the proposed firmware architecture for operating a 1 and 2) sensed the boundary position of the active robot joints. The
robotic system was made by a) designing the mechanical structure of limit switches 3 and 4 detected the upper and lower boundary positions
the robot arm, b) deriving the kinematic equations of the robot arm, of the platform, in Z-up and Z-down directions, respectively.
c) designing the embedded hardware to actuate the mechanical system
and d) implementing the firmware in a deployable SoC microcontroller 2.2.2. Inverse kinematics of the robot arm
of the hardware. The following sections explain the practical implemen- The input joint angles θa1 and θa2 for the given gripper position were
tation of the proposed firmware architecture. computed with inverse kinematics. The geometric line diagram of the
parallel manipulator is shown in Fig. 5 with three operating regions to
2.2.1. Design of mechanical structure of robot arm visualize easily and to develop the implementable robot equations.
The 3D CAD model of a 4 DOF parallel robot arm is presented in The regions are separated based on the position of the end effector,
Fig. 4. It consisted of two active links those were driven by stepper mo- i.e., C(x, y) in the planar space.
tors through timing belt and pulley. One end of the two passive links Using cosine rule in triangle O1AC (referring Fig. 5), the inner angle
was joined together and the other ends were connected with the corre- of active arm 1.
sponding active links. Thus the combined motion of the active joints
provided a planner manipulation in X and Y direction. ! qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
2
L1 2 −L2 2 þ k
The arm assembly platform, which comprised the robot links and ac- γ 1 ¼ acos where, k ¼ ðx þ L0 Þ2 þ y2 ð1Þ
tuation stepper motors was mounted on the support frames through a 2 L1 k
slider with a linear bearing that permitted motion only in ‘Z’ direction. An- Similarly, using cosine rule at triangle O2DC, the inner angle of active
other stepper motor mounted on top of the frame drove a lead screw arm 2

Fig. 4. CAD model of the developed robot arm to implement the proposed firmware.

175
R. K., H. Raheman and V. Paradkar Artificial Intelligence in Agriculture 4 (2020) 172–183

Fig. 5. Geometric line diagram of the manipulator with its operating regions.

!  
qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi y
L1 2 −L2 2 þ s2 β1 ¼ atan and β2
γ2 ¼ acos where, s ¼ ðx−L0 Þ2 þ y2 ð2Þ jL0þ xj 
2 L1 s
y
¼ π−atan for the location of‘C’in region−1 ð4Þ
jL0 −xj
Based on the location of the manipulator point ‘C’ within the region,
the active arm linear angle with horizontal (β1 and β2) varies as:
   
y y
β1 ¼ atan and β2 β1 ¼ π−atan and β2
jL0 þ xj jL0 þ xj
y y
¼ atan for the location of‘C’in region−2 ð3Þ ¼ π−atan for the location of‘C’in region−3 ð5Þ
jL0 −xj jL0 −xj

Fig. 6. Block diagram of the electronic control circuit of the robot arm.

176
R. K., H. Raheman and V. Paradkar Artificial Intelligence in Agriculture 4 (2020) 172–183

Table 2 The vertical distance of link, L1y ¼ L1 sin θa1 ð8Þ

List of components used in electronic hardware.

Component Specifications Functions The vertical distance of link, L2y ¼ L2 sin θa2 ð9Þ
ESP 32 Clock frequency: 240 MHz SGF firmware installed to drive the
CPU: 2 cores (32 bit), robot joint actuators Length between points Band C, LBC
Tensilica LX6   abs L1y , L2y
microprocessors ¼ min L1y , L2y þ ð10Þ
Memory: 520 Kib SRAM
2
Peripheral interfaces: 3
UART, 2 I2C 2.3. Design of embedded hardware for the robot arm
Arduino Clock frequency: 16 MHz PGF firmware installed to generate and
Mega CPU: 8 bit ATMEGA 2560 supply the motion control codes
Memory: 8Kb SRAM The electronic system incorporates the necessary hardware compo-
Peripheral interfaces: 4 nents to realize the robot arm motion. The block diagram of electronic
UART, 1 I2C components used and their associated connections are shown in
Buck Input: 36 V DC (max), To supply recommended constant DC
Fig. 6. The list of hardware components used with their specifications
converter current: 5A (max) power to actuators and
Output: 5 V and 12 V microcontrollers and functions are summarized in Table 2.
HMI Keypad To receive input from the user and to The SGF firmware deployed on dual-core ESP 32 microcontroller actu-
LCD display the operating parameters ated the stepper motors of robot joint by supplying the essential signals to
Actuators NEMA 17 Stepper motors To provide driving torque to the robot the corresponding motor drivers. It calculated both robot kinematic and
and joints
Servo motors (up to 180°
trajectory equations. Limit switches were also connected to the ESP 32,
rotation) to sense the robot joints whenever it reached the robot boundary. The
PGF firmware installed on the Arduino Mega board took input from the
HMI keypad and displayed the necessary data through an LCD. The
Mega board also controlled the gripper open/close and wrist rotation of
The active joint angles (θa1 and θa1) are determined by
the robot arm. A COM bus connected these two microcontrollers, trans-
θa1 ¼ β1 þ γ1 ð6Þ ferred data between them. Buck converters provided the required DC
voltage to these electronic components for their operation and actuation.
θa2 ¼ β2 −γ2 ð7Þ
2.4. Robot path planning
The transmission angle influences the manipulator stability θtr be-
tween active and passive links and is directly proportional to the dis- Robot path planning involves the generation of optimal reachable
tance between points B and C; which is determined by using the points in spatial coordinates from the present position of the manipulator
following equations. to the target location. Command G00 (rapid motion) and G01 (motion of

Fig. 7. Flow diagram of robot kinematics computation through dual-core.

177
R. K., H. Raheman and V. Paradkar Artificial Intelligence in Agriculture 4 (2020) 172–183

Fig. 8. User interactive menu sequence to store data in EEPROM memory.

manipulator at specified speed) were used to move the end effector be- higher the resolution, better is the straight-line approximation, but
tween the specified coordinates (X, Y and Z) with a straight-line trajectory at the cost of increased computation. Hence, the processor needs to
using Bresenham algorithm. calculate the kinematic equations at a higher speed to supply the ac-
tuation signals to the corresponding stepper motors. By utilizing the
2.5. Computation of kinematic equations through dual-core advantage of dual-core computing capability of ESP 32 microcontrol-
ler, the complex algorithms were solved at a minimal time. Since all
The robot kinematics involves the computation of complex geo- the kinematic equations were required to be solved only by the SoC
metric equations to determine the angular positions and velocities microcontroller, the equations were divided into two parts, each
of the joints at minimal time. As discussed in the earlier section, was computed individually in different cores and the results were

Fig. 9. EEPROM memory stack and allocation of data in its associated storage locations.

178
R. K., H. Raheman and V. Paradkar Artificial Intelligence in Agriculture 4 (2020) 172–183

combined to obtain the target angular position. The algorithm for en- address Mpp by navigating the corresponding menu and by entering
abling dual-core computation is given in Algorithm 1 and the block a numeric entity. The dropping position data were stored in the
diagram of dual-core computation is shown in Fig. 7. ESP32 was sup- memory by directing to ‘End effector position’ in dropping position
ported with RTOS (Real Time Operating System); thus multiple tasks menu. With the help of a keypad, the end effector could be moved
were created and ran those tasks simultaneously in different CPU to the desired position and the corresponding coordinates (X and
cores. Two sets of kinematic equations ran independently in cores Y) could be fetched from the SGF through COM bus to store the
1 and 2 by using ‘Semaphore’ (give or take) function by defining dropping point data in memory address Mdp. Elbow position for the
global variables that shared common data between both the cores. dropping coordinate were stored in address Mdep. In this particular
Finally, the computed data from both the cores were collected and
the target angular positions θa1 and θa2 were calculated.

Algorithm 1. Dual-core computation of kinematic equations.

2.6. Construction of position generation firmware

The position generation firmware produced the robot control codes

(G-codes) required for the actuation of the robot arm to perform a partic-
ular task (i.e., in this case, continuous pick and place application of pot
seedlings was considered). The generated signals were sent through the
internal serial communication bus (COM bus) of the robot controller
upon the receipt of handshake signals from the receiver. In order to pro-
duce the continuous control codes which perform pick and place task,
the controller was required to access the position data. Considering the
case of pick and place application as explained in the Introduction section,
the user could provide all the required position data at once which would
be stored in EEPROM memory. Then the controller could access the data
at any time repeatedly without consideration of power supply ON/OFF
criteria. Besides, provision was also made to change specific data in the
memory according to the user's requirement.
The position data storage for the robot actuation was made
through a user interactive menu-based system as shown in Fig. 8.
Navigation buttons of the keypad were used to change the menu se-
quence to access a particular address of the EEPROM memory
(Fig. 9). The number of pickup positions value was stored in memory

179
R. K., H. Raheman and V. Paradkar Artificial Intelligence in Agriculture 4 (2020) 172–183

application, there were multiple numbers of pickup points; which It consisted of five pickup positions and one dropping position. The
required multiple end-effector coordinates and elbow positions. algorithm was tested for continuous pick and drop of pot seedlings
These data were accessed and modified by the user by entering the loaded on a feeding conveyor. Forty numbers of pot seedlings were
pickup position number ‘k’ in menu sequence which could access picked and dropped by rotating the feeding conveyor to supply pot
the memory locations Mpdx(k), Mpdy(k) and Mpep(k) and these par- seedlings to the pickup positions. The average computation time (for
ticular data were stored in the specific memory address. 20 pickup positions i.e., four row of pot seedlings) required to solve
the kinematic equations were recorded. Function ‘micros ( )’ was used
Algorithm 2. Continuous pick and place operation. in the kinematics code block within the SoC microcontrollers of three
The procedural steps for the continuous pick and place operation are different modes and the data were recorded. Table 3 presents the com-
provided in Algorithm 2. It initialized the required variables and the parison of the average computation time to solve the computation of in-
data for pick and place were fetched from the EEPROM memory verse kinematic equation in different modes.
(Fig. 9). It generated the G code string required for the robot motion It was found that, ESP 32 SoC microcontroller computed the kine-
for approach coordinates and transmitted the string to the SGF which matics at less time as compared to the utilization of widely used
actuated the elbow and gripper. After the receipt of acknowledgement Arduino board. Further, usage of dual-core capability through RTOS im-
signal from SGF, the algorithm generated strings for gripping and drop proved the computation speed, which increased the operating speed.
coordinates for the pick and place of the present position. It
incremented the position data and actuated the robot until it was 3.3. Performance evaluation of the developed firmware implemented on the
equal to Mpp. The position data was again reset back to pick and place hardware to pick and drop the pot seedlings
from the initial point.
The robot arm was tested for seedling pickup and dropping. A total
3. Results and discussion of 40 pot seedlings were loaded on the cells of the feeding conveyor
for testing. The sequence of operation for picking and dropping of seed-
3.1. Data entry through HMI lings by the robotic arm is shown in Fig. 14(a–d).
The cycle time taken for pickup and dropping of each seedling was
The HMI system was connected to the PGF microcontroller to display 3.5 s (which maintained delicate picking and dropping of pot seedlings
data and to provide input to the SGF microcontroller for the actuation of without any damage at an average of 17 seedlings per min which is
the robot arm. The developed HMI control for the robotic system, which sufficient for walk behind type vegetable transplanters, Kumar and
comprises an LCD display and Keypad is shown in Fig. 10. The buttons of Raheman, 2011). The average success rate
the keypad were configured for multiple functions (up, down, left, right, (S ¼ Number of successfully picked seedlings
Totalnumberofseedlingsloaded
 100Þ of the robot arm deployed
select, back and numerical entries) to access the data. with the proposed firmware for three trials was computed as 93.3%.
The entry of position data in EEPROM memory of the PGF microcon-
troller is shown in Fig. 11. By navigating the menu system through the 3.4. Comparison of the proposed firmware and the associated hardware
up/down and select buttons user could obtain ‘Enter value’ tab in the with existing systems
display (Fig. 11.a). User could change the number of pickup positions
by the buttons (Fig. 11.b). Then by pressing the select button, the en- It is required to compare the developed firmware and the hardware
tered data were saved in the corresponding EEPROM memory location on which the firmware was deployed. Table 4 presents the comparison
of the microcontroller (Fig. 11.c). This updated data were used by the of various features of the developed firmware and the associated hard-
PGF for the next operation cycle. ware with the existing systems.
The data entry for the dropping position in the corresponding mem- The SMC4-4-16A 16B CNC motion controller has built in firmware
ory address of the EEPROM is shown in Fig. 12. and already well established in several industrial NC machines. In cost
wise this controller is not suitable for operating with agricultural ma-
3.2. Operation of robot arm by the proposed firmware implemented on the chines. Compared to Raspberry pi based ROS system, the developed
embedded microcontrollers firmware with its associated hardware is much lower in cost. Also, the
performance of the developed system proved that it is well suited for
The developed firmware (both SGF and PGF) were deployed in SoC metering pot seedling for transplanting.
microcontrollers of the embedded hardware and it was implemented
on a 4 DOF parallel robot (Fig. 13). 4. Conclusions and future work

In this paper, design of a 4 DOF parallel robot arm, the development

of control firmware and the implementation of the developed firmware
on embedded system to handle paper pot seedlings for vegetable trans-
plantation were studied. A general design of a robot firmware was pro-
posed to produce control codes (G-codes) based on the predefined user
input and motor actuation signals obtained for the internal commands
received, and by sensing the joint angular positions. An embedded elec-
tronic hardware was developed and deployed the proposed firmware to
generate control codes and actuation signals using SoC microcontrollers
(tested using ESP 32, Arduino Mega microcontrollers, LCD-Keypad HMI,
limit switch position sensors and the required power converters). 3D
Bresenham algorithm was used in the firmware to approximate the lin-
ear trajectory of the parallel robot in between initial and target
positions.
Region-based inverse kinematic equations were developed and im-
plemented in the robot firmware, coupled with the Bresenham line al-
Fig. 10. HMI system for the robot arm. gorithm to realize the linear trajectory of the robot in real time. The

180
R. K., H. Raheman and V. Paradkar Artificial Intelligence in Agriculture 4 (2020) 172–183

a) Navigate to positions b) Numerical entry c) Saving the data

Fig. 11. Modifying position data in memory through HMI.

a) Navigate to drop menu b) Motor positions selection c) Move the robot joint

Fig. 12. Setting of motor positions for drop point in memory through HMI.

Fig. 13. Robot arm actuated by the proposed firmware for pick and drop of pot seedlings.

181
R. K., H. Raheman and V. Paradkar Artificial Intelligence in Agriculture 4 (2020) 172–183

Table 3 developed firmware was successfully implemented on an embedded

Time taken to solve the kinematic equation with different microcontrollers. hardware to pick and drop of pot seedlings loaded on a feeding con-
Device used Average computation time per step (μs) veyor using a parallel robot arm with an average cycle time of 3.5 s
(For 20 different positions) with a success rate of 93.3%.
Arduino Mega 4027
ESP 32 (without dual-core) 276
ESP 32 (dual core enabled) 198 Declaration of competing interest

The authors declare that they have no known competing financial

kinematic equations were computed at a lesser time using the dual-core interests or personal relationships that could have appeared to influ-
capability of the ESP 32 microcontroller, which resulted in an average ence the work reported in this paper.
computation time of 198 μs per step as compared to 276 μs and
4027 μs with single-core mode and other commercial microcontrollers. References
A user interactive menu sequence was developed to store the position Angeles, J., 1997. Fundamentals of Robotic Mechanical Systems: Theory, Methods, and Al-
data related to pick and drop in the EEPROM memory using HMI. The gorithms. 2nd edition. Springer-Verlag, New York.

a) Manipulator approaching a seedling b) Manipulator picked the seedling

d) Released the seedling inside the

c) Transferred the seedling to drop point
seedling delivery tube
Fig. 14. Pick and drop of a pot seedling with robot arm under laboratory conditions.

Table 4
Comparison of developed firmware and its associated hardware with existing systems.

Robot control firmware with hardware Cost (approx.) Number of controllable axis Applicability

Proposed system (Microcontrollers with HMI) Rs. 1800 Up to 5 axis Robotic applications
Raspberry pi with ROS and HMI Rs. 4000 Up to 6 axis Robotic applications
SMC4-4-16A 16B Rs. 60,000 Up to 4 axis Industrial CNC machines

182
R. K., H. Raheman and V. Paradkar Artificial Intelligence in Agriculture 4 (2020) 172–183

Araujo, A., Portugal, D., Couceiro, M.S., Rocha, R.P., 2015. Integrating Arduino-based edu- Rahul, K., Raheman, H., Paradkar, V., 2019. Design and development of a 5R 2DOF parallel
cational mobile robots in ROS. J. Intell. Robot. Syst. 77 (2), 281–298. robot arm for handling paper pot seedlings in a vegetable transplanter. Comput. Elec-
Arroyo G, Ochoa C, Silva J and Vidal G., 2004. Towards CNC Programming Using Haskell. In tron. Agric. 166, 105014.
Ibero-American Conference on Artificial Intelligence. Springer, Berlin, Heidelberg, pp. Surange, V.G., Gharat, P.V., 2016. 3D printing process using fused deposition modelling
386–396. (FDM). Int. J. Res. Eng. Technol. (IRJET) 3, 1403–1406.
Chen, W., Khamis, H., Birznieks, I., Lepora, N.F., Redmond, S.J., 2018. Tactile sensors for fric- Tiansong, L., Feng, G., Yilong, Y., 2019. Design of low-cost desktop robot based on 3D
tion estimation and incipient slip detection - toward dexterous robotic manipulation: printing technology and open-source control system. 2019 IEEE 3rd Information
a review. IEEE Sensors J. 18 (22), 9049–9064. Technology, Networking, Electronic and Automation Control Conference (ITNEC).
Dihingia, P.C., Kumar, G.P., Sarma, P.K., Neog, P., 2018. Hand-fed vegetable transplanter for IEEE, pp. 739–742.
use with a walk-behind-type hand tractor. International Journal of Vegetable Science Wijnen, B., Anzalone, G., Haselhuhn, A., Sanders, P., Pearce, J., 2016. Free and open-source
24 (3), 254–273. control software for 3-D motion and processing. Journal of Open Research Software 4,
Hu, J., Yan, X., Ma, J., Qi, C., Francis, K., Mao, H., 2014. Dimensional synthesis and kinemat- e2. https://doi.org/10.5334/jors.78.
ics simulation of a high-speed plug seedling transplanting robot. Comput. Electron.
Agric. 107, 64–72.
Rahul K. is persuing Master of Science at Indian Institute of Technology Kharagpur in Farm
Hwang, H., Sistler, F.E., 1986. A robotic pepper transplanter. Appl. Eng. Agric. 2 (1), 2–5.
Machinery specialization under Agricultural and Food Engineering Department. His re-
Joseph Raj, A.N., Sundaram, R., Mahesh, V.G., Zhuang, Z., Simeone, A., 2019. A multi-sensor
search focuses on implementation of robotics and automation in agricultural machinery.
system for silkworm cocoon gender classification via image processing and support
He is also interested in artificial intelligence, machine vision and mechatronic system de-
vector machine. Sensors. 19 (12), 2656.
sign.
Kamel, M., Stastny, T., Alexis, K., Siegwart, R., 2017. Model predictive control for trajectory
tracking of unmanned aerial vehicles using robot operating system. Robot Operating
Hifjur Raheman is a Professor in the Agricultural and Food Engineering Department of
System (ROS). Springer, Cham, pp. 3–39.
Indian Institute of Technology Kharagpur, India. His research areas are use of robotics in
Kruger, J., Gu, W., Shen, H., Mukelabai, M., Hebig, R., Berger, T., 2018, February. Towards a
agri-machinery, renewable energy operated agricultural machinery, alternate fuels from
better understanding of software features and their characteristics: a case study of
biomass (Biodiesel, producer gas) and bioelectricity production and tillage and traction
marlin. Proceedings of the 12th International Workshop on Variability Modelling of
improvement in tractors and power tillers.
Software-Intensive Systems, pp. 105–112.
Kumar, G.P., Raheman, H., 2011. Development of a walk-behind type hand tractor
Vikas Paradkar is Ph.D Student, Specialization in Farm machinery and Power, Agricultural
powered vegetable transplanter for paper pot seedlings. Biosyst. Eng. 110 (2),
and Food Engineering Department, Indian Institute of Technology Kharagpur, India. His re-
189–197.
search focus is on implementation of robot in vegetable transplanter.
Nejatimoharrami, F., Faina, A., Jovanovic, A., St-Cyr, O., Chignell, M., Stoy, K., 2017. UI De-
sign for an Engineering Process: Programming Experiments on a Liquid Handling
Robot. In 2017 First IEEE International Conference on Robotic Computing (IRC).
IEEE, pp. 196–203.

183