Development and Integration of Laser Sensor Tracking System in Robotic Arm For Path Correction During Welding Operation
Development and Integration of Laser Sensor Tracking System in Robotic Arm For Path Correction During Welding Operation
Corresponding Author:
Tejaswini P.
Industrial Automation & Robotics, Department of Mechanical Engineering, The National Institute of
Engineering
Mysore, India
Email: [email protected]
1. INTRODUCTION
The process includes various technologies of welding, robotics, and sensor technology. Sensor
technology plays a very important role in observing and measuring the parameters in the process. It has the
ability to gather information from the sensor and analyze it. This sensor information is the input source to the
robot controller [1]. The time taken by the light to emit back to the sensor after hitting the welding plates
helps in measuring the difference in the welding path [2]. The change in the welding path affects the position
of the light concentrated on the detector. This makes the sensor identify the welding joints [3]. The
adjustments and setting of the sensor can be done using an application software. Thus, use of sensor
technology in robotic welding leads to risk-free manual welding operation in dangerous work environments.
This helps in maintaining the quality of welding.
The main benefits of laser seam tracking system with welding robot are that the process is faster and
accurate than manual welding, the adjustments and setting of the sensor make the welding accurate, thus the
process helps in maintaining the quality of welding.
The challenge faced in the automobile industries while welding continuously is searching the start
point, end point, and real time tracking [4]–[6]. Another problem is that the taught line and welding (actual)
line may differ. Robots should be automated to detect the path and complete welding continuously. The one
more problem faced in the welding operation is selecting the joint type and setting the joint parameters. So,
this project gives an accurate solution to these features. in the user interface application.
An automatic laser seam tracking system for welding operations plays a very important role in
automobile industries [7]. There are lots of studies done in seam tracking for welding, but no research has
been done in selecting the joint type for the welding and also for customizing the sensor parameters and joint
welding parameters. These inspired us to develop this system.
The main objective of the project is i) to develop most accurate welding operating robot with laser
sensor in all different environment (like in oily surface, uneven surface, in distractive surface etc.), ii) to find
the starting and ending point of the welding line, iii) the laser sensor should be able to detect the path
automatically if there is any difference in taught line and welding (actual) line, iv) to build a user interface
application for selecting the type of joint and for the customized joint setting, and v) to make it fully
automated system in real time monitoring of welding operation.
2. LITERATURE REVIEW
Michalos et al. [8] used a laser diode to remove the effects of lighting and any random disturbances
in the surrounding welding area. The paper explains that the disturbance in the laser lines helps in identifying
the part to be welded, flange, and the path correction. This makes the proper welding in the required area.
This proposed system is deployed on an assembly cell that produces a part of a commercial vehicle door.
Graaf et al. [9] describes an algorithm for real time seam tracking. This makes the laser welding
accurate. The algorithm is based on the trajectory-based control architecture. The position and orientation in
cartesian coordinates are added to the robot trajectory while the robot is moving. This helps in getting the
sensor information and generate the robot trajectory.
Huang et al. [10] describes a system for identifying the information of welding seam deviation. The
information can be taken by four parts: sensors, controller, servo system and actuator. The sensors detect the
path deviations of the laser beam from the welding path. The control system controls the output signals. Then
the servo system drives the actuator. At last, the laser beam welds the path in the real seam line.
3. RESEARCH METHOD
3.1. System architecture
3.1.1. Motoplus SDK
Motoplus SDK is an application software used for controlling and monitoring robot systems. The
SDK has a set of programming instructions for the custom application task which simultaneously runs on the
robot controller. These are the real time tasks to control and monitor the robot with high performance. The set
of programming instructions is written in C programming.
As the features, the Motoplus key can be integrated with any device to develop an application and
can run on offline mode using windows PC. It also does not require any hardware, and there is a list of
application programming interfaces (APIs) for programming.
Motoplus APIs which are used for giving commands to robot are mpUsrRoot to start the entire
application, mpCreateTask to create a task, mpConnect to send a connection request from client to server,
mpSocket to create a Socket, mpTaskDelay to delay a task, mpClose to close a socket,
mpReceiveSkillCommand to receive a sensor command, mpEndSkillCommandProcess to notify the system
that the sensor command is received, mpClockAnnounce to wait for an interpolation control cycle to start,
and mpMeiPutCorrPath to give the path correction for every interpolation control cycle.
The API mpMeiPutCorrPath() used for path correction synchronize with the system’s interpolation
control cycle by using an API mpClockAnnounce(), which waits to start the per interpolation control cycle.
Therefore, mpMeiPutCorrPath() corrects the path for every interpolation control cycle. Thus, we can say
interpolation control cycle plays a very important role in path correction.
Therefore, the interpolation control cycle per calibrated length is given by:
Length of the welding line
Interpolation Control Cycle Per Calibrated Length
Development and integration of laser sensor tracking system in robotic arm ... (Tejaswini P.)
198 ISSN: 2722-2586
For every interpolation control cycle, the request for path correction is checked, and if there is any request for
path correction, then the amount of path correction is calculated by an API mpMeiPutCorrPath().
Figure 1. Motoman and YRC1000 Robot Controller Figure 2. Seam tracking laser sensor
Laser sensor operates on the principle of triangulation. Figure 3 explains the working principle of
triangulation [13]–[15]. The laser triangulation sensor consists of a laser, detector and a lens, a lens focuses
the beam on the detector. Firstly, the laser emits light on the surface. The laser beam on the surface is
reflected on the detector through the lens. Depending on the position of the beam on the detectors, the angle
(α) is calculated. The angle is inversely proportional to the height of the surface. Therefore, when the height
increases the angle decreases and vice versa [16]. Thus, the height from the sensor to the target surface can
be calculated.
IAES Int J Rob & Autom, Vol. 11, No. 3, September 2022: 196-204
IAES Int J Rob & Autom ISSN: 2722-2586 199
3.2. Implementation
The setup of the proposed system is explained in Figure 5 with a block diagram. The proposed
system of the laser seam tracking system with Yaskawa robot for continuous welding operation is developed.
The robot controller should be set up properly with the robot and laser sensor for tracking [18]. So, connect
the ethernet cable from the ethernet switch to the ethernet port 1 inside the robot controller.
Development and integration of laser sensor tracking system in robotic arm ... (Tejaswini P.)
200 ISSN: 2722-2586
The robot communicates with the sensor through TCP/IP protocol. The IP address of the robot
should be compatible with sensor’s subnet mask to communicate [1]. Then run the sample program (JOB
file) for testing the turning ON of the sensor. If the communication between sensor and robot is established,
then run the path correction program (JOB file) from the robot pendant. The JOB file for path correction is
explained further. JOB file is a file which is created in the pendant to communicate and give instruction to the
robot.
NOP
*TOP
MOVJ P000 VJ=5.00
MOVL P003 V=11.0
SKILLSND "SET_SEAM" TIMER T=1.00
SKILLSND " START_TR "
SKILLSND "GET_M"
MOVL P004 V=11.0
SKILLSND "END_TRAC"
TIMER T=1.00
ADVINIT
JUMP *TOP
END
1) Run the JOB file. The command “NOP” is the no operation function.The command “*TOP” is used for
the continuous looping of the operation. “MOVJ P000 VJ=5.00” is the command for robot to return to
its home position. “MOVL P003 V=11.0” command moves the robot linearly to the point where the
welding has to be started.
2) Next, the JOB file sends a command to check the seam number by a command SKILLSND
"SET_SEAM" and waits for 1 second.
3) Then the Motoplus application receives the command and sets the seam number as it is customized in
the application software.
4) Then the JOB file sends a command to start the laser sensor by giving an instruction SKILLSND
“START_TR”.
5) The Motoplus application receives the command and requests the laser sensor to turn ON and start to
track.
6) After turning ON the laser sensor, the instruction SKILLSND “GET_M” command is sent to starts the
path correction. The API mpMeiPutCorrPath() is used to give the amount of path correction. This
process runs in an infinite loop.
7) The amount of path correction is received by the Motoplus application and applied to the process while
executing MOVL command in the JOB file. MOVL is the command which is between start track
SKILLSND “START_TR” and end track SKILLSND “END_TRAC” commands.
8) After the path correction, the JOB file sends a command to turn off the laser sensor by the instruction
SKILLSND “END_TRAC”.
9) The Motoplus application receives the command and requests the laser sensor to turn OFF.
10) After executing once, the path correction values are stored in the robot controller.
11) So, to initialize to the home position, an instruction ADVINIT is executed in the JOB file. ADVINIT is
an instruction that clears all the path correction value for every interpolation cycle.
Application software is a user interface application used to customize the system process. The
executable file of this application software is made to run on the PC. This application software works inside
the laser sensor system and starts the sensor functioning.
An application software has the following screens: i) home screen, which consists of live profile
image of welding, measurements, system status and laser sensor status, ii) info, to display the message
logging, robot communication log and the live correction values during the process, iii) setup screen which
displays the system parameters that need to be customized for the best performance during system operation,
and iv) system parameters, which are joint type, joint width, joint mismatch, analysis sensitivity and tracking
position.
The application software is launched by clicking the executable file. Once the communication
between the application software and laser sensor is established, the display shows a correct initialization
with no errors occurring. Therefore, the system is ready to begin tracking. Now the robot communicates with
the sensor through TCP/IP. The IP address of the sensor has to be correctly set up in the robot controller. The
IP Address of the robot should match the sensor’s subnet so the robot can communicate with the sensor.
Once the communication is established, in the info screen we can see “Connected to Robot” as shown in
Figure 6.
IAES Int J Rob & Autom, Vol. 11, No. 3, September 2022: 196-204
IAES Int J Rob & Autom ISSN: 2722-2586 201
The setting of sensors and its parameters can be adjusted by an application software, which is shown
in Figure 8. The figure shows the successful live vision of tracking in the form of graphical form, the status
of the proposed system and the successful connection of robot with the sensor.
In searching the start point, end point and real time tracking. The start point of the welding line is
registered while teaching the robot. This position is registered in the command “START_TR” in the JOB file.
By calling this command, the seam tracking laser sensor detects the start point of the welding line. Similarly,
by calling the command “END_TRAC” the end point of the welding line can be detected.
The real time tracking can be achieved by using the position data received from the laser sensor.
This helps in preparing the corrected data of the welding line either in base coordinate system or user
Development and integration of laser sensor tracking system in robotic arm ... (Tejaswini P.)
202 ISSN: 2722-2586
coordinate system. Based on the data the welding torch is corrected. The data received from the laser sensor
is sent to the YRC1000 robot controller. The data are in a three-dimensional position. The YRC1000 converts
the input sensor data to robot position data. By converting, the sensor data and the robot position data
relatable. Now the laser sensor will be able to detect the joint of the work piece at the sensor, which can be
seen in the application software Figure 8.
The values of path correction can be viewed in the telnet software: Telnet is based on client-server
protocol. In this system, Telnet client is our computer. Our computer establishes a connection to a remote
host. Remote host is our robot controller which gives the path correction values to be displayed on the screen.
To access this remote host, we have to login. The logging can be done by giving the host name. In this
system, our host’s name is “MOTOMAN”.
Once logging is enabled, the data can be logged and saved. The successful data logging of the
proposed system is as shown in Figure 9. In Figure 9, yCorr and zCorr are the path correction values in y and
z direction respectively. These values are calculated by the path correction API mpMeiPutCorrPath(). The
calculations of this API are explained in section 4.5.2. The path correction can be seen in this video:
https://drive.google.com/file/d/1GlqTZnmFV4kT5RN_W7slwioIn40Bbdii/view?usp=drivesdk
IAES Int J Rob & Autom, Vol. 11, No. 3, September 2022: 196-204
IAES Int J Rob & Autom ISSN: 2722-2586 203
In this video, first the robot follows the taught line of the welding path. In the second part (i.e., at 38
secs), let us consider there is deviation in the welding line. The laser sensor continuously scans and follows
the welding line by getting the path correction values simultaneously from the robot controller. Thus, the real
time tracking is made automatic and continuous.
ACKNOWLEDGEMENTS
I would take this opportunity to express my sincere thanks to my guide Dr. M. V. Achutha,
Professor and Dean, Department of Mechanical Engineering, N.I.E., Mysore for his warm support for
project. I would like to thank all my professors and faculty members from N.I.E., Mysuru, for their support
and encouragement for the project.
REFERENCES
[1] J. J. Craig, Introduction to robotics: Mechanics and control. Boston: Addison Wesley Publishing Company, Inc.
[2] J.-Y. Yu and S.-J. Na, “A study on vision sensors for seam tracking of height-varying weldment. Part 2: Applications,”
Mechatronics, vol. 8, no. 1, pp. 21–36, Feb. 1998, doi: 10.1016/S0957-4158(97)00024-X.
[3] A. De and D. Parle, “Real time seam tracking system for automated fusion arc welding,” Science and Technology of Welding and
Joining, vol. 8, no. 5, pp. 340–346, Oct. 2003, doi: 10.1179/136217103225005471.
[4] P. Kiddee, Z. Fang, and M. Tan, “Visual recognition of the initial and end points of lap joint for welding robots,” in 2014 IEEE
International Conference on Information and Automation (ICIA), Jul. 2014, pp. 513–518, doi: 10.1109/ICInfA.2014.6932709.
[5] M. Kong, F. H. Shi, S. B. Chen, and T. Lin, “Recognition of the initial position of weld based on the corner detection for welding
robot in global environment,” in Robotic Welding, Intelligence and Automation, Berlin, Heidelberg: Springer Berlin Heidelberg,
2007, pp. 249–255.
[6] X. Chen, S. Chen, T. Lin, and Y. Lei, “Practical method to locate the initial weld position using visual technology,” The
International Journal of Advanced Manufacturing Technology, vol. 30, no. 7–8, pp. 663–668, Oct. 2006, doi: 10.1007/s00170-
005-0104-z.
[7] Y. Xu, H. Yu, J. Zhong, T. Lin, and S. Chen, “Real-time seam tracking control technology during welding robot GTAW process
based on passive vision sensor,” Journal of Materials Processing Technology, vol. 212, no. 8, pp. 1654–1662, Aug. 2012, doi:
10.1016/j.jmatprotec.2012.03.007.
[8] G. Michalos, S. Makris, A. Eytan, S. Matthaiakis, and G. Chryssolouris, “Robot path correction using stereo vision system,”
Procedia CIRP, vol. 3, pp. 352–357, 2012, doi: 10.1016/j.procir.2012.07.061.
[9] M. de Graaf, R. Aarts, B. Jonker, and J. Meijer, “Real-time seam tracking for robotic laser welding using trajectory-based
control,” Control Engineering Practice, vol. 18, no. 8, pp. 944–953, Aug. 2010, doi: 10.1016/j.conengprac.2010.04.001.
[10] Y. Huang, X. Gao, D. You, and Z. Li, “Investigation of Laser Welding Seam Tracking Based on Visual Sensing,” 2012, doi:
10.2991/emeit.2012.263.
[11] “No Title.” https://www.motoman.com/en-us/products/controllers/yrc1000 (accessed Jul. 31, 2021).
[12] “No Title.” https://kmlsensors.com/catalogs (accessed Jul. 31, 2021).
[13] M. Sulaiman, H. N. M. Shah, M. H. Harun, and M. N. F. M. Kazim, “Defect inspection system for shape-based matching using
two cameras,” Journal of Theoretical and Applied Information Technology (JATIT), vol. 61, no. 2, pp. 288–297, 2014.
[14] Z. Fang, W. Weng, W. Wang, C. Zhang, and G. Yang, “A vision-based robotic laser welding system for insulated mugs with
fuzzy seam tracking control,” Symmetry, vol. 11, no. 11, Nov. 2019, doi: 10.3390/sym11111385.
[15] X. Gao, C. Zhao, T. Bai, and D. Ding, “Application of fourier transform in extraction of molten pool image character,”
Transactions of the China Welding Institution, vol. 2, pp. 1–19, 2006.
[16] J.-Y. Yu and S.-J. Na, “A study on vision sensors for seam tracking of height-varying weldment. Part 1: Mathematical model,”
Mechatronics, vol. 7, no. 7, pp. 599–612, Oct. 1997, doi: 10.1016/S0957-4158(97)00023-8.
[17] “Socket Programming in C/C++.” https://www.geeksforgeeks.org/socket-programming-cc/ (accessed Jul. 31, 2021).
[18] P. Kim, S. Rhee, and C. H. Lee, “Automatic teaching of welding robot for free-formed seam using laser vision sensor,” Optics
and Lasers in Engineering, vol. 31, no. 3, pp. 173–182, Mar. 1999, doi: 10.1016/S0143-8166(99)00024-X.
[19] L. Gao, G. Bo, and Z. Dacheng, “Automation technology and Application of Heilongjiang,” vol. 14, 1995.
[20] D. Baek, H. S. Moon, and S.-H. Park, “Development of an automatic orbital welding system with robust weaving width control
and a seam-tracking function for narrow grooves,” The International Journal of Advanced Manufacturing Technology, vol. 93,
no. 1–4, pp. 767–777, Oct. 2017, doi: 10.1007/s00170-017-0562-0.
[21] F. Rotaru, S. Bejinariu, C. D. Niţâ, R. Luca, I. Pâvâloi, and C. Lazâr, “Robot vision application based on complex 3D pose
computation,” International Journal of Mathematical and Computational Sciences, vol. 3, no. 10, pp. 837–840, 2009, doi:
10.5281/zenodo.1328126.
Development and integration of laser sensor tracking system in robotic arm ... (Tejaswini P.)
204 ISSN: 2722-2586
[22] Z. Fang, D. Xu, and M. Tan, “Visual seam tracking system for butt weld of thin plate,” The International Journal of Advanced
Manufacturing Technology, vol. 49, no. 5–8, pp. 519–526, Jul. 2010, doi: 10.1007/s00170-009-2421-0.
[23] P. Kiddee, Z. Fang, and M. Tan, “A real-time and robust feature detection method using hierarchical strategy and modified
Kalman filter for thick plate seam tracking,” International Journal of Automation and Control, vol. 11, no. 4, 2017, doi:
10.1504/IJAAC.2017.087054.
[24] K.-Y. Bae, T.-H. Lee, and K.-C. Ahn, “An optical sensing system for seam tracking and weld pool control in gas metal arc
welding of steel pipe,” Journal of Materials Processing Technology, vol. 120, no. 1–3, pp. 458–465, Jan. 2002, doi:
10.1016/S0924-0136(01)01216-X.
BIOGRAPHIES OF AUTHORS
IAES Int J Rob & Autom, Vol. 11, No. 3, September 2022: 196-204