Multi-Sensor Measurement System for Robotic Drilling

Multi-Sensor Measurement System for Robotic Drilling

Andreas Frommknecht1*, Jens Kuehnle1, Sergej Pidan2, and Ira Effenberger1

1 2
Department Machine Vision and R&D
Signal Processing BA Assembly & Turnkey Systems GmbH
Fraunhofer Institute for Manufacturing Bietigheim-Bissingen, Baden-Württemberg, D-74321,
Engineering and Automation IPA Germany
Stuttgart, Baden-Württemberg, D-70569, Germany

ABSTRACT

A multi-sensor measurement system for robotic drilling is presented. The system enables a robot to measure its 6D
pose with respect to the work piece and to establish a reference coordinate system for drilling. The robot
approaches the drill point and performs an orthogonal alignment with the work piece. Although the measurement
systems are readily capable of achieving high position accuracy and low deviation to perpendicularity,
experiments show that inaccuracies in the robot’s 6D-pose and especially slippage on the work piece when
exerting clamping force considerably impact the results. With the current robotic drilling system an average
position deviation of 0.285 mm and an average deviation to perpendicularity of 0.27° are achieved.

1. INTRODUCTION

Robotic drilling is the process in which a robot is equipped with an end effector able to drill holes. Due to
inaccuracies in the robot’s 6D-pose, achieving high accuracy drill holes is a challenge. Compared with processing
machines and manually drilled holes, robotic drilling is much more flexible with respect to different kinds of work
pieces and geometries. In addition, robotic drilling is much more adaptable than processing machines and much faster
than manual drilling. Robotic drilling is and can be used in different fields of production e.g. automotive and aerospace.
Especially in the aerospace industry the requirements for the position and the orthogonality of drilled holes are high [1].
A new approach for referencing and orthogonal alignment for high accuracy requirements will be outlined in this paper.
Especially for curved parts orthogonality is very important but difficult to achieve. Different approaches for robotic
orthogonal alignment are currently used in the industry. Carlos et al. [2] describes a movable clamp nose whose
deviation is measured by four inductive sensors. With this method a deviation to perpendicularity of ±0.5° is achieved.
A method for freeform surfaces is described by Lee [3]. Here five light laser spots are used to calculate the
orthogonality. With this approach a mean angular deviation of 0.846° degrees is achieved.
There are two different ways to reference the position of the work piece: external or internal measurement systems.
External measurement systems are separated from the robot and use added targets e.g. reflectors on the work piece for
measurement. There are different kinds of external measurement systems on the market: indoor GPS (iGPS), laser
tracker or camera-based photogrammetry [4]. These external systems make it very easy to reference the part position in
the working area, because they are able to track the part position within high accuracies. The disadvantage of these
methods is that targets have to be fixed on the work piece and need to be visible.
Internal measurement systems are on-board of the end effector. So they are carried along by the robot, which means
that the referencing process depends on the robot accuracy. The internal measurement is not able to increase the robot
accuracy. It can only ensure that negligible errors are produced by the system. So for the referencing process the highest
possible robot accuracy can be used. If targets on the work piece are necessary depends on the kind of internal
measurement system. In the following a system will be presented, which uses high accuracy drill holes as reference
features, so no special kind of targets have to be added.
Referencing is the first step in the drilling process and has an enormous influence on the final quality of the drill
holes. Typical values in aircraft industry for robot drill holes are in the range of ±0.5 mm [5].

* Corresponding author: Tel.: (0049) 970-1818; Fax: (0049) 970-1004; E-mail: [email protected]

398
 Flexible Automation and Intelligent Manufacturing, FAIM2015, Wolverhampton, UK

2. COMPONENTS OF THE MULTI-SENOR MEASUREMENT SYSTEM

The robotic drilling cell consists of an industrial robot KUKA KR210 with a maximum reach of 2901 mm and a
maximum payload of 210 kg. Its repeatability is specified as ±0.06 mm. The robot is equipped with a tool changer
carrying an end effector with a drilling spindle (see Figure 1). The drilling spindle is mounted on linear actuators which
move the spindle either along the drilling axis (along the z axis) or across the end effector frame (along the y axis) e.g.
in order to place measurement systems or other tools. Contact with the work piece is established by a clamp nose with
a three-point bearing mounted on a linear actuator with 24 mm stroke length.
All linear actuators can be moved accurately (<0.02 mm) using the absolute encoders of their motors. The linear
actuators of the drilling spindle along z and the clamp nose are equipped with force sensors which measure the pressure
applied to the work piece e.g. in order to avoid deformation of the work piece. Metal chips and excess lubricants are
extracted by a suction device mounted to the clamp nose.
The following paragraphs describe the components of the multi-sensor measurement system for referencing and
orthogonal alignment mounted on the robot end effector.

Figure 1: Design of the end effector (source Lübbering GmbH)

2.1. 2D CAMERA SYSTEM

As the drilling axis defines the main reference axis, a 5 mega pixel camera (Basler acA2500-14gm) is mounted to
the side of the spindle (see Figure 1). The camera can be accurately moved by means of the linear actuators of the
spindle along the y axis in order to align it with the drilling axis and along the z axis in order to adjust the depth of focus.
There are two kinds of illumination. A bright field coaxial illumination unit mounted to the camera lens (Opto GmbH)
and a custom designed dark field illumination unit integrated into the clamp nose (Lübbering GmbH). To protect the
camera system from metal chips, dust and lubricants it is enclosed in a custom designed protective housing.
Since the camera takes images through the clamp nose, the measurement area is limited to 25 mm x 25 mm resulting
in an effective resolution of approximate 12.5 µm per Pixel. The standoff distance is 360 mm, and the focal length is
approximately 60 mm.

399
Multi-Sensor Measurement System for Robotic Drilling

2.2. LASER DISTANCE SENSORS

Three laser distance sensors (Sick OD Mini) with a measurement range of 100 mm and a resolution of 20 µm are
used. These sensors are arranged around the contact tool and the field of view is in the working area (see Figure 2).
They are actually used to check if the work piece is at the expected position. This is especially important in the
referencing process, because at this moment the system by itself doesn’t know anything about the real position of the
work piece. At the beginning of the referencing process the system expects that the work piece is positioned according
to defined tolerances of the setting. In the drilling process itself the values of sensors can be used for a first orthogonal
alignment. If this step is performed depends on its influence on the position accuracy of the robot (repeatability verses
absolute accuracy).

Figure 2: Laser distance sensors arranged around working tool (source: Lübbering GmbH)

2.3. CONTACTLESS HIGH ACCURACY DISTANCE MEASUREMENT SENSORS

In contrast to the state of the art, contactless but not optical high accuracy sensors are used. So on one hand their
independency from the surface is better than with optical sensor and on the other hand it’s not necessary to get in
contact with the part like with a moveable clamp nose. So the part by itself is not affected by the measurement. These
sensors offer a high accuracy of 10 µm and have a measurement range of 3 mm. They are directly built in the clamp
nose of the end effector. The angular distance between neighbouring sensors is 90° (see Figure 3).

Figure 3: Contactless high accuracy distance sensors (blue) built in clamp nose (source: Lübbering GmbH)

3. MEASUREMENT TASKS

The multi-sensor measurement system performs two tasks: referencing of the work piece and orthogonal alignment
with the work piece. Since it requires the 6D pose of the robot for achieving these tasks (e.g. as provided by the robot
controller or an external measurement system), it cannot be used to improve the 6D pose accuracy of the robot.

400
 Flexible Automation and Intelligent Manufacturing, FAIM2015, Wolverhampton, UK

The measurements are processed as follows: The robot controller loads the drilling program which consists of a list
of drill holes. Each drill hole is specified with respect to a certain reference coordinate system. The reference
coordinate systems are defined by means of reference features on the work piece. Paragraph 3.2 describes the
procedures used to measure reference features and to establish reference coordinate systems for drilling.
At each drill point, the angle between the drill axis and the surface normal is computed and the robots position is
iteratively corrected. Paragraph 3.1 describes the procedures used to measure the deviation to perpendicularity.
In order to establish contact with the work piece, the robot maintains its position and the clamp nose is actuated until
enough clamping force is applied on the work piece. No additional mechanical fixture is used, so slippage cannot be
completely avoided.
Whenever the robot approaches the work piece, the laser distance sensors are used for collision avoidance. The
distance to the work piece is estimated and an advance point for the robot at a secure distance to the work piece is
computed. A possible advance point can e.g. be chosen such that the work piece is within the measurement range of the
high accuracy distance sensors. At the advance point the multi-sensor measurement system is used for referencing (see
paragraph 3.2) and for orthogonal alignment (see paragraph 3.1).
In case the accuracy of the 6D-pose of the robot is not sufficient, either a laser tracker or an automatic pose
optimization model [6] is used to boost the accuracy.

3.1. ORTHOGONAL ALIGNMENT

The orthogonal alignment is done by using the distance measurement sensors and an approximated surface plane.
No external measurement system is needed. The centre of rotation is the computed intersection point between the drill
axis and the approximated surface plane. Alternatively the expected drill point given by e.g. the corresponding CAD
can be used, so the point keeps constant during iterations, in contrast to the before mentioned intersection point. The
robot iteratively adjusts its pose until the drill axis and normal of the approximated plane are within tolerance or the
maximum number of iteration steps is reached. Which of the end criteria occurs first depends once again on the
accuracy of the robot.
The approximated plane is calculated out of the sensors measurement points. The position of each sensor in the
clamp nose can be taken out of the end effector CAD or out of a sensor position calibration step. The sensors
themselves measure the distance to the surface of the work piece. Out of this information a measurement point on the
surface can be calculated for every sensor. The numbering, for sensors and the measurement points: P1, P2, P3 and P4,
is anti-clockwise, as shown in Figure 4. So the opposing points are P1 and P3 or respectively P2 and P4. Now two
vectors P1P3 and P2P4 are calculated. The cross product of these two vectors gives the normal of the approximated
surface. This method ensures, apart from measurement or converting inaccuracy, the exact orthogonal vector for plane,
cylindrical or spherical parts. The next step consists of calculating the angle between the approximated surface normal
and the spindle axis. With the used distance sensors an angular resolution of 0.081° is achieved. The before mentioned
intersection point and the cross product of the approximated surface normal and the direction of the spindle axis define
a rotation axis. The clamp nose and the whole end effector is turned around this axis by the before calculated angle.
These steps can, as described above, be repeated till one of the two mentioned ending criteria occurs.

Normal

P1 P4

P3
P2

Figure 4: Calculation of approximated normal out of measurement points

401
Multi-Sensor Measurement System for Robotic Drilling

3.2. REFERENCING

The coarse position and orientation of the work piece and the robot with respect to a common reference frame is
assumed to be known and thus, the coordinates of the actual reference features are assumed to be within ±10 mm of the
nominal coordinates. Subject to these assumptions the robot is able to find the reference features on the work piece.
Several reference features are measured to establish a reference coordinate system. High accuracy drill holes on
planar surface patches are used, but other features are possible as well (e.g. riveted bolts, material edges). A minimum
of three reference drill holes are required to define a reference coordinate system, but more can be used to mitigate
measurement errors, especially 6D-pose inaccuracies.
In order to measure a reference drill hole, the robot approaches the work piece at its nominal coordinates. The laser
distance sensors check, if the work piece is at the expected position (see paragraph 2.2). After that the clamp nose
moves towards the work piece until the contactless high accuracy distance measurement sensors (see paragraph 2.3)
are in range. The actual surface around the reference drill hole is approximated by a plane. The clamp nose moves
backwards and repositions the dark field illumination unit for image acquisition. In addition, the linear actuators of the
spindle move the camera to look through the clamp nose and to focus on the reference drill hole. The camera acquires
an image and the centre of the hole is detected. The corresponding line of sight is intersected with the approximated
plane of the surface resulting in the actual coordinates of the reference drill hole. In doing so, the offsets of all linear
actuators are accounted for. Using the 6D-poses of the robot during all these measurements, the coordinates are
transformed to a common reference frame. These steps are repeated for each reference drill hole and a reference
coordinate system is computed following predefined rules.
The described method does not require the use of an external measurement system. If however the 6D-pose
accuracy of the robot is not sufficient, an external measurement system can be used for accuracy improvement.

4. SYSTEMS CALIBRATION

The calibration of the measurement systems is performed in several steps. Each sensor is calibrated individually in
order to ensure that all sensors deliver metric data (intrinsic calibration). The displacement transducers and laser
distance sensors are factory calibrated. The perspective camera is calibrated using a standard camera model (with
5 distortion parameters). For the calibration a factory calibrated laser tracker is used. The linear actuators are aligned
on a coordinate measuring machine (CMM) with angular misalignments <0.1°.
Using a custom designed calibration standard (see Figure 5) that connects to the spindle via the hollow shank tool
connector (HSK) all sensors are simultaneously calibrated with respect to a common coordinate system defined by the
calibration standard (extrinsic calibration). The calibration standard is designed such that its z axis coincides with the
z axis given by the linear actuators of the spindle and clamp nose, respectively, and its origin defines the tool center
point (TCP). It is manufactured with maximum precision; the shaft wobbles by <0.015 mm and the plate deviates by
<0.05 mm from perpendicularity. Manufacturing inaccuracies are corrected by means of a CMM-measurement.

laser tracker
reflector

calibration pattern

Figure 5: Calibration standard

402
 Flexible Automation and Intelligent Manufacturing, FAIM2015, Wolverhampton, UK

The calibration standard is equipped with several calibration fixtures for the different sensors. In order to perform
extrinsic calibration the calibration standard is gradually turned by 360° while measurements are taken simultaneously
with all for the calibration necessary sensors (2D camera, distance sensors and laser tracker). These measurements are
evaluated as follows:
The calibration plate is printed with a coded calibration pattern within the field of view of the 2D camera. The bright
field coaxial light unit is used to illuminate the calibration pattern. For each image, the extrinsic transformation of the
camera with respect to the common coordinate system is estimated. The projection centers corresponding to these
transformations define a 3D-ellipse which can be determined by best-fit; the difference between the semiaxes
resembles the deviation due to wobbling. By compensating this 3D-ellipse, an average extrinsic transformation of the
camera with respect to the common coordinate system defined by the calibration standard is obtained.
The calibration plate is also equipped with a reflector for the laser tracker. By turning the calibration standard, the
laser tracker measurements determine a 3D-circle in the coordinate system of the laser tracker. Its normal direction
defines the spindle axis, and its center defines the origin of the common coordinate system, i.e. the TCP.
In order to align the y axis of the calibration standard with the y axis given by the linear actuator of the spindle, a test
drill is performed. For the camera to take an image of the drill hole through the clamp nose, the camera is actuated along
the y axis of the spindle. The deviation of the actual coordinates of the drill hole and the nominal coordinates of the TCP
is used to compute the misalignment of the y axis of the calibration standard.

5. RESULTS

Several experiments with the measurement systems and the robotic drilling cell were conducted. For this purpose,
an experimental setup was chosen that allows a systematic study of the drilling process in different configurations of the
robot end effector (see Figure 6). The mounting fixture is an aluminium plate (approximate 2000 mm x 400 mm)
having several high accuracy drill holes on planar surface patches as reference features (see Figure 7). The mounting
fixture can hold four plates with different curvatures representing a work piece (planar, 100 mm, 200 mm and 500 mm
curvature radius) and of different materials (aluminium, titanium, carbon fibre reinforced plastic and stacks thereof).
The actual drill holes were measured with a CMM and compared to the nominal positions (x, y coordinates). A
position deviation <0.1 mm and a perpendicularity deviation <0.2° was intended.

Figure 6: Experimental setup

403
Multi-Sensor Measurement System for Robotic Drilling

Figure 7: Mounting fixture with high accuracy drill holes A1-A23 on planar surface patches as reference features

5.1. POSITION ACCURACY

With the calibration procedure described in chapter 4, a referencing error <0.015 mm for high accuracy drill holes
on planar surface patches as reference features has been expected. The average deviation of a representative test with
100 drill holes was 0.285 mm (max. 0.651 mm). This test has been done without using an external measurement system
for pose accuracy improvement. So initially, it has been assumed the error is entirely due to the robot’s 6D-pose
inaccuracies. A more detailed error analysis however revealed that the clamp nose slips when exerting clamping force
on the test plate (about 400 N). Nevertheless the results are, compared to the typical position accuracy of ±0.5 mm in
aircraft industry, mentioned in section 1, competitive. Furthermore shows the above remarked very low referencing
error the high optimization potential of the presented approach.

5.2. ORTHOGONAL ACCURACY

The performed tests have shown a maximal orthogonal accuracy error of 0.48° (entry 12 in Table 1) and an average
accuracy failure of 0.27° (considering the mean value of all in Table 1 listed deviations), for the whole process . The
listed values have been calculated for a cylindrical test part with a radius of curvature of 200 mm (see Figure 8). Whole
process means that the drilled holes have been measured on a coordinate measurement machine and their axis failure
has been calculated. So the failure of the drilling process, the robot inaccuracy and the above mentioned slipping during
clamping affects the mentioned results. For the tests furthermore a maximum number has been set of three iterations.
Assumed the avoiding of slipping during clamping and a higher number of iterations or respectively a better accuracy
of the robot positioning, an orthogonal failure below 0.2° is achievable. Compared e.g. to the value of ±0.5° of the
moveable clamp nose system mentioned in section 1 a considerable accuracy improvement is possible. Final tests have
to be performed to prove this.

Table 1: Deviation of drill hole axis to surface normal, measured with coordinate measurement machine

ID Deviation [deg] ID Deviation [deg]

1 0.28 10 0.20
2 0.26 11 0.15
3 0.31 12 0.48
4 0.27 13 0.20
5 0.24 14 0.20
6 0.30 15 0.26
7 0.34 16 0.40
8 0.31 17 0.11
9 0.27 18 0.27

404
 Flexible Automation and Intelligent Manufacturing, FAIM2015, Wolverhampton, UK

Figure 8: Test parts with 200 mm curvature radius

6. CONCLUSION

A multi-sensor measurement system for referencing and orthogonality alignment for robotic drilling has been
developed. It enables the robot to accurately measure the position and the orientation of the work piece and establish a
reference frame for drilling. Due to the precise calibration and the measurement resolution, the errors introduced by the
multi-sensor measurement system are negligible, and the accuracy of the drilling process is only determined by the
6D-pose accuracy of the robot. So the measurement system creates the condition to achieve the highest possible
accuracy of the robotic drilling process. With increased 6D-pose accuracy of the robot and avoidance of clamp nose
slippage, a position deviation <0.1 mm and a perpendicularity deviation <0.2° is within reach. This fulfills the high
requirements of the aerospace industry.

ACKNOWLEDGEMENTS

The presented system has been developed and implemented in collaboration with Broetje-Automation GmbH and
Lübbering GmbH. The authors would like to thank our partners for the excellent cooperation during the project.

REFERENCES

[1] T. Olsson, M. Haage, H. Kihlman, R. Johansson, K. Nilsson, A. Robertsson, M. Björkman, R. Isaksson, G. Ossbahr and T.
Brogårdh, “Cost-efficient drilling using industrial robots with high-bandwidth force feedback”, Robotics and
Computer-Integrated Manufacturing, Vol.26, No. 1, pp. 24–38, 2010
[2] C. C. A. Eguti, G. Trabasso, E. Villani, G. K. Coracini and L. F. F. Furtado, “Development of a robotic end-effector of
drilling and fasteners inserter for aircraft structures”, SAE 2012 Aerospace Manufacturing and Automated Fastening
Conference & Exhibition, Fort Worth, Texas, United States, 2012
[3] R.-T. Lee and F.-J. Shiou, “Multi-beam laser probe for measuring position and orientation of freeform surface”,
Measurement, Vol.44, No. 1, pp. 1-10, 2011
[4] C. H. Cheng, D. Huo, X. Zhang, W. Dai and P. G. Maropoulos, “Large Volume Metrology Process Model: Measurability
Analysis with Integration of Metrology Classification Model and Feature-Based Selection Model”, Proceedings of the 6th
CIRP-Sponsored International Conference on Digital Enterprise, Vol.66, Springer-Verlag Berlin Heidelberg, pp.
1013–1026, 2010
[5] W. Zhu, W. Qu, L. Cao, D. Yang and Y. Ke, “An off-line programming system for robotic drilling in aerospace
manufacturing”, The International Journal of Advanced Manufacturing Technology, Vol. 68, No. 9-12, pp 2535-2545, 2013
[6] U. Schneider, J. R. Diaz Posada, and A. Verl, “Automatic Pose Optimization for Robotic Processes”, IEEE International
Conference on Robotics and Automation (ICRA) 2015, Seattle, Washington, USA, 2015

405