Available online at www.sciencedirect.

com

Computers & Industrial Engineering 55 (2008) 311320 www.elsevier.com/locate/caie

Reverse engineering of a symmetric object

Abstract This paper presents a procedure to nd an approximate symmetry plane from a point cloud for the reverse engineering of a symmetric object. Even if, the point cloud acquired from a symmetric object does not have exact symmetry planes in the mathematical sense, the data usually carries adequate symmetric information for reverse engineering. To generate a CAD model satisfying the original aesthetic and functional design, it is essential to extract approximate symmetry planes. This paper proposes a new procedure to extract a symmetry plane, and it includes a registration step, aligning three-dimensional models. To improve the computational eciency of the proposed procedure, the registration step has been modied by using the inherent attributes of the problem: (1) the measured point data may include noise and (2) the point cloud might be partially asymmetric. The proposed procedure has been implemented and proven to be ecient enough to handle very heavy data sets including millions of entities (points and triangles). 2007 Elsevier Ltd. All rights reserved.
Keywords: Symmetry plane; Reverse engineering; Segmentation; Registration; Approximate symmetry plane

1. Introduction While conventional engineering creates a CAD model based on the functional specications of a new product, reverse engineering uses a manufactured part to produce a CAD model (Ke et al., 2006; Lee & Woo, 2000; Varady, Martin, & Cox, 1997; Xinmin, Zhongqin, Tian, & Ziping, 2001). The existence of a CAD model provides enormous gains in improving the quality and eciency of an analysis, because we can exploit the advantages of the extensive use of CAD/CAM technologies. Reverse engineering typically starts with measuring a physical object to reconstruct a CAD model for applications like redesign, reproduction and quality control. The owchart shown in Fig. 1 characterizes the typical procedure of reverse engineering, which consists of ve steps: (1) data acquisition, (2) preprocessing (noise ltering and merging), (3) triangulation, (4) feature extraction, and (5) segmentation and surface tting. The most critical step of reverse engineering is the segmentation
*

Corresponding author. Fax: +82 31 219 1610. E-mail address: [email protected] (S.C. Park).

0360-8352/$ - see front matter 2007 Elsevier Ltd. All rights reserved. doi:10.1016/j.cie.2007.12.015

312

M. Chang, S.C. Park / Computers & Industrial Engineering 55 (2008) 311320

1. Data acquisition (Laser scanner)

2. Preprocessing (Noise filtering & Merging)

Point cloud

3. Triangulation Triangular mesh 4. Feature extraction (Symmetry plane)

5. Segmentation & Surface fitting

CAD model
Fig. 1. Typical ow of reverse engineering.

process of the fth step. The segmentation process splits a triangular mesh into sub-meshes to which an appropriate single surface can be tted, and it seriously aects the quality of the resulting CAD model. To improve the quality of segmentation, it is essential to make use of the features (sharp edges and symmetry planes) extracted in the fourth step. Among various features, a symmetry plane is a very important feature for the segmentation process (Langbein, Marshall, & Martin, 2004; Momi et al., 2006; Tate & Jared, 2003), because the symmetric property is the most general feature of industrial products. This paper mainly focuses on how to nd an approximate symmetry plane from a triangular mesh that is generated by processing a point cloud. The concept of symmetry has been of major interest in many elds of research over many years. Especially, the problem of determining exact symmetry of a solid model has been well studied for several decades (Jiang & Bunke, 1992; Sugihara, 1984; Wolter, Woo, & Volz, 1985). Previous results showed that polyhedral symmetry can be determined in three dimensions in O(n log n) time (where n is the number of vertices) by the use of symbolic sorting algorithms. However, the algorithms for determining exact symmetry can not be applied to approximate cases, because they require the ability to tell locally whether one point can be mapped to another, which is not true for a point cloud with various noises. Although, a point cloud acquired from a symmetric object does not have exact symmetry planes in the mathematical sense, the data usually carries adequate symmetric information for reverse engineering. To generate a CAD model satisfying the original aesthetic and functional design, it is essential to extract approximate symmetry planes. There have been various previous results in determining the approximate symmetry planes. Fukushima and Kikuchi (2006) proposed an articial neural network that extracts symmetry axes from visual patterns. One of the attributes of this method is the use of blur for extracting a symmetry axis to reduce the computational cost. Tuzikov, Colliot, and Bloch (2003) proposed an algorithm to detect the best symmetry plane in 3D magnetic resonance (MR) brain images. The symmetry plane is initialized by a plane obtained from principal inertia axes and is optimized by using the downhill simplex method. Although the algorithm is stable with respect to noise and intensity nonuniformity, it is sensitive to the position of the initial plane in the optimization procedure. Mills, Langbeing, Marshall, and Martin (2001) proposed an algorithm for nding symmetry planes consisting of two steps. The rst step of the algorithm replaces certain groups of points by their centroid, and the second step nds the approximate symmetries of the collection of points remaining after this grouping. Ke et al. (2006) proposed an algorithm to extract a symmetry plane from a point cloud.

M. Chang, S.C. Park / Computers & Industrial Engineering 55 (2008) 311320

313

The method is based on the iterative closest point (ICP) algorithm which is a data registration technique, and requires a user input of choosing two subsets, A and B in the point cloud that possess an approximate symmetric property. Although, this is a simple and robust method, there are two potential problems: (1) the accuracy of the symmetry plane might be sensitive to the manual input of choosing two subsets in the point cloud and (2) when the ICP algorithm is applied to two point clouds (subsets), the performance of the registration is sensitive to the point distributions of the two point clouds (i.e., the point distribution should be very dense and even). The focus of this paper is to nd an approximate symmetry plane from a triangular mesh that is generated by processing a point cloud. The proposed algorithm expects a triangular mesh instead of a point cloud, and this is unproblematic because there are many available algorithms with which to triangulate a point cloud. For example, a point cloud is obtained by measuring a physical object having a symmetry plane. A triangular mesh can be obtained by processing the point cloud, but the obtained mesh is not perfectly symmetrical because of various inaccuracies resulting from sensing errors arising during data acquisition as well as approximation and numerical errors in the reconstruction process. In this case, nding the approximate symmetry plane is very important for the segmentation process, because the resulting CAD model can be improved by adjusting the approximate symmetry plane so that it becomes exactly symmetrical. As shown in Fig. 2, we can perform the surface tting process to only the left half portion of the mesh, and can complete a CAD model by the mirroring operation. The objective of the paper is to demonstrate an algorithm to extract an approximate symmetry plane from a given triangular mesh. The proposed procedure includes a registration operation, aligning threedimensional models (a point cloud and a triangular mesh). For the registration operation, we adapt the ICP technique to the inherent attributes of the problem of the symmetry plane extraction. The remainder of this paper is organized as follows. Section 2 addresses the approach of this paper to the extraction of an approximate symmetry plane from a triangular mesh, and Section 3 provides a detailed description of the proposed algorithm including the modied ICP technique. Finally, concluding remarks are presented in Section 4.

Fig. 2. Object with a symmetry plane.

314

M. Chang, S.C. Park / Computers & Industrial Engineering 55 (2008) 311320

2. Approach to extract symmetry plane This section presents the approach of this paper to identify an approximate symmetry plane from a triangular mesh. The behind idea of the proposed approach is that it is very easy for a user to make a rough estimate of a symmetry plane from a point cloud. We call the rough estimate of a symmetry plane an initial symmetry plane. Although the initial symmetry plane provides a good approximate symmetry, it is not accurate enough for the reverse engineering purpose. To nd an accurate symmetry plane, the proposed approach requires a couple of additional steps, including mirroring and registration operations. Since those operations are simple and well-known, we can easily implement the proposed approach. Fig. 3 shows the overall procedure. The proposed algorithm can be informally specied as follows. 2.1. Symmetry plane extraction  Input: a point cloud (PC), and a triangular mesh (TM) generated by processing PC (Fig. 3(a)), an initial symmetry plane (IP) specied by the user.  Output: an approximate symmetry plane (SP). The proposed algorithm consists of four major steps: (1) construct an initial symmetry plane by the users input (Fig. 3(b)), (2) obtain PCM by mirroring the original point cloud (PC) with respect to the initial symmetry plane (Fig. 3(c)), (3) register PCM to the triangular mesh by using the iterative closest point (ICP) algorithm (Fig. 3(d)), and (4) compute the approximate symmetry plane (SP) by making use of the point mapping relations between PC and PCM (Fig. 3(e)). Conceptually, the proposed algorithm is simple and intuitive. The rst step requires an initial symmetry plane specied by the user, but this initial symmetry plane does not have to be completely accurate, as shown
P2 P1 P6 P11 TM: Triangular mesh

PC: Point cloud

(a) Preparation of a triangular mesh

PC TM

IP: Initial symmetry plane

SP1 P11 TM PCM P2 P1

(b) Initial symmetry plane by user

(c) PCM by mirroring PC with respect to IP

PCM TM

P11 PC

(d) Register PCM to TM

(e) Computing an approximate symmetry plane

Fig. 3. Overall procedure of the proposed algorithm.

M. Chang, S.C. Park / Computers & Industrial Engineering 55 (2008) 311320

315

in Fig. 3(b). The second step, the mirroring operation, is a simple transformation of a point cloud, and the third step can use the well-known ICP algorithm. The fourth step is also quite simple. Each point (Pi) of PC has a corresponding point P0i in PCM. If the point cloud (PC) is exactly symmetrical, then there should exist a symmetry plane that contains all the mid points of Pi and P0i , where 1 6 i 6 number of points of PC. Since the point cloud is approximately symmetrical, the fourth step involves determining a symmetry plane by tting the mid points of Pi and P0i with a plane. Although the proposed procedure is conceptually simple, there are two potential problems to be solved for its successful implementation, and both problems are related to the ICP algorithm of the third step: (1) the ICP algorithm may take too much computational time when it has to deal with a large point cloud that includes millions of points and (2) the ICP algorithm may inaccurately estimate a pose (a local minimum), especially in the presence of noise in the point cloud. For practical implementation of the proposed procedure, it is necessary to modify the ICP algorithm so that it can deal with these potential problems. A detailed explanation of the proposed algorithm is given in the next section. 3. Symmetry plane extraction For the successful implementation of the proposed symmetry plane extraction procedure, it is necessary to consider couple of issues. The rst issue is how to modify the ICP algorithm for the purpose of symmetry plane extraction. The ICP algorithm is a widely used method for aligning three-dimensional models when an initial estimate of the relative pose is known. The proposed procedure for the symmetry plane extraction employs the ICP algorithm to move (register) a point cloud PC to be in optimal alignment with a triangular mesh TM. The ICP algorithms starts with two data sets (a point cloud and a triangular mesh) and an initial estimate of the aligning rigid-body transform. It then iteratively renes the transform by alternately choosing corresponding points in the two data sets and nding the best translation and rotation that minimizes an error metric based on the distance between them. The quality of alignment depends

Point Cloud PC = {Pi} Point Cloud PC = {Pi} Triangular Mesh TM = {Tj}

Triangular Mesh TM = {Tj} Point Sampling PC' = {Pk'}

Compute the closest point for each Pi Compute the registration Apply the registration to PC Compute mean distance error (e) between PC and TM e < given threshold (t)
Yes No

Compute the closest point for each Pk' Noise filtering Compute the registration Apply the registration to PC & PC' Compute mean distance error (e) between PC' and TM e < given threshold (t)
Yes Data flow Algorithm flow No

End End

(a) ICP algorithm

(b) Adapted ICP algorithm

Fig. 4. Typical procedure of the ICP algorithm.

316

M. Chang, S.C. Park / Computers & Industrial Engineering 55 (2008) 311320

heavily on choosing good pairs of corresponding points in the two models. Fig. 4(a) shows the overall procedure of the ICP algorithm, and the details of the procedure can be found in Tao and Jiyong (2007), Desl and McKay (1992). Although the original ICP algorithm produces reasonably good results, there are diculties in applying the algorithm to heavy data sets that include noise. For heavy data sets that include millions of entities (points and triangles), it is not feasible to apply the original ICP algorithm, because it would require too heavy computational loads. To relieve the computational diculty, it is necessary to modify the ICP algorithm by making use of the inherent attributes of the symmetry plane extraction problem. The modied ICP algorithm should be more ecient and more stable, even in the presence of noise in the data sets. The most time-consuming step of the ICP algorithm, shown in Fig. 4(a), is the rst step, computing the closest point for each P0i . To nd the closest point for a point Pi, it is necessary to compute the distance between Pi and all triangles belonging to the mesh. Intuitively, we can consider the problem of computing the closest point as a set of point-to-triangle distance (PTD) problems. However, this simple approach would be very time-consuming because the worst case time complexity of the step is O(n * m), where n is the number of points of PC and m is the number of triangles of TM. If both n and m are in the millions, then the computation time could be weeks. Although, an optimal PTD algorithm is employed, the rst step of the ICP algorithm would be computationally expensive if the basic PTD algorithm has to be applied too many times. Thus, the key aspect in improving the eciency is to restrict as much as possible when and where the basic PTD algorithm is applied. To reduce the number of PTD operations, it is necessary to reduce n and m, factors of the time complexity O(n * m). To reduce n, our method employs a sampling approach for the point cloud, PC. There are several sampling methods (Rusinkiewicz & Levoy, 2001) for the ICP algorithm, and this paper employs a random sampling method with a dierent sample of points at each iteration. At this time, the sampling ratio should be determined by considering both the accuracy of the alignment and the computation time. From our experience, a reasonable sampling ratio is in the range of [0.10.5%], when the number of points is larger than one million. Although we can reduce n by using the sampling method, the computational burden might be still heavy because of the following reason. For each sampled point P, we need to nd the closest point from TM, and this requires m times of PTD operations where m is the number triangles belonging to TM (m might be millions). As shown in Fig. 5, it is necessary to test all triangles to nd the closest point pairs. To reduce the number of testing triangles, it is necessary to make use of the inherent attributes of the symmetry plane extraction problem. As shown in Fig. 6, we can identify two attributes of the problem: (1) the measured point data may include noise and (2) the point cloud might be partially asymmetric. Since the ICP technique is quite sensitive to noise and asymmetric points, it is necessary to exclude such points from the ICP process. This means that we do not have to use all the sampling points for the ICP algorithm. Since the accuracy of the ICP algorithm is heavily dependent on choosing good pairs of corresponding points in the two models, it is necessary to select good sampling points by ltering out noise and points from asymmetric portions. To do so, we introduce the concept of the acceptance range for ltering sampled points, as shown in Fig. 7. Let us assume that we only accept the closest point pairs (Pi, CPi) with a distance shorter than a certain acceptance range (r). By using the acceptance range, we derive two benets: (1) good pairs of corresponding

C D E

CP

B F

Fig. 5. Finding the closest point pair (P, CP).

M. Chang, S.C. Park / Computers & Industrial Engineering 55 (2008) 311320

317

Fig. 6. Noise and partial asymmetry.

C D E

P r

CP

B F

r: acceptance range
Fig. 7. Restricting search space.

points can be chosen by ltering with the acceptance range and (2) the number of testing triangles can be signicantly reduced by restricting search space. To implement the concept of the acceptance range, our method employs the octree based space partitioning technique (Boada, 2004). Given a triangular mesh, we rst calculate its bounding box, quantize the vertex coordinates of each triangle, and build up an octree structure through recursive 3D space partition with each node in the octree representing a 3D cell. After that, it is necessary to traverse the octree from the root to the lowest level in a top-down manner. During the traversal each 3D cell in the tree front is subdivided into eight child cells with three subdivisions in three directions: one along the X axis, one along the Y axis, and the other along the Z axis. Nonempty cells are recursively subdivided until the nest resolution is reached. Each 3D cell in the tree has the IDs of triangles which are overlapping with the cell. For the eciency of the overlapping test between a 3D cell and a triangle, we use the bounding box of a triangle for the test rather than the triangle itself. Since the bounding box of a triangle may be overlapped with multiple 3D cells, it can be pointed to by multiple nodes. Fig. 8 shows an example of the 2D cell subdivision. For a sampled point P, we can easily identify the corresponding cell, cell_1. However, there is a problem. As shown in Fig. 8, the closest triangle from P is not the triangle A but the triangle D which does not belong to the same cell, cell_1. That means we can not limit the searching space to within the corresponding cell of a given point. This problem can be solved by considering the acceptance range (r) when we evaluate the bounding box of a triangle. Fig. 9 shows the concept of the expanded bounding box. By using the expanded bounding box, we can limit

318

M. Chang, S.C. Park / Computers & Industrial Engineering 55 (2008) 311320

root Cell_1 A P Cell_3

D Cell_1 H A Cell_2 B, E Cell_3 D, H node

B E Cell_2

C Cell_4 F Cell_5

Cell_6 G Cell_7 C Cell_4 F Cell_5 H Cell_6 G Cell_7

Fig. 8. An example of 2D cell subdivision.

r
C C

(a) Bounding box

(b) Expanded bounding box (to consider the acceptance range, r)

Fig. 9. Expanded bounding box.

the searching space within the corresponding cell of a given point. Fig. 10 shows an example of the 2D subdivision by using the expanded bounding boxes. Observe that the triangle D belongs to cell_1, which is the corresponding cell of the point P. For a sampled point, we can easily nd the corresponding 3D cell, and we can nd the closest pair by performing the PTD operations with the triangles belonging to the cell. If the corresponding cell is empty, then we exclude the point from the ICP algorithm. In this way we can naturally avoid the problems of noise and partial

root Cell_1 A P D Cell_1 H B E Cell_2 C Cell_4 F Cell_5 Cell_6 G Cell_7 A, D Cell_2 B, E Cell_3 C, D, H node Cell_3

C, F Cell_4

F Cell_5

H Cell_6

G Cell_7

Fig. 10. Subdivision with expanded bounding boxes.

M. Chang, S.C. Park / Computers & Industrial Engineering 55 (2008) 311320

319

Fig. 11. An example of the symmetry plane extraction.

asymmetry. Most of all, the computation time of the symmetry plane extraction is signicantly reduced. Fig. 11 shows an example of a symmetry plane which extracted by using the proposed algorithm. 4. Summary The most critical part of reverse engineering is the segmentation process, because it seriously aects the quality of the resulting CAD model. To improve the quality of segmentation, it is essential to make use of features (sharp edges and symmetry planes). This paper has presented a procedure to nd an approximate symmetry plane from a triangular mesh that is generated by processing a point cloud. The proposed procedure requires user input of a rough estimate of a symmetry plane from a point cloud. Because of this requirement, the proposed procedure is not fully automatic; however, it can provide more exibility in some situations. If the point cloud includes multiple symmetry planes, the user can be allowed to choose one of them. Once an initial symmetry plane is identied, the user performs a couple of additional steps, including a registration operation, aligning three-dimensional models. For the registration operation, we employ the ICP technique and adapt it to the inherent attributes of the symmetry plane extraction problem: (1) the measured point data may include noise and (2) the point cloud might be partially asymmetric. Since the proposed procedure consists of well-known elementary operations, it is easy to implement. The experimental results of the proposed procedure prove that it is ecient enough to handle very heavy data sets including millions of entities. Acknowledgements This research was supported by BK21 program through the Industrial Engineering Department of Ajou University. Authors thank Solutionix Ltd for providing measuring machines and reverse engineering software systems. Thanks also go to anonymous reviewers for their suggestions. References
Boada, I. (2004). An octree-based multiresolution hybrid framework. Future Generation Computer Systems, 20, 12751284. Desl, P. J., & McKay, N. D. (1992). A method for registration of 3D-shapes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 14(2), 239256.

320

M. Chang, S.C. Park / Computers & Industrial Engineering 55 (2008) 311320

Fukushima, K., & Kikuchi, M. (2006). Symmetry axis extraction by a neural network. Neurocomputing, 69, 18271836. Jiang, X. Y., & Bunke, H. (1992). A simple and ecient algorithm for determining the symmetries of polyhedra. GVGIP: Graphical Models and Image Processing, 54(1), 9195. Ke, Y., Fan, S., Zhu, W., Li, A., Liu, F., & Shi, X. (2006). Feature-based reverse modeling strategies. Computer-Aided Design, 38, 485506. Langbein, F. C., Marshall, A. D., & Martin, R. R. (2004). Choosing consistent constraints for beautication of reverse engineered geometric models. Computer-Aided Design, 36, 261278. Lee, K. H., & Woo, H. (2000). Direct integration of reverse engineering and rapid prototyping. Computers & Industrial Engineering, 38, 2138. Mills, B. I., Langbeing, F. C., Marshall, A. D., & Martin, R. R. (2001). Approximate symmetry detection for reverse engineering. In Proceedings of the sixth ACM symposium on solid modeling and applications (pp. 241248). New York: ACM Press. Momi, E. D., Chapuis, J., Pappas, I., Ferrigno, G., Hallermann, W., Schramm, A., et al. (2006). Automatic extraction of the mid-facial plane for cranio-maxillofacial surgery planning. International Journal of Oral & Maxillofacial Surgery, 35, 636642. Rusinkiewicz, S., & Levoy, M. (2001). Ecient variants of the ICP algorithm. In Proceedings of the third international conference on 3-D digital imaging and modeling (pp. 145152). Quebec City, Canada. Sugihara, K. (1984). An n log n algorithm for determining the congruity of polyhedra. Journal of Computer and System Science, 29, 3647. Tao, J., & Jiyong, K. (2007). A 3-D point set registration method in reverse engineering. Computers & Industrial Engineering, 53, 270276. Tate, S. J., & Jared, G. E. M. (2003). Recognising symmetry in solid models. Computer-Aided Design, 35, 673692. Tuzikov, A. V., Colliot, O., & Bloch, I. (2003). Evaluation of the symmetry plane in 3D MR brain images. Pattern Recognition Letters, 24, 22192233. Varady, T., Martin, R. R., & Cox, J. (1997). Reverse engineering of geometric models An introduction. Computer-Aided Design, 29, 255268. Wolter, J., Woo, T., & Volz, R. (1985). Optimal algorithms for symmetry detection in two and three dimensions. The Visual Computer, 1, 3748. Xinmin, L., Zhongqin, L., Tian, H., & Ziping, Z. (2001). A study of a reverse engineering system based on vision sensor for free-form surfaces. Computers & Industrial Engineering, 40, 215227.