Advanced Engineering Informatics 28 (2014) 37–49

Contents lists available at ScienceDirect

Advanced Engineering Informatics

journal homepage: www.elsevier.com/locate/aei

Vision-based material recognition for automated monitoring

of construction progress and generating building information
modeling from unordered site image collections
Andrey Dimitrov a,⇑, Mani Golparvar-Fard b,c,1
a
Department of Civil Engineering and Engineering Mechanics, Columbia University, New York, NY, United States
b
Department of Civil and Environmental Engineering, University of Illinois at Urbana-Champaign, Urbana, IL, United States
c
Department of Computer Science, University of Illinois at Urbana-Champaign, Urbana, IL, United States

a r t i c l e i n f o a b s t r a c t

Article history: Automatically monitoring construction progress or generating Building Information Models using site
Received 7 June 2013 images collections – beyond point cloud data – requires semantic information such as construction mate-
Received in revised form 10 October 2013 rials and inter-connectivity to be recognized for building elements. In the case of materials such informa-
Accepted 12 November 2013
tion can only be derived from appearance-based data contained in 2D imagery. Currently, the state-of-
Available online 24 December 2013
the-art texture recognition algorithms which are often used for recognizing materials are very promising
(reaching over 95% average accuracy), yet they have mainly been tested in strictly controlled conditions
Keywords:
and often do not perform well with images collected from construction sites (dropping to 70% accuracy
Material recognition
Building information models
and lower). In addition, there is no benchmark that validates their performance under real-world con-
Texton struction site conditions. To overcome these limitations, we propose a new vision-based method for
Support vector machine material classification from single images taken under unknown viewpoint and site illumination condi-
tions. In the proposed algorithm, material appearance is modeled by a joint probability distribution of
responses from a filter bank and principal Hue-Saturation-Value color values and classified using a multi-
ple one-vs.-all v2 kernel Support Vector Machine classifier. Classification performance is compared with
the state-of-the-art algorithms both in computer vision and AEC communities. For experimental studies,
a new database containing 20 typical construction materials with more than 150 images per category is
assembled and used for validation. Overall, for material classification an average accuracy of 97.1% for
200  200 pixel image patches are reported. In cases where image patches are smaller, our method
can synthetically generate additional pixels and maintain a competitive accuracy to those reported above
(90.8% for 30  30 pixel patches). The results show the promise of the applicability of the proposed
method and expose the limitations of the state-of-the-art classification algorithms under real world con-
ditions. It further defines a new benchmark that could be used to measure the performance of future
algorithms.
Ó 2013 Elsevier Ltd. All rights reserved.

1. Introduction Current state-of-the-art methods for automatically monitoring

construction progress or generating building information models
Material classification is an important part of any vision-based (BIM) primarily focus on using laser scanners or image-based
system for automated construction progress monitoring or gener- 3D reconstruction methods to generate 3D point cloud models
ation of semantically-rich as-built 3D models. Beyond 3D geomet- [1–13]. These methods can generate the required geometrical
rical information in form of point cloud models, such tasks require information for producing BIM [13]. Nonetheless, for generating
additional semantic information such as construction materials semantically-rich models, beyond geometrical information, mate-
and interconnectivity to be recognized for building elements. In rial and spatial relationship/interconnectivity among elements
the case of materials such information can mainly be derived from (e.g., a beam is supported by a column) needs to be extracted
appearance-based data contained in 2D imagery. from the collected data. In this paper, we focus primarily on the
problem of automated material classification. Automated material
classification (term used interchangeably with recognition in
⇑ Corresponding author. Tel.: +1 713 854 7744.
computer vision literature) not only helps with deriving appear-
E-mail addresses: [email protected] (A. Dimitrov), [email protected] (M.
ance-based data required for construction progress monitoring
Golparvar-Fard).
1
Tel.: +1 217 300 5226. purposes, but it also helps with segmentation of elements for

1474-0346/$ - see front matter Ó 2013 Elsevier Ltd. All rights reserved.
http://dx.doi.org/10.1016/j.aei.2013.11.002
38 A. Dimitrov, M. Golparvar-Fard / Advanced Engineering Informatics 28 (2014) 37–49

automated modeling of the semantically rich as-built 3D models. automated registration of 3D laser scanning point clouds with
In the following, the overall process of automatically generating BIM (e.g., [6,13,19]) for the purpose of controlling the quality of
BIM, the process of construction progress monitoring, and the the data collection. Turkan et al. [4,5] and Bosche et al. [7–9] have
need for material recognition are discussed in detail. Next, the also focused on automated recognition of 3D CAD objects in site la-
state-of-the-art techniques for material classification both in the ser scans for project performance control purposes.
Architecture/Engineering/Construction (AEC) and computer vision While collecting data for as-built modeling using laser scan-
communities are reviewed and their limitations are presented. ners can generate very accurate and complete geometrical mod-
Building on the state-of-the-art texture recognition methods, a vi- els, in most cases the approach suffers from the lack of a
sion-based method for AEC industry is presented and is also semantic understanding of the scene; in particular the ability to
accompanied with exhaustive validation experiments to bench- account for construction materials. Indeed, there is a need for dis-
mark its performance on all aspects of feature extraction, cluster- tinguishing different materials (e.g., formwork vs. finished con-
ing, and learning. A comprehensive dataset and a set of validation crete or concrete masonry units vs. facade bricks) during the 3D
methods that can be used in the field for development and bench- modeling process. Nonetheless, the laser scanning point clouds
marking of future algorithms are also introduced. The perceived in their XYZ form do not provide the data necessary for such anal-
benefits and limitations of the proposed method in the form of ysis. The recent addition of mounted cameras to laser scanners
open research challenges are presented. Detailed performance has allowed RGB color information to be associated with each
data, the experimental and validation codes, the benchmarking 3D point. Recent works by Kim et al. [20] and Son et al. [21] pro-
dataset, along with additional supplementary material for the posed using color to identify concrete elements and construction
proposed vision-based method can be found at http://raa- equipment. However, this approach has not yet been shown to be
mac.cee.illinois.edu/materialclassification. applicable for the simultaneous classification of multiple classes
of construction materials.
2. Automated construction progress monitoring and building Over the past few years, cheap and high-resolution digital cam-
information modeling eras, extensive data storage capacities, and the availability of Inter-
net connections on construction sites have enabled capturing and
3D modeling of the as-built environment is used by the AEC transmitting information about construction performance on a
industry in a variety of engineering analysis scenarios. Significant truly massive scale. Photographic data has become the favored
applications include progress monitoring of construction sites, documentation medium, as it contains rich information (geometry
quality control of fabrication and on-site assembly, energy perfor- and appearance), and data is collected quickly and inexpensively.
mance assessment, and structural integrity evaluation. The model- In the meantime, exciting research progress has been made on
ing process mainly consists of three sequential steps [11,13]: data techniques that can assist with 3D point cloud modeling of con-
collection, modeling, and analysis. In current practice, these steps struction sites or individual building elements using digital imag-
are performed manually by surveyors, designers, and engineers. ery. For example, Golparvar-Fard et al. [1,22] presented a new
Such manual tasks can be time-consuming, expensive, and prone algorithm based on Structure-from-Motion (SfM) coupled with
to errors. While the analysis stage is fairly quick, taking several Multi-View Stereo (MVS) [23–25] and Voxel Coloring [26] for gen-
hours to complete, data collection and modeling can be the bottle- erating dense 3D point cloud models from unordered imagery.
necks of the process. The data collection can spread over a few Fathi et al. [27–29] also presented a method for generating 3D
days, nonetheless the modeling stage can span over multiple point cloud models of infrastructure using video streams. Several
weeks or even months. Additionally, modeling often tends to be research projects such as [30–35] have also focused on evaluating
specific to certain analysis, making the application of one model the accuracy of as-built 3D modeling from photos and laser scan-
to multiple analyses very challenging. Thus, the applicability of ners. In majority of the experiments conducted using laser scan-
as-built modeling has been traditionally restricted to high latency ners, compared to the image-based reconstruction methods, a
analysis, where results need not be updated frequently. In fast higher density and completeness of the point cloud models has
changing environments such as construction sites, due to the diffi- been observed. Despite the recent advancements, the generation
culty in rapidly updating 3D models, model-based assessment of semantically-rich Solid Geometric Models (SGM) compatible
methods for purposes such as progress or quality monitoring have with the Industry Foundation Classes (IFC) format still poses three
had very limited applications. In consequence, there is a need for a main open research challenges:
fast, low-cost, reliable, and automated method for as-built model-
ing. This method should quickly generate and update accurate and 1. Segmentation: robust techniques in recognition are required
complete semantically-rich models in a master format that is to segment a 3D point cloud dataset based on geometric and
translatable to any engineering scenario and can be widely applied appearance information into distinct subsets;
across all construction projects [11,13]. The development of such a 2. Object placement: techniques in geometrical modeling and
3D modeling method also needs to be accompanied with a low- recognition are needed to populate the scene with distinct
cost and fast data collection process that is able to generate accu- BIM objects based on the segmented subsets;
rate and complete data sets for modeling purposes. 3. Inter-object relationships: techniques in recognition are
Over the past decade, research on as-built modeling has primar- required to identify the physical relationships between
ily focused on data collection and modeling techniques that can BIM objects in the scene.
generate 3D geometrical information. Cheok et al. [14] is among
the early works that demonstrated how LADAR (Laser Distance Ongoing research on image-based 3D modeling techniques can
and Ranging) can be used as an as-built 3D modeling tool for con- benefit from appearance information during the production of
struction monitoring purposes. This led to more systematic studies semantically-rich SGMs and progress monitoring; in particular:
of structured light data collection techniques [13,15,16]. Several
research studies have focused on showing that Terrestrial Laser 1. For producing surface and solid geometry: by segmenting
Scanning (TLS) techniques provide sufficient accuracy for use in point clouds based on appearance information so each
construction dimensional surveying and Quality Assurance/Quality subset can be separately used for extracting geometric
Control (QA/QC) [8,7,16–18]. Another line of research focuses on information (see Fig. 1),
 A. Dimitrov, M. Golparvar-Fard / Advanced Engineering Informatics 28 (2014) 37–49 39

Fig. 1. The process of generating semantically-rich SGMs from point cloud data and how material recognition can help with segmentation.

Fig. 2. Semantically labeling a 3D point cloud model with construction material info.

2. For associating semantic information to geometry: by recog- class of objects can be made from different materials [36]. Partic-
nizing construction materials to semantically label each ularly, in the context of construction projects, shape and material
generated 3D element (see Fig. 2), of objects tend to have a weak correlation. Similar shapes can be
3. For producing semantically-rich SGMs: by assembling adjacent made up of different construction materials; e.g., a foundation
3D elements of the same material into complete objects, and element as well as formwork section can both be shaped are rect-
4. For progress monitoring: by recognizing construction angular boxes. Also objects of a same material can have high
materials to detect the correct stage of the construction shape variability; e.g., (1) a concrete column can be rectangular
operation (see Fig. 3). as well as circular; (2) a column, a beam, or a slab can all be
made of concrete.
3. Previous work Material classification for automation in construction is closely
related to, but different from material and texture classification in
3.1. Material and texture recognition in computer vision computer vision in interesting ways:

The need for extracting semantic information from a scene has (1) In computer vision, most methods assume there is no strong
yielded two main computer vision research thrusts: object recog- prior available for material recognition, and thus these
nition and material recognition. Although object recognition methods should be able to handle classification on surfaces
methods have made significant advances in recent years [36], with certain randomness in orientation and periodicity in
they tend to rely on material-invariant features and overlook both their texture and geometry. In contrast, for the purpose
material specificity. Although the visual characteristics of an ob- of progress monitoring in construction, model-based meth-
ject is to some degree a function of its material category, different ods such as [1,4,5] can take advantage of an expectation of
classes of objects can be made of the same material and a given materials present in the scene;
40 A. Dimitrov, M. Golparvar-Fard / Advanced Engineering Informatics 28 (2014) 37–49

Fig. 3. Example of several stages for constructing a concrete foundation wall and how progress monitoring can benefit from appearance information. Using geometrical
information, methods such as Golparvar-Fard et al. [1] can detect if an element is constructed, but without material information, it will not provide enough information on the
current stage of its operation.

(2) Several works in computer vision such as [37] define texture monitoring and quality control. Brilakis et al. [47–49] are the first
in terms of dimensions such as periodicity, orientedness, and to use texture recognition techniques to facilitate image retrieval
randomness. As addressed by Liu et al. [36], surfaces made of as well as identify building elements from 2D images. Their
different materials can have similar texture patterns and ground-breaking work validated applicability of texture recogni-
thus existing computer vision methods for texture recogni- tion for material classification. It also showed that a material-
tion such as [38,39] may not be ideal for material based construction site image retrieval method based on texture
classification. recognition can successfully support image queries on construc-
tion sites by pre-identifying the materials in each image and com-
In the context of material classification, a variety of feature paring signatures of materials instead of image signatures [47,48].
based strategies have emerged and proven applicable to more In [49], Brilakis and Soibelman showed that shape features in-
loosely defined scenes. They are usually coupled with machine crease the flexibility of multi-modal image retrieval methods
learning techniques for training and testing purposes. Popular fea- and can further improve detection of elements by differentiating
tures include filter responses [38,40,41] and image patches [42], them into nonlinear and linear materials of certain directionality
and have been coupled with a Nearest Neighbor (NN) learning (e.g., steel columns, concrete beams). Zhu and Brilakis [50] further
[39,43]. Although they have shown promising results reaching over improved the method for automated concrete detection in image
95% accuracy, yet they do not perform well with images collected data. These methods clearly established the applicability of im-
from construction sites. The authors conducted exhaustive experi- age-based texture recognition for material classifications. Kim
ments (see Section 6.1) and in most cases accuracy dropped to 70% et al. [51] developed a scanning technique that utilizes laser scan-
and lower using real-world construction imagery. Brostow et al. ner point cloud data to automatically identify the sizes of stone
[44] have proposed to incorporate SfM point cloud information aggregates.
to appearance features for recognition and segmentation, however Nonetheless, to support automated generation of BIM and/or
only accuracies of below 70% have been reported from real-world generate models for progress and quality monitoring purposes,
data. Caputo et al. [45] have shown that Support Vector Machine the state-of-the-art approaches for material classification both in
(SVM) learning can improve on NN, yet its performance has not computer vision and AEC communities still need to be improved
been tested under real world construction conditions with high de- in the following ways:
gree of variability. Classification in computer vision has been
tested mostly on controlled and/or limited photographic data sets 1. There is a need for testing various types of features and
like CUReT (Columbia-Utrecht Reflectance and Texture Database), machine learning classification methods to form a robust
KTH-TIPS2 [45], and Brodatz textures (http://www.ux.uis.no/ automated material classification technique that maintains
tranden/brodatz.html) and University of Illinois at Urbana- a high level of accuracy in uncontrolled construction envi-
Champaigns textures (http://www-cvr.ai.uiuc.edu/poncegrp/data/ ronments. In Section 6, the performance of the state of the
). Dissatisfaction with these benchmarks has driven the emergence art algorithms are compared for uncontrolled construction
of real world data sets, like Flicker Materials Database [46], to take environments.
into account inconsistencies in data quality and context. In this 2. There is a need for systematic data collections and compre-
respect, there is still no comprehensive construction materials hensive datasets of material classification in uncontrolled
library that could be used for training and testing of material construction environments that can be used for bench-
classification algorithms for the purpose of semantically-rich marking material recognition algorithms.
as-built 3D modeling.
In the following sections, a new method and a comprehensive
3.2. Material classification in AEC community dataset for material classification are introduced. The perfor-
mance of our proposed method is also compared against the
Automated recognition of materials in a construction setting state-of-the-art methods in both computer vision and AEC
has been of particular importance to construction progress communities.
 A. Dimitrov, M. Golparvar-Fard / Advanced Engineering Informatics 28 (2014) 37–49 41

4. Proposed material recognition approach where I½x; y is the normalized gray-scale intensity at ðx; yÞ, f ½x; y
represents intensity of filter f at ðx; yÞ, and ðn1 ; n2 Þ and ðm1 ; m2 Þ rep-
Our proposed image-based material recognition algorithm is resent the range of the filter bank. The filter bank consists of 48 fil-
based on a statistical distribution of filter responses over the ters similar to [39], first and second derivatives of Gaussian at 6
images in form of primitives such as edges, spots, waves, and orientation and 3 scales (total of 36), 8 Laplacian of Gaussian filters,
Hue-Saturation-Value (HSV) color values, and is divided into two and 4 Gaussians. The scales of filters in this filter bank range be-
stages: learning and classification. The statistical distribution of fil- tween r = 1 to 10. These filters are shown in Fig. 6.
ter responses has been shown to be a good descriptor for texture Next, following a bag-of-words model, we cluster the filter re-
recognition [39,45]. The main reason for choosing color values sim- sponses to generate codebooks of material appearances using the
ilar to others such as [45,47,48] as it is validated in Section 6.3 is to k-means clustering method [52]. The cluster centers (codewords)
improve robustness of the proposed method for recognition of will be Textons which are visual atoms of the visual perception of
materials when images are taken far from objects and texture materials [38,53]. Thus, the texture of the materials will be defined
information is less apparent. In addition, the HSV color values are by modeling the texton frequencies learned from the training
chosen to improve the robustness to changes in brightness which images. For all images captured under various illumination and
can be dominant in construction imagery. Fig. 4 shows how the viewpoints for all different materials in the training dataset, the fil-
brightness and color gradients of a concrete foundation wall can ter responses are concatenated. Next, a fixed number of texton
change when images are taken far from the elements. cluster centers are computed using the k-means clustering algo-
The main technical contributions of this work are (1) a Bag of rithm where k is the number of visual words (the size is discussed
Words (BoW) pipeline for forming statistical distributions of both in Section 6.1). After quantizing individual features into k visual
filter responses and HSV color values; (2) a multiple binary SVM words, where each feature is assigned a membership to the closest
classifier that can robustly learn and infer construction material cluster, the distribution of visual words per image is calculated for
categories; and (3) a construction material library and validation all of the different features by assigning each pixel in the image to
metrics for future algorithmic developments. Fig. 5 shows an over- the nearest visual word index and forming the histogram over the
view of the learning and classification components of our proposed frequency of the visual words.
algorithm. Each step is discussed in the following subsections. The histogram of texton frequencies is then used to form the
codebooks and the material models that are corresponding to our
4.1. Generating codebooks of material appearances training images. In our work, HSV colors are also incorporated to
leverage the color information for a robust material classification.
In the learning stage of our algorithm, training images are ini- It was hypothesized that since HSV instead of standard R, G, and
tially converted to gray-scale and their color intensities are nor- B color channels can separately model the brightness, it can pro-
malized to have zero mean and unit standard deviations. This vide a higher degree of invariance to varying illumination condi-
will make the distribution of the color values less dependent on tions on the sites or the time images are captured. Similar to the
the illumination which varies significantly on a construction site. texture recognition, for all images in the training datasets, the nor-
Next, these images are convolved with a comprehensive filter bank malized HSV color values are aggregated and clustered into HSV
[38] to generate filter responses using following equation: textons using the (k-means clustering algorithm). Next, a histo-
gram of color textons, i.e., the frequency with which each color tex-
X
n2 X
m2
I½x; y  f ½x; y ¼ I½k; l:f ½x  k; y  l ð1Þ ton occurs in the images, forms the codebook corresponding to
k¼n1 l¼m1 each training image. The texture and color histograms are then
concatenated to generate a unique codebook for each image.
Fig. 6 shows the details of the training process.

4.2. Discriminative machine learning models of material categories

Our visual representation based on codebooks for construction

materials is leveraged by training discriminative machine learning
models of material categories. We train binary C-Support Vector
Machine (SVM) classifiers [54] with linear, v2 , and radial basis
function (RBF) kernels for each material category independently.
SVM classifiers are discriminative binary classifiers that optimize
a decision boundary between two classes. The SVM classification
solves the following optimization problem:

1 X
z
sðw; nÞ ¼ min kwk2 þ C ni ð2Þ
2 i

subject to yi ðwT xi þ bÞ P 1  ni ; i ¼ 1; 2; . . . ; m ð3Þ

ni P 0; i ¼ 1; 2; . . . ; m ð4Þ

Here, w is the normal vector to the hyperplane that separates the

images of a material class from others, b is the offset of the hyper-
plane from the origin, ni is the slack variables for measuring the de-
gree of misclassification of the m-dimensional histogram data x,
C > 0 is the tradeoff between regularization and constraint viola-
Fig. 4. Example of how the color of a concrete foundation wall can change during a
few hours. Here time-lapse images are captured every 5 min. As seen beyond the
tion, and finally yi is the outcome of the classification. After solving,
shadow from the column located just next to the image patch, the color gradient is the SVM classifier predicts 1 if wT x þ b P 0 and 1 otherwise. The
significantly changing throughout the day. decision boundary is given by the line wT x þ b ¼ 0 that separates
42 A. Dimitrov, M. Golparvar-Fard / Advanced Engineering Informatics 28 (2014) 37–49

Fig. 5. An overview of the proposed learning and inference model for material recognition.

Fig. 6. The training process in our proposed material recognition method.

 A. Dimitrov, M. Golparvar-Fard / Advanced Engineering Informatics 28 (2014) 37–49 43

the images from a material class from others in a m-dimensional Form Work, Grass, Gravel, Marble, Metal-Grills, Paving,
space (m: the size of Texton and HSV cluster centers). Soil-Compact, Soil-Vegetation, Soil-Loose, Soil-Mulch,
Note that the above definition applies to a linear classifier. We Stone-Granular, Stone-Limestone, Wood.
also use non-linear kernels due to their suitability for classification 2. Sub-categories: Each course material definition is broken
of histograms. As validated in Section 6.1, the histograms cannot be down to several representative sub-types, depending of
separated linearly in a reasonable way, and thus v2 (5) and RBF (6) the material. For example, concrete can be present in a
kernels are used to map the data into a high dimensional feature scene as smooth finish, course finish, good condition, or
space. deteriorated. Additionally, each material exhibits a specific
X appearance under different lighting conditions. Two to four
n
ðxi  xj Þ2
kðxi ; xj Þ ¼ 1  ð5Þ lighting conditions per material where documented.
1
i¼1 2
ðxi þ xj Þ 3. Poses: Scale and orientation variability is documented for
2
each sub-category. For a digital resolution of 200  200 pix-
kðxi ; xj Þ ¼ ecjxi xj j ð6Þ
els, images representing an approximate physical length of
where kðxi ; xj Þ ¼ huðxi Þ; uðxj Þi denotes the inner product in the Hil- 1200 ; 2400 and 4800 were gathered. Three to five view orienta-
bert space and corresponds to applying the algorithm to the tions in respect to the surface normal were also considered.
mapped material histogram data point uðxi Þ. In this dual formula- Thus, a total of 9–15 poses make up each material sub-
tion, we maximize the following: category. One sample for the 2400 and 4800 scales and two
non-overlapping samples for the 1200 scale were extracted
X
m
1X
WðaÞ ¼ ai  ai aj yi yj kðxi ; xj Þ ð7Þ from each pose. The entire dataset is made public at:
i¼1
2 ij http://raamac.cee.illinois.edu/materialclassification. Fig. 7
X
subject to 0 6 ai 6 C and ai yi ¼ 0 ð8Þ shows examples of the materials in the CML.

In this formulation, the decision function will be signðhðxÞÞ, where: 5.2. Performance evaluation measures
X
q
hðxÞ ¼ al yl kðx; xl Þ þ b ð9Þ To quantify and benchmark the performance of the material
i¼1 classification algorithm, we plot the Precision–Recall curve and
where q refers to the number of kernel computations needed to use a confusion matrix to represent average classification accura-
classify a joint texton and color material histogram with the kernel- cies. These metrics are both set-based measures; i.e., they evaluate
ized SVM. For solving these equations, we use LibSVM [55] and v2 the quality of an unordered set of data entries. Here, we define pre-
kernel module of [56]. cision and recall as follows:
In order to extend the binary classification decision of each SVM TP
classifier to multiple classes, we adopt the one-versus-all multi- precision ¼ ð10Þ
TP þ FP
class classification scheme where a classifier for each material is TP
defined. When training the SVM classifier that corresponds to each recall ¼ ð11Þ
TP þ FN
material class, we set all the examples from that class as positive
and the examples from all other classes as negatives. The result where in TP is the number of True Positives, FN is the number of
of the training process is one binary SVM classifier per material False Negatives and FP is the number of False Positives. For instance,
of interest. Given a novel testing image for which we need to iden- if a brick image is correctly recognized under the brick class, it will
tify the material, we apply all binary classifiers and select the be a TP; if a concrete image is incorrectly recognized as brick, it will
material class corresponding to the classifier with the highest clas- be a FP for the brick class. When a brick image is not recognized un-
sification score. der the brick class, then the instance is a FN. The particular rule used
to interpolate precision at recall level i is to use the maximum pre-
cision obtained from the material class for any recall level great
5. Experiments and validation metrics
than or equal to i. For each recall level, the precision is calculated,
and then the values are connected and plotted in form of a curve.
5.1. Data collection and experimental setup
The performance of the one-vs.-all multi-class material classifi-
ers is analyzed using a confusion matrix. The confusion matrix re-
Due to the lack of existing datasets for benchmarking perfor-
turns the average accuracy per material class. The average accuracy
mance of material recognition algorithms in real-world conditions,
of the material classification is calculated using the following
it was necessary to create a new comprehensive Construction
formula:
Materials Library (CML) that enables recognition for a large quan-
tity of material types recorded in a variety of as-built contexts. This TP þ TN
accuracy ¼ ð12Þ
dataset is for both training and testing purposes so that it can be TP þ TN þ FP þ FN
released to the community for further development and validation
of new algorithms. For this purpose, we collected a library of 3000 A confusion matrix shows for each pair of material classes hC 1 ; C 2 i,
material samples, each being an uncompressed color image of res- how many material images from C 1 were assigned to C 2 . Each col-
olution 200  200 pixels. These images were recorded from seven umn of the confusion matrix represents the predicted material class
different scenes (i.e., 5 construction sites and 2 existing buildings). and each row represents the actual material class. The detected TP
In order to create a comprehensive dataset, varying degrees of and FP are compared and the percentage of the correctly predicted
viewpoint, scale, and illumination where accounted for during classes with respect to the actual class is created and represented in
the collection period span of seven months. The following de- each row.
scribes the dataset:
6. Experimental results
1. Material categories: The CML comprises 20 major construc-
tion material types: Asphalt, Brick, Cement-Granular, Experiments to benchmark the performance of the proposed
Cement-Smooth, Concrete-Cast, Concrete-Precast, Foliage, material classification method were conducted using the CML
44 A. Dimitrov, M. Golparvar-Fard / Advanced Engineering Informatics 28 (2014) 37–49

Fig. 7. The construction material library.

(see Section 5.1). For all cases, the data was split into two subsets: Table 2
a learning set and a classification (testing) set. Thus, a randomly se- Effect of various features used in the classification algorithm on the overall accuracy.

lected 70% of the CML samples were allocated to learning and the Feature SVM (Linear) (%) SVM (v2 ) (%) SVM (RBF) (%) v2 (%)
remaining 30% were used for testing the classification perfor-
LM 70.2 80.6 80.1 78.4
mance. We performed cross validation to randomly partition the rLM 69.9 75.6 76.3 74.2
training and testing datasets ten times into 70% training, 30% test- IP 59.1 63.8 64.6 63.7
ing, and report the average performances. The performance of the HSV 73.3 92.7 89.2 91.1
proposed method is validated against the state-the-art algorithms RGB 71.1 92.6 87.1 88.0
LM + RGB 83.1 95.2 92.5 91.2
by comparing the effect of different feature sets, clustering granu- LM + HSV 87.1 97.1 93.7 92.1
larities, and learning strategies. In addition, we validate the robust-
ness of the method under real world conditions by examining the
effect of image size and quality on classification accuracy. Table 1
shows the optimal material classification parameters. In the next The results show that non-linear SVM generally performs better
few subsections, we study the impact of each of these parameters than v2 histogram matching and that linear SVM is the worst. RBF
on the overall classification accuracy. and v2 kernels show similar performance when used on texture-
based features. However, the latter performs significantly better
on color based features. For all testing algorithms, the combination
6.1. Effect of feature selection and training parameters
of LM and HVS features yields the best results, suggesting the
applicability of this feature type for material recognition.
In total, six types of features were tested (see Table 2) falling
To assess the computation cost of classification versus accuracy,
into two categories: texture (grayscale) and color. The filter based
several clustering granularities were tested. Granularity defines
approach of Leung and Malik [38] was implemented in its original
the number codewords (hence of bins) in each training and testing
48-dimensional form (LM) as well as in its 18-dimensional rota-
histogram. In theory, given N bins, computation of each histogram
tionally invariant form (rLM). Image patch (IP) features of Varma
is an OðNÞ operation and classification with SVM is an OðN 2 Þ oper-
and Zisserman [43] were implemented with a 7  7 pixel neighbor-
ation. This means that, as the number of bins increase, the compu-
hood as it produces the best results in accordance to the literature.
tation time increases quadratically. It is thus important to limit the
For color based features, normalized RGB and HSV color channels
number of bins for practical purposes. Table 3 shows a summary of
are used to create 3-dimensional feature descriptors. We also
the results. In our implementations, the ranges shown in this table
tested a combinations of these categories by concatenating their
translated to classification times of 0.2 ms/image for 3 bins to
frequency histograms. For classification, we test SVM classifiers
9 ms/image for 20 bins. The results show that the computational
using linear, v2 , and RBF kernels and we compared the perfor-
time stays in near real-time performance zone. The benchmarking
mance with simple v2 histogram matching which is proposed in
on computational time was conducted on an Intel Core i7-2860QM
the state-of-the-art [43]:
@ 2.50 GHz with 16 GB RAM running Windows 7SP1 and Matlab
1X K
½hi ðmÞ  hj ðmÞ
2 11b.
v2 ðhi ; hj Þ ¼ ð13Þ All features achieve best performance around 15 textons per
2 m¼1 hi ðmÞ þ hj ðmÞ
material. The texture based features hover around in the 70% range
wherein hi and hj represent two different histograms and v2 ðhi ; hj Þ with LM showing the highest average precision. However, color
shows the distance between the two histograms. To illustrate the based features perform even better, ranging between 80% and
effect on the testing algorithm alone, we compare these approaches 93%, with HSV outperforming RGB in all cases. A significant
in Table 2. Each SVM kernel classifier has been optimized by per- improvement is shown when LM features are combined with
forming a systematic sweep over the range of its respective param-
eters (see Section 4.2) and retaining the best performing values for
subsequent testing. Table 3
Effect of Clustering Granularity.

Table 1 Feature 3 (%) 4 (%) 5 (%) 7(%) 10 (%) 15 (%) 20 (%)

Optimal classification parameters. LM 66.9 70.4 70.0 73.8 74.9 80.6 77.7
rLM 62.9 66.9 68.0 73.5 73.7 75.6 73.4
Features LM (48-dimensional) + HSV (3-dimensional)
IP 55.3 59.9 61.0 61.4 63.0 63.8 62.9
No. bins per material 15 Textons + 15 normalized-HSV centers HSV 87.7 92.2 90.8 92.1 91.8 92.7 92.4
Classifier SVM, v2 kernel RGB 81.5 83.9 84.6 87.9 91.6 92.6 91.3
Classifier Parameters C ¼ 700 LM15 + HSV 94.00 96.4 96.9 95.7 96.5 97.1 96.0
 A. Dimitrov, M. Golparvar-Fard / Advanced Engineering Informatics 28 (2014) 37–49 45

HSV features. The results of between 94% and 97% remain steady The JPEG format was chosen for testing image quality due to its
across the range of granularities, with a best performance of wide use and range of compression settings. By adjusting these set-
97.1% from combined LM features ar 15 bins and HSV features at tings, common sources of photographic data where replicated, as
15 bins. shown in Fig. 11. The entire image set of the Construction Materials
We can note that the performance of the LM filter bank for the Library was compressed to different degrees and tested. The results
CML dataset is well under the one reported on the CURet dataset. in Table 4 show that texture features become unreliable when the
The average of 78.4% classification rate is much lower than the quality of images decreases. On the other hand, color based fea-
95%+ rate as reported in the literature [43,39]. This implies that tures retain a reasonable accuracy level even for highly com-
this approach by itself is not robust enough for use in the context pressed images. Combined texture and HSV color features
of as-built modeling for on-site photographs. The state-of-the-art produce the best results for all quality levels.
AEC algorithm [48] of Brilakis et al. was implemented and tested
against the CML benchmark, and an average of 52.8% was reported
(see the precision-recall graph in Fig. 8). Here, we trained their 6.3. Effect of image size and synthesis
classification method of [48] on the same training dataset as our
classifier. Each on-site photographs can depict a collection of objects of
Fig. 9 shows the confusion matrix on the performance of our different sizes and distances from the camera. Thus the available
proposed material classification method. As observed in this figure, number and distribution of pixels that represent a single material
most material classifications have very high average accuracies cannot be predicted. Material classification first requires the selec-
(above 90%). Fig. 10 shows several examples of cases where the tion of a region (set of pixels) of unknown size and shape within
classification of images were incorrect. In several categories such the image. Due to the probabilistic nature of the proposed method,
as cement and concrete classes, there is still 2–7% confusion among a reduction in the available pixels used to generate features can re-
them. For these categories, human observation only based on the sult in a decrease in accuracy. To this end, we test several region
200  200 pixel images in the library may not result in right clas- sizes to expose the practical limitations of the system. In addition,
sification either, which is primarily related to the absence of con- we test image synthesis techniques to test if the available real pix-
text information. In Section 7, we discuss how multiple els can be combined with synthetically generated pixel in order to
observations (several images of the same material) or shape/con- overcome the afore mentioned limitation.
text information can improve the classification results. The original data contains images of 200  200 pixels in size.
The 49  49 pixel size of the filters used to generate textural fea-
tures reduces the effective area to 152  152 pixels. For an input
6.2. Effect of image quality image size of 100  100 pixels, this area is reduced by a factor of
4, and for 50  50 pixels, features can be extracted for only four
On-site data can come from a variety of sources. Photographs pixels. To test the effects of size reduction, the images of the
can be captured by many devices ranging from high end SLR cam- CML were progressively subsampled with an finite impulse re-
eras to smart phones. In addition, images can be altered (com- sponse (FIR) filter [57] to retain consistency in respect to anti-
pressed) during transfer or archiving. This can produce a data set aliasing.
that is both inconsistent and poor in terms of image quality. Thus To improve classification results, the smaller sized versions of
it is important to validate our method against a variety of image the CML where expanded by two synthesis techniques: (1) A naive
quality conditions and determine which combination of features reflection/rotation (RR) method was used to expand the images
and training parameters works best. with pixels similar to the ones around the edges; and (2) An image
quilting (IQ) algorithm [58] was used to generate a larger sized im-
age from samples of the original. The effect of these techniques on
a sample of brick is shown in Fig. 12.
The results in Table 5 and Fig. 13 show several important re-
sults. First, we note that, in all experiments, the use of joint texture
and color codebooks outperforms texture or color alone. Further-
more, it is shown to be viable for images as small as 30  30 pixels
in size, maintaining an average precision of above 90.8%. Second,
for smaller image sizes, expansion by synthesis increases precision
significantly, with the RR method outperforming image quilting.
Third, with the reduction of image size, texture based classification
deteriorates drastically while color based classification suffers only
slightly. This is consistent with the intuition that textures become
less apparent when described by fewer pixels while the mean color
of the image remains constant. Other techniques for image synthe-
sis have been reported in literature, however the search for the
optimal synthesis algorithm for material classification is left for fu-
ture research.

7. Discussion on the proposed method and research challenges

The authors introduce a robust material classification method

and the first comprehensive dataset for semantically-rich as-built
Fig. 8. Precision-recall graph benchmarking performance of the proposed method
3D modeling purposes. The average accuracies of 97.1% across all
with the state-of-the-art algorithms. Algorithm 1 refers to [43,39] and Algorithm 2 materials promise the applicability of the proposed method. For
refers to [48]. detecting material in smaller image patches, we also presented
46 A. Dimitrov, M. Golparvar-Fard / Advanced Engineering Informatics 28 (2014) 37–49

Fig. 9. Confusion matrix representing the cross validated average accuracies of the proposed method for material classification.

Fig. 10. Several examples of incorrect material classifications.

Fig. 11. A range of compression settings were used to simulate an array of possible image sources.
 A. Dimitrov, M. Golparvar-Fard / Advanced Engineering Informatics 28 (2014) 37–49 47

Table 4
Effect of Image Compression.

Feature BMP (%) JPEG100 (%) JPEG90 (%) JPEG75 (%) JPEG50
LM15 80.6 74.6 74.8 39.9 16.6
HSV15 92.7 91.9 90.8 91.5 85.9
LM15 + HSV15 97.1 96.0 96.6 96.4 92.7

Fig. 13. The effect of image size on classification precision for images expanded
Fig. 12. Synthesizing the input images to increase area. synthetically with RR (except for original images).

the RR method and illustrated that our method can stay maintain
accuracies higher than 90%. 2. Expanded image sources: Additional effects on quality,
Such results also indicate the robustness of the method to dy- such as images originating from video frames or scenes
namic changes of illumination, viewpoint, camera resolution, and with motion.
scale which is typical for unordered construction image collections. 3. Robust image synthesis methods: To better classify mate-
The method was also shown to remain effective for very low qual- rial for very small image patches (e.g., when time-lapse
ity images. While this paper presented the initial steps towards images are used for detecting progress of concrete col-
processing unordered image collection for the purpose of material umns that are very far from the camera. In these cases,
recognition, several critical challenges remain. Some of the open the number of pixels associated with the element could
research problems include: be very minimal).
4. Multiple images: The effect of leveraging multiple images
1. The impact of depth textures: The effect of texture maps gen- registered to reconstructed point cloud models – typically
erated from depth values of a reconstructed point cloud generated through Structure-from-Motion algorithms –
from the observed material. through various voting/scoring methods;

Table 5
Effect of Image Size on Accuracy of Material Classification.

Size (n  n pixels) Feature Original RR IQ

200 LM15 80.6% N/A N/A
HSV15 92.7% N/A N/A
LM15 + HSV15 97.1% N/A N/A
150 LM15 69.7% 76.4% 63.7%
HSV15 91.5% 92.1% 89.7%
LM15 + HSV15 96.1% 96.6% 93.7%
100 LM15 56.0% 66.4% 53.3%
HSV15 93.0% 91.7% 85.2%
LM15 + HSV15 94.3% 96.4% 93.6%
75 LM15 39.9% 59.6% 47.9%
HSV15 91.5% 91.6% 88.5%
LM15 + HSV15 92.1% 94.5% 92.2%
50 LM15 16.6% 53.9% 37.1%
HSV15 88.4% 87.8% 87.3%
LM15 + HSV15 92.0% 93.4% 86.8%
30 LM15 N/A 30.1% 30.3%
HSV15 84.8% 87.6% 84.2%
LM15 + HSV15 N/A 90.8% 85.7%
48 A. Dimitrov, M. Golparvar-Fard / Advanced Engineering Informatics 28 (2014) 37–49

5. Leveraging shape/context prior: For the purposes of as- References

built modeling, the effect of 3D shape/context informa-
tion extracted from a point cloud reconstruction (e.g., this [1] M. Golparvar-Fard, F. Pea-Mora, S. Savarese, Automated progress monitoring
using unordered daily construction photographs and ifc-based building
material belongs to a wall as opposed to a slab) on clas- information models, J. Comput. Civil Eng. 0 (2012) 147.
sification accuracy; [2] X. Xiong, A. Adan, B. Akinci, D. Huber, Automatic creation of semantically rich
6. Leveraging 4D BIM: For the purposes of model-based pro- 3d building models from laser scanner data, Autom. Constr. 31 (2013) 325–
337.
gress monitoring, the impact of material expectation (e.g., [3] M. Golparvar-Fard, F. Pea-Mora, C. Arboleda, S. Lee, Visualization of
for monitoring construction of a concrete wall, the construction progress monitoring with 4d simulation model overlaid on
expected materials are rebar cage, formwork, and concrete) time-lapsed photographs, J. Comput. Civil Eng. 23 (2009) 391–404.
[4] Y. Turkan, F. Bosche, C.T. Haas, R. Haas, Automated progress tracking using 4d
can reduce the set of possible classes and could increase
schedule and 3d sensing technologies, Autom. Constr. 22 (2012) 414–421
accuracy; (Planning Future Cities-Selected papers from the 2010 eCAADe Conference).
[5] Y. Turkan, F. Bosche, C. Haas, R. Haas, Toward automated earned value tracking
using 3d imaging tools, J. Constr. Eng. Manage. 139 (2013) 423–433.
8. Conclusion and future work [6] F. Bosche, Automated recognition of 3d cad model objects in laser scans and
calculation of as-built dimensions for dimensional compliance control in
construction, Adv. Eng. Informat. 24 (2010) 107–118.
This paper presented a robust material classification method for [7] F. Bosche, C. Haas, Automated retrieval of 3d cad model objects in construction
semantically-rich as-built 3D modeling and construction monitor- range images, Autom. Constr. 17 (2008) 499–512.
ing purposes. A Construction Materials Library was formulated to [8] F. Bosche, C. Haas, B. Akinci, Performance of a new approach for automated 3d
project performance tracking, ASCE J. Comput. Civil Eng., Spec. Issue 3D
train and test the proposed method. This real-world dataset ex-
Visualiz. (2008).
posed the limitations of current material classification methods [9] F. Bosche, C.T. Haas, B. Akinci, Automated recognition of 3d cad objects in site
and validated the performance of the proposed algorithm. From laser scans for project 3d status visualization and performance control, J.
our testing, an average classification rate of 97.1% was achieved. Comput. Civil Eng. 23 (2009) 311–318.
[10] H. Fathi, I. Brilakis, Machine Vision-Based Infrastructure As-Built
The effects of various parameters such as the choice of the filter Documentation Using Edge Points, pp. 757–766.
bank, the number of Textons and color bins, the size of the histo- [11] I. Brilakis, M. Lourakis, R. Sacks, S. Savarese, S. Christodoulou, J. Teizer, A.
grams, the impact of camera parameters, and various classifiers Makhmalbaf, Toward automated generation of parametric BIMS based on
hybrid video and laser scanning data, Adv. Eng. Informat. 24 (2010) 456–465
and kernels were considered. Furthermore, we have shown that, (Construction Informatics).
by leveraging image synthesis techniques, our method can produce [12] I. Brilakis, S. German, Z. Zhu, Visual pattern recognition models for remote
accurate results for images as small as 30  30 pixels in size. The sensing of civil infrastructure, J. Comput. Civil Eng. 25 (2011) 388–393.
[13] P. Tang, D. Huber, B. Akinci, R. Lipman, A. Lytle, Automatic reconstruction of as-
addition of HSV color channels as a feature for training results in built building information models from laser-scanned point clouds: a review
a more robust classification algorithm, and can be implemented of related techniques, Autom. Constr. 19 (2010) 829–843.
as critical process in an automated as-built 3D modeling system. [14] G.S. Cheok, W.C. Stone, R.R. Lipman, C. Witzgall, Ladars for construction
assessment and update, Autom. Constr. 9 (2000) 463–477.
In case of images with larger resolutions, our algorithm can act [15] C. Gordon, B. Akinci, Technology and process assessment of using ladar and
as a sampling method by classifying materials for a given number embedded sensing for construction quality control, in: Construction Research
of 200  200 pixel image patches. Alternatively, larger resolution Congress, ASCE Reston, VA, pp. 5–7.
[16] P. Tang, B. Akinci, D. Huber, Quantification of edge loss of laser scanned data at
images could also be subsampled to a smaller resolution to de-
spatial discontinuities, Autom. Constr. 18 (2009) 1070–1083.
crease computation time. [17] E.B. Anil, P. Tang, B. Akinci, D. Huber, Assessment of the quality of as-is
The presented method has not yet addressed the challenge of building information models generated from point clouds using deviation
segmentation. Given a photograph taken on site, a region belong- analysis, in: IS&T/SPIE Electronic Imaging, International Society for Optics and
Photonics, pp. 78640F–78640F.
ing to a single material needs to be identified. Material classifica- [18] G. Cheok, J. Filliben, A. Lytle, Guidelines for accepting 2D building plans,
tion can then be performed on the pixels in that region. In the Technical Report, NIST Interagency/Internal Report (NISTIR)-7638, 2009.
presence of BIM, these regions can be derived by back-projecting [19] D. Belton, A. Gibson, B. Stansby, S. Tingay, K.-H. Bae, Modelling of an
inexpensive 9M satellite dish from 3D point clouds captured by terrestrial
BIM element surfaces with a prior expectation of material onto laser scanners, in: D.D. Lichti, A.F. Habib (Eds.), ISPRS Workshop Laser
site images. In the presence of a segmented point cloud, the re- Scanning 2011, Aug 29–31 2011, Volume XXXVIII-5/W12, Calgary, ISPRS,
gions can be derived from back-projecting points and in-painting Canada, 2011.
[20] H. Son, C. Kim, C. Kim, Automated color modelbased concrete detection in
them. Our ongoing research will explore the use of the outlined construction-site images by using machine learning algorithms, J. Comput.
method as a technique for random sample consensus within Civil Eng. 26 (2012) 421–433.
non-uniform regions in a single image as well as across multiple [21] C. Kim, B. Kim, H. Kim, 4d {CAD} model updating using image processing-
based construction progress monitoring, Autom. Constr. 35 (2013) 44–52.
view-points. Further applications of the proposed material classi- [22] M. Golparvar-Fard, F. Pena-Mora, S. Savarese, Monitoring changes of 3d
fication for segmentation of 3D point cloud models and model- building elements from unordered photo collections, in: IEEE International
based progress monitoring can be studied. In addition, material Conference on Computer Vision Workshops (ICCV Workshops), 2011, pp. 249–
256.
classification performance could be improved by exploring the
[23] Y. Furukawa, B. Curless, S.M. Seitz, R. Szeliski, Towards internet-scale multi-
incorporation of 3D geometrical information. The geometric sur- view stereo, in: IEEE Conference on Computer Vision and Pattern Recognition
face properties of a given surface could supply additional descrip- (CVPR), IEEE, 2010, pp. 1434–1441.
tive features that can be integrated along with the aforementioned [24] Noah Snavely, Steven M. Seitz, Richard Szeliski, Skeletal graphs for efficient
structure from motion, in: IEEE Conference on Computer Vision and Pattern
image based features. Recognition, 2008, vol. 2.
[25] D. Crandall, A. Owens, N. Snavely, D. Huttenlocher, Discrete-continuous
optimization for large-scale structure from motion, in: IEEE Conference on
Acknowledgments Computer Vision and Pattern Recognition (CVPR), IEEE, 2011, pp. 3001–
3008.
The support of Turner Construction, Holder Construction, Gil- [26] S.M. Seitz, C.R. Dyer, Photorealistic scene reconstruction by voxel coloring, Int.
J. Comput. Vis. 35 (1999) 151–173.
bane Construction, and Skanska with data collection and members [27] H. Fathi, I. Brilakis, Machine Vision-Based Infrastructure As-Built
of the RAAMAC lab with collecting and annotating construction Documentation Using Edge Points, pp. 757–766.
material images is acknowledged. Any opinions, findings, and con- [28] I. Brilakis, H. Fathi, A. Rashidi, Progressive 3d reconstruction of infrastructure
with videogrammetry, Autom. Constr. 20 (2011) 884–895.
clusions or recommendations expressed in this paper are those of
[29] H. Fathi, I. Brilakis, Automated sparse 3d point cloud generation of
the authors and do not necessarily reflect the views of the infrastructure using its distinctive visual features, Adv. Eng. Informat. 25
companies mentioned above. (2011) 760–770.
 A. Dimitrov, M. Golparvar-Fard / Advanced Engineering Informatics 28 (2014) 37–49 49

[30] Z. Zhu, I. Brilakis, Comparison of optical sensor-based spatial data collection [44] G.J. Brostow, J. Shotton, J. Fauqueur, R. Cipolla, Segmentation and recognition
techniques for civil infrastructure modeling, J. Comput. Civil Eng. 23 (2009) using structure from motion point clouds, in: Computer Vision–ECCV 2008,
170–177. Springer, 2008, pp. 44–57.
[31] A. Bhatla, S.Y. Choe, O. Fierro, F. Leite, Evaluation of accuracy of as-built 3d [45] B. Caputo, E. Hayman, M. Fritz, J.-O. Eklundh, Classifying materials in the real
modeling from photos taken by handheld digital cameras, Autom. Constr. 28 world, Image Vis. Comput. 28 (2010) 150–163.
(2012) 116–127. [46] L. Sharan, R. Rosenholtz, E. Adelson, Material perception: what can you see in a
[32] M. Golparvar-Fard, J. Bohn, J. Teizer, S. Savarese, F. Pena-Mora, Evaluation of brief glance?, J Vis. 9 (2009) 784.
image-based modeling and laser scanning accuracy for emerging automated [47] I. Brilakis, L. Soibelman, Y. Shinagawa, Material-based construction site image
performance monitoring techniques, Autom. Constr. 20 (2011) 1143–1155. retrieval, J. Comput. Civil Eng. 19 (2005) 341–355.
[33] A. Rashidi, F. Dai, I. Brilakis, P. Vela, Optimized selection of key frames for [48] I.K. Brilakis, L. Soibelman, Y. Shinagawa, Construction site image retrieval
monocular videogrammetric surveying of civil infrastructure, Adv. Eng. based on material cluster recognition, Adv. Eng. Informat. 20 (2006) 443–
Informat.s (2013). 452.
[34] F. Dai, A. Rashidi, I. Brilakis, P. Vela, Comparison of image-based and time-of- [49] I.K. Brilakis, L. Soibelman, Shape-based retrieval of construction site
flight-based technologies for three-dimensional reconstruction of photographs, J. Comput. Civil Eng. 22 (2008) 14–20.
infrastructure, J. Constr. Eng. Manage. 139 (2013) 69–79. [50] Z. Zhu, I. Brilakis, Parameter optimization for automated concrete detection in
[35] L. Klein, N. Li, B. Becerik-Gerber, Imaged-based verification of as-built image data, Autom. Constr. 19 (2010) 944–953.
documentation of operational buildings, Autom. Constr. 21 (2012) 161–171. [51] H. Kim, C.T. Haas, A.F. Rauch, C. Browne, Dimensional ratios for stone
[36] C. Liu, L. Sharan, E.H. Adelson, R. Rosenholtz, Exploring features in a bayesian aggregates from three-dimensional laser scans, J. Comput. Civil Eng. 16
framework for material recognition, in: IEEE Conference on Computer Vision (2002) 175–183.
and Pattern Recognition (CVPR), IEEE, 2010, pp. 239–246. [52] J. MacQueen, et al., Some methods for classification and analysis of
[37] F. Liu, R.W. Picard, Periodicity, directionality, and randomness: wold features multivariate observations, in: Proceedings of the Fifth Berkeley
for image modeling and retrieval, IEEE Trans. Pattern Anal. Mach. Intell. 18 Symposium on Mathematical Statistics and probability, California, USA,
(1996) 722–733. vol. 1, p. 14.
[38] T. Leung, J. Malik, Representing and recognizing the visual appearance of [53] O.G. Cula, K.J. Dana, Compact representation of bidirectional texture functions,
materials using three-dimensional textons, Int. J. Comput. Vis. 43 (2001) 29– in: Proceedings of the IEEE Computer Society Conference on Computer Vision
44. and Pattern Recognition, CVPR, IEEE, vol. 1, 2001, pp. I–1041.
[39] M. Varma, A. Zisserman, A statistical approach to texture classification from [54] C. Cortes, V. Vapnik, Support-vector networks, Mach. Learn. 20 (1995) 273–
single images, Int. J. Comput. Vis. 62 (2005) 61–81. 297.
[40] C. Schmid, Constructing models for content-based image retrieval, in: [55] C.-C. Chang, C.-J. Lin, Libsvm: a library for support vector machines, ACM
Proceedings of the IEEE Computer Society Conference on Computer Vision Trans. Intell. Syst. Technol. (TIST) 2 (2011) 27.
and Pattern Recognition, CVPR, IEEE, vol. 2, 2001, pp. II–39. [56] S. Maji, A. C. Berg, J. Malik, Classification using intersection kernel support
[41] J.-M. Geusebroek, A.W. Smeulders, J. Van De Weijer, Fast anisotropic gauss vector machines is efficient, in: IEEE Conference on Computer Vision and
filtering, IEEE Trans. Image Process. 12 (2003) 938–943. Pattern Recognition, CVPR, IEEE, 2008, pp. 1–8.
[42] S. Lazebnik, C. Schmid, J. Ponce, A discriminative framework for texture and [57] L.R. Rabiner, B. Gold, Theory and application of digital signal processing,
object recognition using local image features, in: Toward Category-Level Prentice-Hall, Inc., Englewood Cliffs, NJ, 1975. p. 1, 777.
Object Recognition, Springer, 2006, pp. 423–442. [58] A.A. Efros, W.T. Freeman, Image quilting for texture synthesis and transfer, in:
[43] M. Varma, A. Zisserman, A statistical approach to material classification using Proceedings of the 28th annual conference on Computer graphics and
image patch exemplars, IEEE Trans. Pattern Anal. Mach. Intell. 31 (2009) 2032– interactive techniques, ACM, pp. 341–346.
2047.