Developments in the Built Environment 4 (2020) 100011

Contents lists available at ScienceDirect

Developments in the Built Environment

journal homepage: www.editorialmanager.com/dibe/default.aspx

Building Information Modelling, Artificial Intelligence and

Construction Tech
Rafael Sacks, PhD a, b, *, Mark Girolami, PhD b, Ioannis Brilakis, PhD b
a
Seskin Virtual Construction Lab, Faculty of Civil and Environmental Engineering, Technion - Israel Institute of Technology, Haifa 32000, Israel
b
Division of Civil Engineering, Department of Engineering, University of Cambridge, 7a JJ Thomson Avenue, Cambridge CB3 0FA, UK

A R T I C L E I N F O A B S T R A C T

Keywords: Adoption of digital information tools in the construction sector provides fertile ground for the birth and growth of
Artificial intelligence companies that specialize in applications of technologies to design and construction. While some of the tech-
Building information modelling nologies are new, many implement ideas proposed in construction research decades ago that were impractical
Construction technology
without a sound digital building information foundation. Building Information Modelling (BIM) itself can be
Digital construction
Digital twins
traced to a landmark paper from 1975; ideas for artificially intelligent design and code checking tools date from
Innovation the mid-1980s; and construction robots have laboured in research labs for decades. Yet only within the past five
years has venture capital actively sought startup companies in the ‘Construction Tech’ sector. Following a set of
digital construction innovations through their known past and their uncertain present, we review their increas-
ingly optimistic future, all through the lens of their dependence on digital information. The review identifies new
challenges, yielding a set of research topics with the potential to unlock a range of future applications that apply
artificial intelligence.

1. Introduction combination of robotics and CAD-CAM providing the basis for entirely new
building systems—the construction of the future.” Construction robotic ma-
Researchers in architecture, engineering and construction have long chines of the first type are only now beginning to become practical and
dreamed of applying information technology, robotics and other new economical, and none have achieved the revolutionary change they
technologies to design and construction. Yet invariably, their conceptual contemplated in their second mode. For many researchers with foresight
understandings of what could be done, and hence their visionary views of and a good conceptual grasp of potential implementations, automation in
the future of construction, far outstripped the practical, technical, com- construction has at times proved to be a frustratingly difficult goal from
mercial, cultural and/or organizational constraints that had to be over- the point of view of implementation in industry.
come for their fulfilment. Within the last five years, however, there has been a steady influx of
Eastman, for example, conceived of a computerised Building Design new, innovative companies specializing in application of a variety of
System (BDS) with all the functionality of what we now know as Building information and automation technologies, developed in other industries,
Information Modelling (BIM) (Eastman, 1975). The basic BIM functions to construction. These startup companies are supported by venture cap-
took 25 years to reach the market, and some – such as Eastman's pre- italists, academic research and public and private incubator programs,
diction that “Later, one can conceive of a BDS supporting automated building together with which they form an ecosystem commonly called ‘Con-
code checking in city hall or the architect's office” – have yet to be realized in struction Tech’ (echoing the name ‘High Tech’ used for the information
full. Indeed, Gholizadeh et al. (2018) found that, as late as 2017, of the 14 and automation technology industry). In the US, the amount of venture
BIM functions whose adoption they investigated, only three were in capital invested in Construction Tech annually is reported to have grown
widespread use. Similarly, Warszawski and Sangrey (1985) wrote that from circa $250 m in 2013 to well over $1,000 m in 2018 (Andersen and
“Implementing robotics in construction may follow several paths. One Forr, 2018). Most of the new companies owe their newfound practicality
approach will involve an evolution of robotic and computer technology into not only to the maturation of their core technologies, but equally to the
existing procedures. The second approach will be more dramatic with the comprehensive building information available in BIM environments. In

* Corresponding author. Seskin Virtual Construction Lab, Faculty of Civil and Environmental Engineering, Technion - Israel Institute of Technology, Haifa 32000,
Israel.
E-mail addresses: [email protected] (R. Sacks), [email protected] (M. Girolami), [email protected] (I. Brilakis).

https://doi.org/10.1016/j.dibe.2020.100011
Received 15 March 2020; Received in revised form 4 May 2020; Accepted 11 May 2020
Available online 16 May 2020
2666-1659/© 2020 The Author(s). Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-
nc-nd/4.0/).
R. Sacks et al. Developments in the Built Environment 4 (2020) 100011

this context, a BIM environment encompasses the various BIM tools, tools and processes are not yet able to support most technology in-
platforms, servers, libraries and workflows within a project or across an novations “out of the box”. Our goal in this paper, therefore, is to explore
organization (for definitions of the terms, see Sacks et al., 2018, Chapter and elaborate the key aspects of the information and data processing
2.3). In this paper, we use the term BIM in its broadest sense, to include barriers to Construction Tech, with special focus on the aspects of BIM
all software applications that generate, manipulate, store and deliver environments and Artificial Intelligence (AI) that require basic research
building information, rather than restricting it to the narrow set of model to underpin commercial development.
authoring platforms. In the following section, we review research and development trends
Building information incorporates both the building product designs in the field of construction over the past half-century, with examples of
and the construction process plans. BIM environments provide this in- R&D for automated code-compliance checking, construction layout,
formation, and it is the foundation upon which the new Construction construction robotics and automated project performance monitoring
Tech applications build and deliver value. Broadly speaking, there are and control. We then consider the last five years, leading up to the pre-
four types of Construction Tech applications: sent – the growth of Construction Tech based on BIM, field monitoring,
robotics, AI and other technologies. In this context, we propose a con-
1. Software tools for design and construction management, most of ceptual ‘House of Construction Tech’ framework that explains how the
which function within or in close relationship with model authoring different theoretical, technical, commercial and conceptual foundations
platforms. underpin the growth of innovative Construction Tech startup companies.
2. Software and hardware systems for delivering information from the Finally, we present and discuss four foundational challenges for the
design to the field – we call this group BIM-to-field tools. This group research community.
includes tools that deliver information via users' mobile devices as
well as tools that deliver product and process information directly 2. Known past: research and development trends
onto workspaces in the field (such as the site laser projection solution
shown in Fig. 1). Following the research and development (R&D) history of three
3. Robotic applications for executing construction operations on site. broad areas of technological innovation in construction, we trace their
4. Software and hardware systems for gathering information from the paths to the present day. Our goal is not to extrapolate into the future, but
site and delivering it to controlling functions – we call this group to identify key research challenges for continued development – to
field-to-BIM tools. These Construction Tech solutions provide identify the essential R in R&D. The areas selected for review represent
copious amounts of data, but that data is of no value in and of itself – three of the four applications types listed in the introduction: (1) software
one must compare the planned to the actual conditions to derive tools for design and planning within BIM environments; (2) BIM-to-field
valuable information, as shown in Fig. 2. tools; and (4) field-to-BIM tools, which are beginning to enable digital
twins for construction.
Naturally, some applications will have more than one use – an app
that delivers process information to a user's mobile device may also so- 2.1. Automated design and code-compliance checking
licit information to update current status, for example.
BIM is an integrating technology that provides an information back- Automating design and code-compliance checking for building con-
bone that transcends organizational boundaries within projects (Sacks struction has been a goal of research and commercial development since
et al., 2018). As such, BIM environments support Construction Tech in- the ideas were floated in Eastman's landmark 1975 paper envisaging BIM
novations by mediating the gap between the information intensive (Eastman, 1975). In the absence of BIM, researchers proposed
technology innovations and the traditionally information poor and stand-alone expert systems (Hayes-Roth et al., 1983), and later, systems
fragmented construction project organisations. However, BIM platforms, that used CAD drawings to represent the buildings. The former, such as
HI-RISE for preliminary structural design of tall buildings (Maher and
Fenves, 1985), SPEX for sizing structural cross-sections (Garrett and
Fenves, 1987) and EIDOCC for design of reinforced concrete columns
(Sacks and Buyukozturk, 1987), used symbolic AI methods. These were
typically rule-based systems that sought to elicit expert knowledge,
capture it in design software, and apply it to automate or to review
design. The need to input building designs explicitly and completely for
each analysis, the limitations on knowledge elicitation, and the capacity
of the computing technology made these systems impractical for com-
mercial application. The advent of 2D CAD did not improve matters
much, because CAD's graphic representations of design are fundamen-
tally different to the semantic, object-oriented representations required
for processing rules.
Initial optimism that design standards themselves could be expressed
as rules and applied to evaluate building designs (e.g. Hakim and Garrett,
1993) proved unfounded, as experiments revealed the challenges posed
by the lexical and logical complexity of building code provisions (Kilic-
cote and Garrett, 1995). Later, natural language processing (NLP) was
applied to building design codes and regulations, resulting in some
progress, but not in commercial application (Song et al., 2018; Zhang and
El-Gohary, 2017).
With the introduction and adoption of BIM, automated design and
code-checking became more practical. Commercial model checking sys-
tems with limited but valuable and viable functionality were developed
(examples include Solibri Model Checker, BIM Assure and SMARTre-
Fig. 1. Projection of a partition layout directly onto a concrete slab. Image view). While they are able to use BIM models, they require users to
courtesy of Lightyx Inc. normalize model data before use, and the repertoire of code clauses they

2
R. Sacks et al. Developments in the Built Environment 4 (2020) 100011

Fig. 2. Image captured with 360
cameras at left and a view of the same scene in a BIM model at right. Image courtesy of Openspace and Lee Kennedy Construction.

can check is limited to clauses that can be expressed as symbolic IF-THEN Mapping (SLAM) and projects images containing the information
rule sets (normalization is the task of pre-processing a BIM model for onto the work surface. It calibrates the projection keystone correction
symbolic code-checking. Users manually add or edit objects, parameter parameters using image analysis.
values and relationships to conform to the naming conventions and ob-
ject typing required by the rule sets). With the growing capabilities of laser scanning and imaging tech-
nologies, improved accuracy of localization, and sophisticated projection
tools, this area appears to offer opportunities for rapid development of
2.2. Construction set out new commercially viable tools. The Lightyx system depicted in Fig. 1 is a
good example of a startup development path for Construction Tech in
Setting out construction work on site is laborious and error prone. The which innovators with expertise from other industries apply their
challenge is to interpret design information within the context of knowledge of advanced technologies to solve construction problems. It is
partially completed scenes and apply physical markings to surfaces with also an example of an innovation that fits entirely within current con-
the required precision. The state-of-the-art method is robotic total station struction practice, automating an isolated operation.
survey layout, in which an operator localizes the total station using
known points in the scene and then ‘shoots’ a laser beam to locate layout
points. This is followed by manual mapping of more complete design 2.3. Automated project performance monitoring and control
information from the points onto other surfaces using chalk-lines, laser
plane projectors, and other tools. The concept of automated project performance control (APPC) was
The scale of direct effort and subsequent rework in case of error have proposed as a way to provide managers with the real-time feedback
prompted R&D efforts to build automated layout systems that deliver necessary for application of the ‘thermostat’ model of control (Navon,
BIM information directly to the field. Three types have been proposed: 2005). The idea was to close the control loop by reporting leading per-
formance indicators, such as labour and equipment productivity by
a) Augmented reality (AR) systems, in which an image of the intended monitoring the movements of workers and materials in real-time (Navon
design is superimposed onto an image of the site recorded with a and Sacks, 2007; Sacks et al., 2006). This line of research might have
camera (Chi et al., 2013; Wang et al., 2013; Woodward et al., 2014). presaged the new concept of ‘Construction 4.0’ or of digital twins for
AR systems require special glasses or masks, or a tablet computer or construction, but it encountered technical and conceptual barriers:
other device on which the images are projected. Users must then
translate that information onto the work surface. These systems are - From a technical standpoint, there was no platform available to
particularly useful for locating hidden system objects behind finished integrate the necessary production process information for compari-
surfaces for building or facility operation and maintenance tasks (Lee son with monitored data. Researchers developed sophisticated
and Akin, 2011; Sacks et al., 2018). methods to extract information from construction site documents and
b) Robotic marking systems, in which a robot localizes itself and then images (Al Qady and Kandil, 2010; Brilakis et al., 2005), but these
travels the work area applying paint or other marking material were not linked to any integrated data management system.
directly onto the surface (Casale, 2013; Prouty, 2013). These are - From a conceptual standpoint, the thermostat model proved to be
generally restricted to environments where the floors are clean and inappropriate and ineffective for planning and controlling production
clear, for marking and for travel, and where the quantity of layout in construction, and it has been replaced over time with methods
work is large enough to justify their setup costs. based on pull production planning and control (Ballard, 2008; Kenley
c) Robotic systems that project BIM information directly onto the work and Sepp€anen, 2010).
surface. For example, Degani et al. (2019) developed a prototype in
which images from a BIM model are projected directly onto a work The notion of automating monitoring work on site originated from
surface. The apparatus consisted of a laser range scanner, an angled the observation that engineers in the field spent a lot of time collecting
adjustable projector, and a camera. The system localizes itself using performance data (McCullouch, 1992; McCullouch and Lueprasert,
the LIDAR and the BIM model using Simultaneous Localization and 1994). A variety of technologies have been proposed for data collection,

3
R. Sacks et al. Developments in the Built Environment 4 (2020) 100011

including computer vision (Brilakis and Haas, 2020), GPS, laser scanning, Despite ten years or more of industrial R&D, there are not yet
radio-frequency ID tags and Bluetooth low energy (Bekkelien et al., 2012; commercially viable solutions for setting out directly onto work surfaces.
Costin et al., 2012). Yet except for systems for monitoring heavy earth- One of the key challenges is to project or mark information on irregular,
works machinery, field-to-BIM automation has not been adopted in the intermediate as-built work surfaces in the real world, because they do not
construction market. correspond directly to the ideal as-designed surfaces of finished products
that exist in the virtual world of BIM models. Ironically, this problem
3. Uncertain present: BIM, AI and Construction Tech might be overcome if the ‘field-to-BIM’ technologies were able to build
accurate, virtual digital twin representations of site conditions. Yet this
A common thread runs through all three examples of Construction too remains a challenge, as we describe next.
Tech innovations detailed in the previous section: dependence on the
availability of digital building information. With the benefit of hindsight, 3.3. Automated project performance monitoring and control
we observe that it is not simply a matter of digitising information, but one
of making information freely available across BIM platforms, with both This area is rife with solutions offered by both established software
syntactic and semantic interoperability. The current lack of direct access and hardware vendors and startup companies. Applications range from
inhibits the application of AI. In this section we review the current status (i) inspection systems, allowing inspectors access to data before, during
of BIM and AI in the construction industry (circa 2020), with reference to and after the inspection process and access to recording functionality to
the same three Construction Tech application areas. collect site data, to (ii) control systems, that enable the control of safety,
site traffic, resource and storage utilisation, and others, to (iii) planning
3.1. Automated design and code-compliance checking and measuring systems, for site logistics and layout planning, production
monitoring, and others. Some investment has gone into this space; yet all
Although the advent of BIM has made commercial code-checking applications are single track, functioning as information islands. They use
applications viable, their core technology has not changed fundamen- one or few data acquisition technologies and interpret that as best they
tally from that envisaged in the 1980's. They all use symbolic AI methods, can into useful construction management information (e.g. Siteaware,
primarily rule-inferencing, which restricts their scope to relatively simple Disperse.io, Holobuilder, Smartvid.io, Versatile Natures, Openspace.ai,
prescriptive clauses (Bloch et al., 2019). The challenge posed by the large Genda, and others). The information they provide is not always reliable
numbers of applicable design and building codes, and the frequency with and needs manual review and intervention, which often invalidates their
which they are updated, has not been solved (Nawari, 2017). The com- automation-borne benefits. Their limited approach also limits what
mercial applications still require explicit representation of the building conclusions can be drawn.
information (Dimyadi et al., 2016), and the effort required for normali- Essentially, there is a need for complex event processing (Buchmann
zation limits their use to isolated milestone points in design processes. and Koldehofe, 2009). This would entail the merging of information from
Breakthrough progress in code-checking will require overcoming multiple monitored data sources with already existing information about
these barriers, and new approaches and technologies will be needed. the as-is status of a construction site and the production plan, to deduce
Among the most promising: accurate information about what has been built, how, and what resources
were used and where merging/fusing data from multiple sources to
 Semantic enrichment of BIM models, using AI methods to automati- compile comprehensive information about project status (in terms of
cally supplement models with explicit information derived using al- both product and process status). However, complex event processing is
gorithms trained to recognize and infer predefined sets of target only possible on the basis of well-integrated and reliable data, something
concepts within patterns of building data, may offer a way to remove we still lack. Although data is available in apparent abundance, the
the need for normalization (Belsky et al., 2016). current lack of comprehensive, accurate and reliable linked data and
 Application of machine learning algorithms to evaluate designs on the information naturally restricts the opportunities to properly exploit the
basis of the training data of known results from human experts (Sacks technologies.
et al., 2019).
 Graph representations of BIM models may offer the explicit repre- 4. Optimistic future: The House of Construction Tech
sentations needed, in particular for making the relationships between
building objects and abstract concepts explicit (Nahar, 2017). They In theory, BIM models of buildings and infrastructure are ideally
are also more amenable to the types of pattern recognition algorithms suited to manipulation by smart software tools that incorporate computer
that may enable semantic enrichment (Jin et al., 2018) and training of vision, rule-inferencing, machine learning, case-based reasoning and
ML algorithms. other AI strategies. The range of potential applications is wide, including
– but by no means limited to – smart tools for:
We note also that all of the companies offering commercial BIM code-
checking applications are startup companies (Compliance Audit Systems  Design support and/or automation, topology optimization, genera-
Limited, Daima, Invicara, SMARTreview) or began as startup companies tive design.
(Solibri, recently acquired by Nemetschek).  Design review, checking compliance to standards and codes.
 Building performance simulations and engineering analyses.
3.2. Construction set out  Construction planning, site layout design, supply chain management.
 Digital delivery of design and construction method directly to
BIM-to-field information delivery has largely been solved with regard workers on site.
to delivering BIM information to personnel via mobile computing de-  Real-time measurement, assessment and interpretation of project
vices. All the major BIM platform companies offer solutions, most of status.
which originated with disruptive Construction Tech startup companies  Quality assurance and control.
whose solutions were acquired by the established BIM companies (e.g.  Production control, resource assignment, material and information
PlanGrid, Trimble Connect, Solibri). Tools that present model informa- flow control.
tion using augmented reality are also available (such as Trimble's XR10  Safety planning and control.
with Hololens 2; Trimble, 2020), although these still suffer from practical  Sustainability and life-cycle costing assessments.
problems such as narrow fields of vision, indistinct display in bright  Acquisition of BIM models from point cloud data, photo- and video-
environments, encumbering workers, etc. grammetry.

4
R. Sacks et al. Developments in the Built Environment 4 (2020) 100011

 Facility operations and management using digital twins.

Researchers of computing in the Architecture, Engineering and Con-

struction (AEC) industry across the world have sought to realize such
tools since the ideas behind AI developed. In early efforts in the 1980's
and 1990's, people attempted to apply expert systems and case-based
reasoning to some of the tasks listed above. It soon became apparent
that CAD technology was not suited to such applications because its
representation of building information was graphic and symbolic, rather
than object-oriented. This led to an intense effort to solve the represen-
tation challenge, which resulted in the BIM model authoring platforms
that are now ubiquitous across the industry, and in an open object-
oriented schema for representing buildings and infrastructure (the IFC
data model) (ISO, 2013).
With the information representation challenge apparently solved, the
stage appeared set for commercial implementation of innovations, and a
wave of technological innovation began. As Andersen and Forr (2018)
and other reviewers (Azevedo, 2019; Blanco et al., 2017) of these de-
velopments have noted, the majority of innovation financing has been
provided to startup companies. There are two main reasons for this: Fig. 3. Conceptual ‘House of Construction Tech’ model.
corporate/organizational fragmentation within the industry, and the
need for expert knowledge and experience with technologies adapted
four application types (design and management, BIM-to-field, field
from other industrial domains.
automation, and field-to-BIM) are dependent on information in one form
Hall et al. (2019) provide compelling evidence of innovations that
or another. The maturation of BIM environments and their broad adop-
develop outside construction project organisations due to fragmentation
tion is the one key common denominator supporting the growth of
of the industry: vertical fragmentation (professional and trade speciali-
Construction Tech within the last decade. Equally, however, BIM tech-
zation), horizontal fragmentation (multiple small firms competing with
nology and processes still have severe limitations that constrain the long-
one another), and longitudinal fragmentation (high turnover of suppliers
promised growth and success of some of the Construction Tech applica-
and clients from project to project). In this environment, systemic in-
tions. Among the key limitations: inadequate interoperability of infor-
novations tend to disrupt existing commercial or organizational bound-
mation, difficulties in framing model data for machine learning
aries and therefore require wholly new vertically and longitudinally
applications for design and management, and the need for an intelligent
integrated organisations, with high startup costs and significant risk
digital twin platform to support integration of field-to-BIM tools.
(Katila et al., 2018). Within this context, it is not surprising that many
The House of Construction Tech must be underpinned by a compre-
Construction Tech innovators fail to overcome the regulatory, commer-
hensive understanding of the theoretical aspects of design, of information
cial, cultural, organizational and technological barriers (Chowdhury et
and data science, and of production in construction. Design theory en-
al., 2019), despite inventing and developing cutting edge technology
compasses design cognition (Winograd and Flores, 1986), philosophy of
applications in the construction domain. Given these risks, almost all the
design (Galle, 2002), and design strategies such as top-down knowl-
innovators adopt an incremental approach to change in the construction
edge-based design (Mitchell et al., 1990; Sacks et al., 2003) and design
industry, as their top priority is to achieve a minimal viable product and
optimization (Gero, 2012). Information and data science include
being to generate income.
methods for representing building information (Braid, 1973; Sacks et al.,
In addition to the fundamentals of entrepreneurship (ideas, invest-
2004; Turk et al., 1994) and methods for artificially intelligent processing
ment and implementation), all Construction Tech innovators require at
of data, including machine learning and pattern recognition (Bishop,
least three essential things: 1) a real process need in the industry, 2) an
2016; Efron and Hastie, 2016; Rogers and Girolami, 2017). Production
application of a new technology that fulfils the need, and 3) a workable
theory concerns our understanding of the products, the processes and the
business model. These are the pillars of the ‘House of Construction Tech’,
operations in the context of construction (Ballard, 2000; Koskela, 2000;
which we propose as a model to explain the components essential for
Sacks, 2016).
success in the sector (shown in Fig. 3). Entrepreneurship provides the
The ‘House of Construction Tech’ can serve as a ‘checklist’ for con-
beams that support the roof, which is the pinnacle of success – adoption
struction startup companies, and as a predictor of success or failure, by
in the construction industry market. The BIM environment, in its
considering whether a company has successfully incorporated the col-
broadest sense as technology, process and people, sits at the base of the
umns, base and foundations. At the foundation level, for example, an AI
house. BIM technology is the hardware and the software that generate
tool for automated construction scheduling using machine learning
and store the information about a construction project, including its
cannot provide real value for construction managers if its authors restrict
physical aspects (a building's design) and its process aspects (construc-
their tool to master planning using the Critical Path Method, ignoring the
tion plans). BIM processes are the information management aspects –
conceptualisation of production in construction as flows of work, prod-
standards, such as ISO 19650 (ISO, 2018) and IFC (ISO, 2013); organi-
ucts and resources that underpins essential more detailed layers of
zation and project level BIM execution plans; level of detail (LOD) defi-
planning (Koskela, 2000). At the base level, applications that use 2D
nitions; etc. The people are those capable of implementing the processes
printed drawings rather than BIM models as their main input will find
using the technology, including not only employees of the innovator
their scope severely limited. A company for whose innovation these as-
(designers, programmers, etc.), but no less important, employees of the
pects are relevant and yet chooses to ignore them, is unlikely to succeed
customers (architects, engineers, and construction managers) skilled in
in the long run. At the level of the columns, innovators must identify
working within BIM environments.
business process need to avoid the common trap of solutions looking for
Building information in a form that can be readily manipulated by
problems. For example, proponents of a virtual reality telepresence
software is essential for almost all Construction Tech innovations, and
technology must identify the business process use case that will underlie
hence placement of the BIM environment at the base of the house. All
market demand for their solution before developing the application.

5
R. Sacks et al. Developments in the Built Environment 4 (2020) 100011

5. Realistic future: foundational research challenges 5.1. Semantic enrichment

While the outlook for innovation in Construction Tech is promising, Semantic enrichment of BIM models is a process in which algorithms
there are still challenges to be met. Some of the technological constraints apply expert domain knowledge to infer any and all information needed
identified in the sections above, that describe the known past and the for a given specific application that is absent from the explicit data in the
uncertain present, reveal that the information representation is still model (Belsky et al., 2016). Fig. 4 illustrates the idea that models are
inadequate. Significant foundational challenges remain to be solved incomplete representations of buildings. The explicit set of geometry,
before Construction Tech innovators can begin broad incorporation of AI properties and relationships in a model is insufficient input for most
techniques within smart software tools that manipulate BIM models. One receiving applications from sub-domains other than the one in which it
of the key problems is that models are commonly discipline-specific, was generated. The information inferred can be added to the model,
representing buildings with the semantics of a single professional view enriching it and facilitating its use for any given receiving application.
(such as architecture, structural engineering, or MEP systems). As such, Semantic enrichment encompasses classification of building objects,
multi-disciplinary collaboration using models is difficult, with most aggregation and grouping of objects in functional groups or systems,
teams using federations of separate models. Another key problem is that unique identification of building elements, generation of missing objects,
many object relationships and properties are still implicit in BIM models, properties and relationships, and reconstruction of occluded objects in
left to the intelligent interpretation of their human users. A third problem the case of application to models compiled from point cloud data (PCD)
is that even where information is complete, the object-oriented repre- (Sacks et al., 2017).
sentations are not suited to the representations of objects and their fea- Semantic enrichment is by no means a mature technology. It is a
tures that are the standard input of existing AI techniques. Similarly, relatively new area of research and the literature on the subject is limited.
design specifications and building codes commonly define parameters Much work has been done toward intelligent semantic query of BIM
that are complex compilations of geometric and other constraints that are models (Borrmann et al., 2006; Mazairac and Beetz, 2013; Wülfing et al.,
very difficult to express using if-then rule sets. 2014), and these efforts have begun to exploit the meaningful topological
These problems are common to all aspects of tool development for the constructs of information that is implied in models, but not stored
applications discussed above. Some of the problems can be solved explicitly. They use the implicit meaning, but no information is added to
through semantic enrichment, in which professional knowledge is enrich the model during their operations.
applied to infer the missing information from the purpose-specific Semantic enrichment software modules are goal driven. The missing
viewpoint of the smart tools receiving data from models. In general or misrepresented information that is to be supplemented is defined in a
terms, this would make the information complete. Others require devel- domain specific Model View Definition (MVD). The first experimental
opment of data engineering strategies to make the information compatible applications applied rule inferencing, a subset of AI methods in which the
for AI processing. system logic is defined as a set of rules, usually in ‘IF-THEN’ form. In
At an implementation level, we identify two distinct research chal- forward-chaining solutions, rules are processed iteratively, adding new
lenges for development of foundational information processing methods information to a model whenever a rule is evaluated as TRUE. Iteration
for BIM models which, if solved, would greatly facilitate development of ends when no new facts can be deduced. Backward-chaining is also
a wide range of smart BIM and AI tools for design and construction: possible, where rules are run to verify hypotheses, seeking evidence
within the existing information. More recent research has shown that
1. Combined, optimal use of topological rule inferencing and machine some aspects of semantic enrichment may be better performed using
learning modules for semantic enrichment machine learning. For example, a supervised machine learning algorithm
2. Representations of BIM models suitable for AI applications, with was successfully applied to classify room types in residential apartments
emphasis on machine learning (Bloch and Sacks, 2018). Unsupervised machine learning has been pre-
viously applied to BIM models to detect anomalies and misclassified

Fig. 4. Explicit and implicit information in BIM models. Each sub-domain (e.g. structural analysis, material take-off, construction detailing) has its own set of im-
plicit concepts.

6
R. Sacks et al. Developments in the Built Environment 4 (2020) 100011

objects (Krijnen and Tamke, 2015). The discrepancies in an IFC based buildings for applications that require machine learning and/or pattern
data exchange have also been tackled by application of machine learning matching.
algorithms such as Support Vector Machine (SVM) (Koo et al., 2019; Koo
and Shin, 2018) to detect misclassifications of specific objects in IFC files. 6. Conclusion
Similarly, Wu and Zhang (2019) applied learning algorithms to identify
common building objects in IFC files. A review of three specific areas of Construction Tech, representing
Basic research is needed to classify BIM information objects according design and planning, BIM-to-field and field-to-BIM applications, reveals
to their suitability for semantic enrichment using different AI methods that the broad adoption of BIM environments in the construction industry
(primarily topological rule inferencing, supervised machine learning or is an insufficient condition to enable effective exploitation of the infor-
deep learning) and to design suites of semantic enrichment software mation they contain, or to leverage the potential of AI in this context. The
methods that could be compiled to build applications in modular fashion. problem is that the information in models is incomplete and inaccessible.
If these goals could be achieved, the basic problem of BIM interopera- Among the many technological challenges facing Construction Tech en-
bility might be solved, as standard models could be enriched for almost trepreneurs, we have identified two specific research challenges that
any purpose. This would provide the technological foundation for Con- concern development of foundational information processing methods
struction Tech startups to implement a wide range of applications (some for digital building information models which, if solved, would greatly
of which are listed in the Conclusions section below). facilitate development of smart BIM and AI tools for design and con-
struction. They are:
5.2. BIM model representations for learning applications
1. Combined, optimal use of topological rule inferencing and machine
Building models are stored in one of three ways – in proprietary file learning modules for semantic enrichment
formats specific to particular BIM authoring platforms, in open IFC 2. Encoding representations of building information in forms that are
format files based on EXPRESS (ISO, 2004), or more recently, in pro- amenable to machine learning
prietary cloud databases. None of these formats are directly compatible
with pattern recognition and/or machine learning algorithms. In all With regard to the nature of innovation in Construction Tech, our
existing applications of AI that use BIM information, whether as complete review of the areas of application supports researchers' predictions that
models, partial assemblies from models, or model components, users technology innovation in construction is more likely to stem from
must extract and compile the relevant information anew for every use. disruptive startup companies than from the traditional project oriented
Extraction generally requires export of tabular schedules of objects and construction companies (e.g. Katila et al., 2018). The growth of invest-
their properties or parsing of IFC files. ment in Construction Tech startup companies demonstrates that the
A core problem with these methods is that meaningful information is market shares this view. The ‘House of Construction Tech’ model may
lost in translation. Perhaps most significant is the loss of relationships help investors and innovators alike in evaluating the soundness of their
between objects that are available in the native BIM software represen- startups' technology and business strategies. Note that the model is
tation, but that cannot be exported easily in tabular schedules and are applicable to incremental innovation; a complete rethinking of the con-
generally absent entirely from IFC export files. Examples are the struction business model may require rethinking of the foundational
embedding relationships between windows and walls and the structural technologies too.
support connectivity relationships between beams and columns. The Naturally, we cannot claim to have identified all possible techno-
result is that solutions either use the narrow aspects of the features of logical challenges to implementation of AI and BIM applications. There
building objects in isolation, or that extensive programming is needed to may be others, and presumably new problems will arise even as so-
extract and express the building object relationships as features of indi- lutions to semantic enrichment and graph representations of BIM
vidual objects. models are developed and implemented. We are confident, however,
This is a severe limitation when one considers that the networks of that these two are key to progress, and thus deserving of the attention
relationships between building objects form quite distinct patterns. For of researchers.
example, the rule-based methods developed to classify bridge compo-
nents in the SeeBridge project relied heavily on topological relationships Declaration of competing interest
between the objects. In that case the information was parsed from IFC
files and the relationships were inferred by the rule processing system The authors declare that they have no known competing financial
from the physical locations of the objects relative to one another before interests or personal relationships that could have appeared to influence
they could be classified (Sacks et al., 2017). the work reported in this paper.
The difficulty in expressing the model objects and their relationships
also points to a promising solution – expression of building models as Acknowledgement
property graphs. In pioneering work, Khalili and Chua (2015) developed
a set of tools to compile a graph data model from an IFC file of a building This work was supported by the Centre for Digital Built Britain, a
model. Their method evaluated the geometry of the building objects to partnership between the University of Cambridge and the Department for
determine topological relationships and supplement the graph model Business, Energy and Industrial Strategy, and the Construction Innova-
with connectivity, containment, separation, and intersection relation- tion Hub. The funding was provided through the Government’s modern
ships, which were modelled as weights on the graph's edges. Ismail et al. industrial strategy by Innovate UK, part of UK Research and Innovation.
(2017) used a more suitable property graph representation, compiling
both an IFC schema graph from EXPRESS and IFC model graphs for in- References
dividual instances of building models stored in IFC files. They did not
convert geometry, nor did they infer topological relationships from the Al Qady, M., Kandil, A., 2010. Concept relation extraction from construction documents
using natural language processing. J. Construct. Eng. Manag. 136 (3), 294–302.
geometry, sufficing with expressing information contained explicitly in
Andersen, K., Forr, T., 2018. The State of Construction Technology. Jones Lang LaSalle IP,
the IFC files. Although neither of these attempted semantic enrichment Inc., p. 12
using domain specific knowledge beyond the five basic topological re- Azevedo, M.A., 2019. Investor Momentum Builds for Construction Tech. Crunchbase News,
lationships (Nguyen and Oloufa, 2001), the graph representations and San Francisco, CA.
Ballard, G., 2000. The Last Planner System of Production Control. PhD Dissertation.
the use cases they demonstrated with those representations strongly University of Birmingham, Birmingham, U.K.
support the idea that property graphs are appropriate for modelling Ballard, G., 2008. The lean project delivery system: an update. Lean Constr. J. 2008, 1–19.

7
R. Sacks et al. Developments in the Built Environment 4 (2020) 100011

Bekkelien, A., Deriaz, M., Marchand-Maillet, S., 2012. “Bluetooth Indoor positioning.” Khalili, A., Chua, D.K.H., 2015. IFC-based graph data model for topological queries on
Master's Thesis. University of Geneva. building elements. J. Comput. Civ. Eng. 29 (3), 04014046.
Belsky, M., Sacks, R., Brilakis, I., 2016. Semantic enrichment for building information Kiliccote, H., Garrett, J.H., 1995. “Obstacles to the Development of Computable Models of
modeling. Comput. Aided Civ. Infrastruct. Eng. 31 (4), 261–274. https://doi.org/ Design standards.” Workshop on Computing and Information in Construction: Modeling of
10.1111/mice.12128. Buildings Through Their Lifecycle. Stanford University, Stanford, CA.
Bishop, C.M., 2016. Pattern Recognition and Machine Learning. Information Science and Koo, B., Shin, B., 2018. Applying novelty detection to identify model element to IFC class
Statistics. Springer, New York, New York, NY. misclassifications on architectural and infrastructure Building Information Models.
Blanco, J.L., Mullin, A., Pandya, K., Sridhar, M., 2017. The New Age of Engineering and J. Comput. design Eng. 5 (4), 391–400.
Construction Technology. McKinsey & Company, Philadelphia, PA. Koo, B., La, S., Cho, N.-W., Yu, Y., 2019. Using support vector machines to classify
Bloch, T., Katz, M., Yosef, R., Sacks, R., 2019. Automated Model Checking for building elements for checking the semantic integrity of building information
Topologically Complex Code Requirements – Security Room Case Study. In: models. Autom. ConStruct. 98, 183–194.
O’Donnell, J. (Ed.), 2019 European Conference on Computing in Construction. EC3, Koskela, Lauri, 2000. An Exploration towards a Production Theory and its Application to
Chania, Crete, Greece. Construction. D. Tech, Helsinki University of Technology, Espoo.
Bloch, T., Sacks, R., 2018. Comparing machine learning and rule-based inferencing for Krijnen, T., Tamke, M., 2015. In: Thomsen, M.R., Tamke, M., Gengnagel, C., Faircloth, B.,
semantic enrichment of BIM models. Autom. ConStruct. 91, 256–272. https:// Scheurer, F. (Eds.), “Assessing Implicit Knowledge in BIM Models with Machine
doi.org/10.1016/j.autcon.2018.03.018. Learning.” Modelling Behaviour. Springer International Publishing, Cham,
Borrmann, A., Van Treeck, C., Rank, E., 2006. Towards a 3D spatial query language for pp. 397–406.
building information models. In: Proc. Joint Int. Conf. of Computing and Decision € 2011. Augmented reality-based computational fieldwork support for
Lee, S., Akin, O.,
Making in Civil and Building Engineering (ICCCBE-XI). equipment operations and maintenance. Autom. ConStruct. 20 (4), 338–352.
Braid, I.C., 1973. Designing with Volumes. Cantab Press, Cambridge University, Maher, M.L., Fenves, S.J., 1985. HI-RISE - an expert system for the preliminary structural
Cambridge UK. design of high rise buildings. In: Gero, J.S. (Ed.), Knowledge Engineering in
Brilakis, I., Haas, C., 2020. Infrastructure Computer Vision. Butterworth-Heinemann, Computer-Aided Design. North-Holland, Amsterdam, pp. 125–135.
Oxford. Mazairac, W., Beetz, J., 2013. “BIMQL ? An open query language for building information
Brilakis, I., Soibelman, L., Shinagawa, Y., 2005. Material-based construction site image models. Adv. Eng. Inf. 27 (4), 444–456.
retrieval. J. Comput. Civ. Eng. 19 (4), 341–355. McCullouch, B.G., 1992. “Automated construction field-data management system.
Buchmann, A., Koldehofe, B., 2009. Complex event processing. IT Inf. Technol. 51 (5). J. Transport. Eng. 118 (4), 517–526.
Casale, D., 2013. Project Lion - A DPR/Trimble Automated Layout Robot. DPR McCullouch, B.G., Lueprasert, K., 1994. “2D bar-code applications in construction.
Construction, Redwood City, CA. https://www.youtube.com/watch?v¼k-lfL3dA1S. J. Construct. Eng. Manag. 120 (4), 739–752.
Chi, H.-L., Kang, S.-C., Wang, X., 2013. Research trends and opportunities of augmented Mitchell, W., Liggett, R., Tan, M., 1990. Top-down knowledge-based design. In:
reality applications in architecture, engineering, and construction. Autom. ConStruct. McCullough, M., Mitchell, W., Purcell, P. (Eds.), The Electronic Design Studio:
33, 116–122. https://doi.org/10.1016/j.autcon.2012.12.017. Architectural Knowledge and Media in the Computer Era. MIT Press, Cambridge, MA,
Chowdhury, T., Adafin, J., Wilkinson, S., 2019. Review of digital technologies to improve pp. 137–148.
productivity of New Zealand construction industry. J. Inf. Technol. Construct. 24, Nahar, A., 2017. “Applying Graph Theory Concepts for Analyzing BIM Models Based on
569–587. https://doi.org/10.36680/j.itcon.2019.032. IFC standards.” Master's Thesis. Technische Universit€at Dresden, Dresden, Germany.
Costin, A., Pradhananga, N., Teizer, J., 2012. Leveraging passive RFID technology for Navon, R., 2005. Automated project performance control of construction projects. Autom.
construction resource field mobility and status monitoring in a high-rise renovation ConStruct. 14 (4), 467–476.
project. Autom. ConStruct. 24, 1–15. Navon, R., Sacks, R., 2007. Assessing research issues in automated project performance
Degani, A., Li, W.B., Sacks, R., Ma, L., 2019. An automated system for projection of control (APPC). Autom. ConStruct. 16, 474–484.
interior construction layouts. IEEE Trans. Autom. Sci. Eng. 16 (4), 1825–1835. Nawari, N.O., 2017. Building Information Modeling: Automated Code Checking and
https://doi.org/10.1109/TASE.2019.2897135. Compliance Processes. CRC Press, Boca Raton.
Dimyadi, J., Pauwels, P., Amor, R., 2016. Modelling and accessing regulatory knowledge Nguyen, T.-H., Oloufa, A.A., 2001. Computer-generated building data: topological
for computer-assisted compliance audit. Spec. Track Compl. Check. 21, 317–336. information. J. Comput. Civ. Eng. 15 (4), 268–274.
ITcon, Special issue CIB W78 2015. Prouty, J. M. (2013). Robotic Construction Site Marking Apparatus. Chula Vista, CA. US
Eastman, C.M., 1975. The use of computers instead of drawings in building design. J. Am. Patent Office Application no. 13/471,883. Filed May 15th 2012. 11 pages.
Inst. Archit. 63 (3), 46–50. Rogers, S., Girolami, M., 2017. A First Course in Machine Learning. Chapman & Hall/CRC
Efron, B., Hastie, T., 2016. Computer Age Statistical Inference: Algorithms, Evidence, and Machine Learning & Pattern Recognition Series. CRC Press, Taylor & Francis Group, a
Data Science. Institute of Mathematical Statistics monographs, Cambridge University Chapman & Hall book, Boca Raton London New York.
Press, New York, NY. Sacks, R., 2016. What constitutes good production flow in construction? Construct.
Galle, P., 2002. Philosophy of design: an editorial introduction. Des. Stud. 23 (3), Manag. Econ. 34 (9), 641–656. https://doi.org/10.1080/01446193.2016.1200733.
211–218. Sacks, R., Buyukozturk, O., 1987. Expert interactive design of concrete columns under Bi-
Garrett, J.H., Fenves, S.J., 1987. A knowledge-based standards processor for structural axial bending. ASCE J. Comp. Civil Eng. 1, 69–81.
component design. Eng. Comput. 2 (4), 219–238. Sacks, R., Eastman, C.M., Lee, G., 2003. Process improvements in precast concrete
Gero, J., 2012. Design Optimization. Elsevier Science. construction using top-down parametric 3-D computer-modeling. J. Prec. Prest. Conc.
Gholizadeh, P., Esmaeili, B., Goodrum, P., 2018. Diffusion of building information Inst. 48, 46–55.
modeling functions in the construction industry. J. Manag. Eng. 34 (2), 04017060. Sacks, R., Eastman, C.M., Lee, G., 2004. Parametric 3D modeling in building construction
Hakim, M., Garrett, J.H., 1993. Using description logic for representing engineering with examples from precast concrete. Autom. ConStruct. 13, 291–312.
design standards. J. Eng. Comp. 9, 108–124. Sacks, R., Eastman, C.M., Lee, G., Teicholz, P., 2018. BIM Handbook: A Guide to Building
Hall, D.M., Whyte, J.K., Lessing, J., 2019. Mirror-breaking strategies to enable digital Information Modeling for Owners, Designers, Engineers, Contractors and Facility
manufacturing in Silicon Valley construction firms: a comparative case study. Managers. John Wiley and Sons, Hoboken, NJ.
Construct. Manag. Econ. 1–18. Sacks, R., Bloch, T., Katz, M., Yosef, R., 2019. Automating design review with artificial
Hayes-Roth, F., Waterman, D.A., Lenat, D.B., 1983. Building Expert Systems. Addison- intelligence and BIM: state of the art and research framework. In: Behzadan, A.H.,
Wesley, Reading, Massachusetts. Cho, Y., Leite, F., Wang, C. (Eds.), Computing in Civil Engineering 2019:
Ismail, A., Nahar, A., Scherer, R., 2017. Application of graph databases and graph theory Visualization, Information Modeling, and Simulation. American Society of Civil
concepts for advanced analysing of BIM models based on IFC standard. In: European Engineers, Atlanta, GA, USA, pp. 353–360.
Group for Intelligent Computing in Engineering, vol. 12. Nottingham, UK. Sacks, R., Ma, L., Yosef, R., Borrmann, A., Daum, S., Kattel, U., 2017. Semantic
ISO 10303-11, 2004. Industrial automation systems and integration — Product data enrichment for building information modeling: procedure for compiling inference
representation and exchange — Part 11: description methods: The EXPRESS rules and operators for complex geometry. J. Comput. Civ. Eng. 31 (6), 04017062
Language Reference Manual. International Standards Organisation. https://www https://doi.org/10.1061/(ASCE)CP.1943-5487.0000705.
.iso.org/standard/57620.html. Sacks, R., Navon, R., Brodetskaia, I., 2006. Interpretation of automatically monitored
ISO, 2013. ISO 16739:2013 Industry Foundation Classes (IFC) for Data Sharing in the equipment data in construction control. J. Comput. Civ. Eng. 20, 111–120. https://
Construction and Facility Management Industries. International Standards doi.org/10.1061/(ASCE)0887-3801(2006)20:2(111).
Organization. Song, J., Kim, J., Lee, J.-K., 2018. NLP and Deep Learning-Based Analysis of Building
ISO, 2018. ISO 19650-1, Organization and Digitization of Information about Buildings Regulations to Support Automated Rule Checking System, 7.
and Civil Engineering Works, Including Building Information Modelling (BIM) — Trimble, 2020. “Trimble XR10 with Hololens 2.” Trimble mixed reality. https://mixedr
Information Management Using Building Information Modelling — Part 1: Concepts eality.trimble.com/.
and Principles. International Standards Organisation. Turk, Z., Isakovic, T., Fischinger, M., 1994. Object-oriented modeling of design system for
Jin, C., Xu, M., Lin, L., Zhou, X., 2018. “Exploring BIM Data by Graph-Based Unsupervised RC buildings. J. Comput. Civ. Eng. 8, 436–453.
Learning:” Proceedings of the 7th International Conference on Pattern Recognition Wang, X., Kim, M.J., Love, P.E.D., Kang, S.-C., 2013. Augmented Reality in built environment:
Applications and Methods. SCITEPRESS - Science and Technology Publications, classification and implications for future research. Autom. ConStruct. 32, 1–13.
Funchal, Madeira, Portugal, pp. 582–589. Warszawski, A., Sangrey, D.A., 1985. Robotics in building construction. J. Construct. Eng.
Katila, R., Levitt, R.E., Sheffer, D., 2018. Systemic innovation of complex one-off products: Manag. 111 (3), 260–280.
the case of green buildings. In: Joseph, J., Baumann, O., Burton, R., Srikanth, K. Winograd, T., Flores, F., 1986. Understanding Computers and Cognition: a New
(Eds.), Advances in Strategic Management. Emerald Publishing Limited, pp. 299–328. Foundation for Design. Intellect Books, Bristol, UK.
Kenley, R., Sepp€ anen, O., 2010. Location-Based Management for Construction: Planning, Woodward, C., Kuula, T., Honkamaa, P., Hakkarainen, M., Kemppi, P., 2014.
Scheduling and Control. Spon Press. Implementation and evaluation of a mobile augmented reality system for building

8
R. Sacks et al. Developments in the Built Environment 4 (2020) 100011

maintenance. In: Dawood, N., Alkass, S. (Eds.), Proceedings of the 14th International Wülfing, A., Windisch, R., Scherer, R., 2014. “A visual BIM query language.” eWork and
Conference on Construction Applications of Virtual Reality, pp. 306–315. Sharjah, eBusiness in Architecture, Engineering and Construction. ECPPM 2014, 157.
UAE. Zhang, J., El-Gohary, N.M., 2017. Integrating semantic NLP and logic reasoning into a
Wu, J., Zhang, J., 2019. New automated BIM object classification method to support BIM unified system for fully-automated code checking. Autom. ConStruct. 73, 45–57.
interoperability. J. Comput. Civ. Eng. 33 (5), 04019033.