A Seminar Report

on

“A GPT-Powered Assistant for Real-Time Interaction

with Building Information Models”

Submitted to the

Savitribai Phule Pune University

In partial fulfillment of Third Year Engineering in

Information Technology
by

Prasad Nanasaheb Paigude

(Roll No.ITTA33)

Under the guidance of

Mrs. P. S. Bangare

Department Of Information Technology

Sinhgad Academy of Engineering, Kondhwa,
(Bk.) Pune-48

2024-2025

1
 CERTIFICATE

This is to certify that the seminar report entitled “A GPT-Powered Assistant for Real-Time
Interaction with Building Information Models” being submitted by Prasad Nanasaheb Paigude
(ITTA33) is a record of bonafide work carried out by her under the supervision and guidance of
Mrs. P. S. Bangare in partial fulfillment of the requirement for TE (Information Technology
Engineering) – 2019 course of Savitribai Phule Pune University, Pune in the academic year 2024-
2025.

Date:

Place: Pune

Mrs. P. S. Bangare Dr. S.S. Kulkarni

Guide Head,
Department of Information Technology Department of Information Technology

Dr. K. P.
Patil
Principal
Sinhagad Academy Of Engineering, Pune-48.

2
 ACKNOWLEDGEMENT

I would like to thank Mrs. P.S.Bangare for helping/Guiding me out in selecting the topic and
contents, giving valuable suggestions in preparation of Seminar report and presentation.
I am grateful to Dr. D.R.Anekar, seminar coordinator, for her suggestions regarding latest trends in
Computer and IT industry, Dr. S.S.Kulkarni, Head of Information Technology Department, for
providing healthy environment and facilities in the department. He allowed us to raise our concern
and worked to solve it by extending his co-operation time to time.
Goal makes us to do work. Vision is more important than goal which makes us to do work in the best
way to make work equally the best. Thanks to Principal, Dr. K.P.Patil for his support and vision.
Consistent achievement requires boost on consistent interval basis. Management has given full
support and boosted us to be consistent and achieve the target. Thanks to management for their
support. Thanks to all the teachers for their extended support and valuable guidance. I would like to
be grateful to all my friends for their consistent support, help and guidance.

3
 ABSTRACT

This study introduces DAVE (Digital Assistant for Virtual Engineering), a Generative Pre-trained
Transformer (GPT)-powered digital assistant prototype, designed to enable real-time, multimodal
interactions within Building Information Modeling (BIM) environments for updating and querying
BIM models using text or voice commands. DAVE integrates directly with Autodesk Revit through
Python scripts, the Revit API, and the OpenAI API and utilizes Natural Language Processing (NLP).
This study presents (1) the development of a practical AI chatbot application that leverages
conversational AI and BIM for dynamic actions within BIM models (e.g., updates and queries) at any
stage of a construction project and (2) the demonstration of real-time, multimodal BIM model
management through voice or text, which aims to reduce the complexity and technical barriers
typically associated with BIM processes. The details of DAVE’s development and system architecture
are outlined in this paper. Additionally, the comprehensive process of prototype testing and evaluation
including the response time analysis and error analysis, which investigated the issues encountered
during system validation, are detailed. The prototype demonstrated 94% success in accurately
processing and executing single-function user queries. By enabling conversational interactions with
BIM models, DAVE represents a significant contribution to the current body of knowledge.

Keywords: Building Information Modeling- BIM; Artificial Intelligence; conversational AI; natural
language processing; Generative Pre-trained Transformer; GPT; virtual assistant.

4
 TABLE OF CONTENTS

Acknowledgment III
Abstract IV
Table of Contents V
List of Abbreviations VI

SR NO. TITLE PAGE NO

1 INTRODUCTION 7
2 LITERATURE SURVEY 8
3 ORIGIN 10
4 PURPOSE AND SCOPE 11
5 OBJECTIVE 13
6 HARDWARE AND SOFTWARE 14
7 ARCHITECTURE 15
8 FEATURES OF SEMANTIC WEB 17
9 NECESSITY 18
10 APPLICATIONS 19
11 CHALLENGES AND PROBLMS 20
12 CONCLUSION 21
13 REFERENCES 22

5
 LIST OF ABBREVIATIONS

1. AEC - Architecture, Engineering, and Construction

2. AI - Artificial Intelligence

3. BIM - Building Information Modeling

4. CV - Computer Vision

5. DLL - Dynamic-Link Library

6. GPT - Generative Pre-trained Transformer

7. JSON - JavaScript Object Notation

8. LLM - Large Language Model

9. NLP - Natural Language Processing

10. PDF - Portable Document Format

11. Revit - Autodesk Revit (BIM software)

12. API - Application Programming Interface

13. IFC - Industry Foundation Classes

14. GUI - Graphical User Interface

15. TTS - Text-to-Speech

6
 1. INTRODUCTION

Over the last two decades, the Architecture, Engineering, and Construction (AEC) industry has
transformed significantly with the integration of Building Information Modeling (BIM). BIM, a digital
representation of a building's physical and operational characteristics, has become essential for
streamlining project management, improving collaboration, and utilizing building data. However,
barriers to its wider adoption persist due to complex software, steep learning curves, and a lack of
sufficient training.
Meanwhile, the rapid advancements in Artificial Intelligence (AI), particularly in areas such as
Machine Learning (ML) and Natural Language Processing (NLP), have opened new possibilities to
address these challenges. AI has revolutionized various industries by automating tasks, enhancing data
analysis, and supporting decision-making processes. In the AEC industry, AI, particularly NLP, can
make BIM systems more intuitive, accessible, and efficient. The use of large language models (LLMs),
like OpenAI’s GPT series, offers the potential to simplify interactions with BIM models by allowing
natural language commands, making even non-technical users able to engage with these systems.
This study introduces DAVE (Digital Assistant for Virtual Engineering), a prototype virtual assistant
that combines AI and BIM technologies. DAVE uses NLP to facilitate real-time updates, queries, and
interactions within BIM models through both text and voice commands. The assistant directly
integrates with Autodesk Revit using Python scripts, the Revit API, and OpenAI's API, enabling users
to interact with BIM models without complex technical inputs. This dynamic interaction aims to reduce
the complexity associated with BIM, making it more accessible for AEC professionals at various stages
of construction projects.
DAVE contributes to the ongoing digital transformation of the AEC industry by providing a
practical AI tool that can enhance daily BIM tasks and support real-time model management. Through
the integration of conversational AI, DAVE addresses the technical barriers of BIM adoption,
facilitating more connected, intelligent, and user-friendly project environments.
The paper is structured as follows: Section 2 covers related literature and compares DAVE with
existing studies, Section 3 outlines the research objectives and scope, Section 4 describes the system’s
architecture and workflow, Section 5 details the testing and validation process, Section 6 discusses
results, and Section 7 concludes with future research directions.

7
 2. LITERATURE SURVEY

1. David Fernandes, Sahej Garg, Matthew Nikkel, and Gursans Guven (2024):
"A GPT-Powered Assistant for Real-Time Interaction with Building Information Models"
This paper introduces DAVE, a GPT-powered assistant that enhances real-time interactions within
BIM environments. The assistant enables users to query and update BIM models using natural
language through voice or text commands. Integrating Autodesk Revit, Python scripts, and OpenAI
API, DAVE simplifies the user interface, making BIM more accessible for non-technical users. The
study showcases how AI can streamline project workflows, reduce technical barriers, and increase
user efficiency in managing complex BIM data.

2. Pan,N.-H. & Isnaeni (2024):

"Integration of Augmented Reality and Building Information Modeling for Enhanced Construction
Inspection—A Case Study"
This paper explores the combination of Augmented Reality (AR) with BIM to improve construction
inspection processes. The authors demonstrate how AR can overlay real-time data from BIM
models onto physical construction sites, allowing more accurate inspections and reducing errors.
The integration enhances the inspector's ability to assess construction progress and compliance,
leading to better decision-making and streamlined operations in the construction phase.

3. Chen N, Lin X, Jiang H, An Y (2024) :

"Automated Building Information Modeling Compliance Check through a Large Language Model
Combined with Deep Learning and Ontology"
The paper proposes a system that automates BIM compliance checks using a Large Language
Model (LLM) in combination with deep learning and ontologies. The system improves the
accuracy and speed of regulatory compliance checking by processing large amounts of building
data and identifying potential violations. The research highlights how combining AI and BIM can
reduce the manual workload associated with ensuring compliance with construction regulations and
standards.

4. Ivanova S, Kuznetsov A, Zverev R, Rada A (2023) :

"Artificial Intelligence Methods for the Construction and Management of Building"
This study investigates the application of AI techniques in construction and building management.
8
The authors focus on how AI can optimize project timelines, improve decision-making, and
enhance resource management. Through methods like machine learning and predictive analytics,
AI helps automate tasks such as material selection, schedule planning, and building performance
monitoring. The paper emphasizes AI's potential to increase efficiency and reduce costs across the
construction lifecycle.

5. Locatelli M, Seghezzi E, Pellegrini L, Tagliabue L.C., Di Giuda (2021) :

"Exploring Natural Language Processing in Construction and Integration with Building Information
Modeling: A Scientometric Analysis"
This paper conducts a scientometric analysis of the role of Natural Language Processing (NLP) in
the construction industry, with a particular focus on its integration with BIM. The authors examine
current research trends, applications, and gaps in combining NLP with BIM, highlighting the
potential for NLP to improve the accessibility of BIM data and enhance collaboration among
stakeholders. The paper concludes that NLP can make complex construction data more
understandable and easier to work with, but challenges remain in the effective implementation of
these technologies in real-world projects.

9
 3. ORIGIN

The primary goal of this study is to develop and evaluate a prototype of an AI chatbot for dynamic
updates and multimodal interactions within BIM environments. This approach aims to enhance the
efficiency and effectiveness of BIM workflows by leveraging the capabilities of an LLM, namely GPT-
4. It extends beyond the typical use of chatbots in the literature for BIM information retrieval to include
real-time model updates and interactions via text or speech.
The developed prototype integrates with a specific, yet widely adopted, BIM-authoring software
(i.e., Autodesk Revit), showcasing the practical application of AI, more specifically NLP, in real-world
BIM tasks. The prototype can perform 12 critical and commonly used actions within the BIM
environment. Each one of them is selected based on the daily tasks performed by a typical Virtual
Design and Construction department in a construction company and the mainstream uses of Revit.
They range from simple tasks like undoing the last action to more complex tasks such as changing the
family type of a selected element and updating room names, numbers, and occupants. The prototype
architecture can be adapted to work with other BIM platforms and other LLMs with a few adjustments
to the current function implementation to adapt to the new syntax as well as input/output formats.
The novelty and originality of this study further lie in the prototype’s applicability to all phases of a
construction project from pre-design and design to construction and demolition. Additionally, its
capabilities have been demonstrated on a basic sample building model as well as a multidisciplinary
real-world office building model proving the prototype’s versatility across different project
requirements. The current limitations of the prototype and some specific challenges, such as scalability
and system efficiency, are discussed in Section 6, and areas for future improvements and research are
identified.

10
 4. PURPOSE AND SCOPE

1. Develop and Evaluate a Prototype:

The primary goal is to create and assess an AI chatbot prototype for dynamic updates and
multimodal interactions within Building Information Modeling (BIM) environments.

2. Enhance BIM Workflow Efficiency:

The prototype aims to improve the efficiency and effectiveness of BIM workflows by utilizing
a Large Language Model (LLM), specifically GPT-4, for real-time model updates and
interactions through text or voice commands.

3. Integration with Autodesk Revit:

The system integrates with widely used BIM-authoring software, Autodesk Revit, to apply AI
and Natural Language Processing (NLP) in practical BIM tasks.

4. Perform Common BIM Actions:

The prototype supports 12 common and critical actions in the BIM environment, ranging from
undoing the last action to updating room names and changing element types.

5. Adaptability for Other Platforms:

The architecture can be adapted to work with other BIM platforms and LLMs with minimal
changes in function implementation and data formats.

6. Application to All Phases of Construction:

The prototype is designed to be applicable throughout the construction process, from pre-design
to demolition phases.

7. Demonstration on Real-World Models:

The system has been demonstrated on both simple sample models and complex real-world
multidisciplinary models to prove its versatility.

8. Addressing Technical Barriers:

The system is designed to reduce complexity and technical barriers, making it more user-
friendly for AEC professionals.

11
9. Identify Limitations and Future Scope:
Limitations like scalability and system efficiency are identified, with future improvements
suggested for expanding the system's capabilities.

12
 5. OBJECTIVE

1. Develop a GPT-Powered AI Chatbot:

To design and create a prototype of an AI chatbot, named DAVE (Digital Assistant for Virtual
Engineering), that integrates conversational AI with Building Information Modeling (BIM)
environments.

2. Enable Real-Time Interactions with BIM Models:

To enable real-time, multimodal interactions with BIM models through text or voice
commands, allowing for dynamic updates, queries, and management of BIM data.

3. Reduce Complexity in BIM Processes:

To simplify the traditionally complex BIM processes and make interactions more intuitive and
accessible for users, especially those without advanced technical knowledge.

4. Demonstrate Practical Application in AEC:

To showcase the practical use of the AI chatbot in the Architecture, Engineering, and
Construction (AEC) industry by integrating it with Autodesk Revit for performing a wide range
of critical tasks.

5. Evaluate the System’s Effectiveness:

To test and evaluate the performance of the prototype through validation processes, Including
response time analysis and accuracy testing, with a focus on error handling and system
efficiency.

6. Contribute to BIM and AI Integration:

To contribute to the digital transformation of the AEC industry by combining BIM and AI
technologies, making future built environments more connected, intelligent, and user-centric.

13
 6. HARDWARE AND SOFTWARE

Hardware:
1. Computer System: A capable PC or workstation capable of running Autodesk Revit and
handling Python scripts effectively.
2. Microphone: For capturing voice commands to interact with DAVE.
3. Speakers: To provide audio feedback through the Text-to-Speech functionalities.

Software:
1. Operating System: Windows (specific version not mentioned, but compatible with Autodesk
Revit).
2. Autodesk Revit: The primary BIM authoring software used for the prototype.
3. Python: Programming language used for developing the AI assistant and implementing scripts.
Python Libraries:
1. OpenAI API: For integrating GPT models.
2. pvporcupine: For voice activation.
3. speech_recognition: For processing voice commands.
4. customtkinter: For creating the Graphical User Interface (GUI).
4. C#: For implementing the File Watcher component and interacting with Autodesk Revit's API.
5. JSON: Format used for data interchange between components.
6. Visual Studio: Development environment for C# components.
7. Other APIs:
Autodesk Platform Services (APS) API: For data extraction from Revit files.

14
 7. ARCHITECTURE

It illustrates the architecture of the proposed AI-assisted BIM system, its components, and their
links. It is a blend of various technologies for smart, real-time interactions within a well-known and
commonly used BIM authoring tool, Autodesk Revit. The system possesses a preliminary stage called
Data Extraction, further explained in Section 4.1.1. During this stage, metadata of the building
elements are extracted from the Revit project file (.rvt) and converted into a JSON model derivative.
These JSON data are then transformed into CSV format for efficient information retrieval. This
conversion from JSON to CSV helps prevent potential data loss that might occur during a direct
conversion of the Revit file into CSV. Additionally, the CSV format makes it simpler and faster to
query specific data points and facilitates smoother integration into AI systems because of the flat,
tabular structure it provides.
The developed system consists of three main components as illustrated in Figure 1. The core of the
prototype is the Python component, which is empowered by the OpenAI Application Programming
Interface (API) and enables smart user interactions, including Text-to-Speech and Speech-to-Text
features for voice responses. Users engage with the system via a custom Graphical User Interface
(GUI) that runs along with the Revit application. More details are provided in Section 4.1.2. The JSON
Bridge Component is a key mechanism enabling real-time communication between the OpenAI
Assistant Class and Revit. Structured around the bridge.json file, it seamlessly integrates Python and
C# within BIM workflows, supporting essential actions with minimal resource use. This component is
expanded upon in Section 4.1.3. Finally, the File Watcher component is crafted in C# and dynamically
monitors changes in the bridge file, coordinating with the Python AI component through the Revit API
and custom DLL classes. Its capabilities and structure are further detailed in Section 4.1.4. All the
system components and their roles are summarized in Table 2.
The architecture of the developed system combines Python scripting with a Dynamic-Link Library
(DLL) and a JavaScript Object Notation (JSON) file to create a responsive system capable of bridging
human input with complex BIM model adjustments. It also addresses the practical needs of AEC
professionals who often struggle with the complexity of interacting with BIM models by making it a
more user-friendly and efficient experience.

15
Fig No. 1 : DAVE’s System Architecture Overview

Fig No. 2 : DAVE’s Workflow Diagram

16
 8. FEATURES OF SEMANTIC WEB

1. Interoperability: Allows data from different sources to be combined and used seamlessly
across systems.
2. Resource Description Framework (RDF): A framework for representing information about
resources on the web.
3. Ontology: Defines relationships between data and establishes common vocabularies for
specific domains.
4. SPARQL: A query language used to retrieve and manipulate data stored in RDF format.
5. Linked Data: Promotes the interconnection of data across different domains, enabling deeper
insights.
6. Machine-readable Data: Ensures that data can be processed not just by humans, but also by
machines, facilitating automation and AI use.

Fig No. 3 : Data Extraction Component of the Prototype

17
 9. NECESSITY

1. Complexity of BIM: The intricacy of BIM processes can present significant challenges for users,
especially those lacking extensive training or experience. DAVE aims to simplify these interactions by
providing an intuitive interface for querying and updating models through natural language.

2. Improving Efficiency: As construction projects become more complex, the need for efficient
management of building data becomes critical. DAVE enables real-time interactions with BIM models,
which can enhance productivity and streamline workflows.

3. Accessibility: The traditional BIM software interfaces can be daunting, creating barriers for non-
expert users. By leveraging conversational AI, DAVE makes BIM accessible to a broader audience,
allowing various stakeholders to interact with building data without needing deep technical knowledge.

4. Integration of AI in AEC: The integration of Artificial Intelligence (AI) into the Architecture,
Engineering, and Construction (AEC) industries is increasingly recognized as a means to enhance
decision-making, automation, and overall efficiency. DAVE represents a step toward the realization of
this integration, demonstrating how AI can facilitate user-friendly interactions with BIM.

5. Enhanced Collaboration: With the growing emphasis on collaboration within project teams,
DAVE serves as a tool to improve communication and information sharing among stakeholders,
ultimately leading to more successful project outcomes.

18
 9. APPLICATIONS

1. Real-Time Model Updates: DAVE allows users to make instant modifications to BIM models
using natural language commands. This can include actions such as changing the family type of
elements or updating room details like names and numbers.

2. Information Retrieval: Users can query specific information from the BIM model, such as
retrieving details about particular building components, which enhances accessibility to critical data
without navigating complex software interfaces.

3. Voice Command Integration: The voice command functionality enables hands-free operation,
which is particularly useful in construction environments where users may need to access information
or make changes without direct interaction with a computer.

4. Support for Multi-Disciplinary Teams: DAVE can be used across various disciplines involved in a
construction project, including architecture, structural engineering, and mechanical systems, facilitating
collaboration and coordination among team members.

5. Training and Onboarding: The intuitive nature of DAVE’s interface can serve as a training tool for
new users, helping them to familiarize themselves with BIM functionalities and workflows without
extensive formal training.

6. Enhanced Decision-Making: By providing instant access to relevant data and insights, DAVE
supports informed decision-making throughout the project lifecycle, leading to better outcomes and
resource management.

7. Integration with Other Systems: DAVE's architecture allows for potential integration with other
software and technologies used in the AEC industry, promoting a more connected and efficient digital
ecosystem.

19
 10.CHALLENGES AND PROBLMS

1. Complexity of Natural Language Processing (NLP): Accurately interpreting and processing the
vast variety of natural language inputs, particularly those containing technical terminology specific to
the Architecture, Engineering, and Construction (AEC) industry, presents a challenge. Ensuring that
DAVE can understand and respond effectively to such inputs requires continuous training and
improvement of the NLP model.

2. Training Data Limitations: The effectiveness of the conversational AI relies heavily on the quality
and quantity of training data. Inadequate or biased datasets can lead to inaccurate responses, which may
affect user trust and satisfaction.

3. User Interface Design: Designing an intuitive user interface that accommodates both text and voice
inputs while remaining user-friendly is a complex task. Users may have varying levels of familiarity
with technology, necessitating a balance between functionality and simplicity.

4. Integration with Existing Systems: Ensuring seamless integration between DAVE, Autodesk Revit,
and other software tools used in BIM workflows can be challenging. Compatibility issues may arise,
requiring additional development efforts to create a cohesive user experience.

5. Performance and Scalability: As the complexity of BIM models increases, ensuring that DAVE can
maintain performance and responsiveness is crucial. Scalability issues may arise when handling larger
datasets or more complex queries.

6. Error Handling and User Feedback: Robust error-handling mechanisms are essential for
maintaining user confidence in the system. DAVE must effectively communicate errors or limitations
back to users to foster understanding and guide them in formulating valid requests.

7. User Adoption: Encouraging users, particularly those accustomed to traditional BIM workflows, to
adopt DAVE may be challenging. Effective change management strategies and training will be
necessary to facilitate this transition.

20
 11. CONCLUSION

Introducing DAVE: a GPT-based prototype for a digital assistant, created to facilitate real-time
interaction within a BIM environment through the use of NLP. Through this interface, users can
interact with the BIM model either by voice or text commands-a clear way of simplifying, and thereby
lifting, a highly technical barrier which has long been rife in far more cumbersome BIM software.
Among the key contributions of this research is the development of a practical AI chatbot application
that can successfully execute 12 critical actions within BIM environments with greater accessibility and
efficiency towards uses independent of their experience levels. The prototype was able to accurately
execute the user queries at a success rate of 94%, meaning it's an effective tool for real-time model
management.
Though promising, the study revealed several issues that have to be taken into consideration in
further research: the difficulty of the understanding of NLP towards technical language, integration
with existing BIM systems, and adoption of BIM by users. Future work will include refinement in the
capabilities of NLP, expansion of DAVE regarding the actions it can perform, and improvement in
training and support provided to the user, as such an approach will inspire more users to work with the
BIM adoption.
Ultimately, the DAVE represents a significant step toward the integration of conversational AI with
BIM within the digital transformation occurring within the Architecture, Engineering, and Construction
(AEC) industry. This integrates more intuitive interactions into BIM to create a better connected, more
efficient, and user-centric built environment.

21
 12. REFERENCE

1. David Fernandes, Sahej Garg, Matthew Nikkel and Gursans Guven, “A GPT-Powered Assistant
for Real-Time Interaction with Building Information Models”, MDPI, Buildings 2024, 14, 2499
2. Pan, N.-H., Isnaeni, “Integration of Augmented Reality and Building Information Modeling for
Enhanced Construction Inspection—A Case Study”, MDPI, Buildings 2024, 14, 612
3. Chen N, Lin X, Jiang H, An Y, “Automated Building Information Modeling Compliance Check
through a Large Language Model Combined with Deep Learning and Ontology”, MDPI,
Buildings 2024, 14, 1983
4. Ivanova S, Kuznetsov A, Zverev R, Rada A, “Artificial Intelligence Methods for the
Construction and Management of Building”, MDPI, Sensors 2023, 23, 8740
5. Locatelli M, Seghezzi E, Pellegrini L, Tagliabue L.C., Di Giuda, “Exploring Natural Language
Processing in Construction and Integration with Building Information Modeling: A
Scientometric Analysis”, MDPI, Buildings 2021, 11, 583

22