Integrating Generative Artificial Intelligence and Problem-Based Learning into the Digitization in Construction Curriculum

Abstract

This study proposes incorporating generative artificial intelligence large language models (LLMs) into the Master of Science (M.Sc.) curriculum on digitization in construction. The aim was to help students generate computer code to solve, automate, and streamline practical challenges in advanced construction engineering and management (CEM). To this end, a host of problem-based learning (PBL) individual assignments and collaborative team projects were developed, alongside a combination of flipped classroom models and blended learning lessons, in order to teach effective interactions with LLMs and mitigate concerns, such as bias and hallucination. The effective interaction with LLMs not only facilitated code generation, which would otherwise be complex without additional formal training, but also provided a platform for strengthening basic project management skills, such as departmentalization, work breakdown structuring, modularization, activity delegation, and defining key performance indicators. The effectiveness of this approach was quantitatively and qualitatively evaluated within two new modules, Digital Engineering and Construction and Digital Technologies in Field Information Modeling. These modules were offered over three semesters each as part of a new M.Sc. program in Technology and Management in Construction at the Karlsruhe Institute of Technology. It was observed that 86.4% of students fully completed the PBL projects, while the remaining 13.6% achieved over 50% completion across all six semesters. Furthermore, anonymous student surveys indicated a teaching quality index of 100% in five semesters and 96.4% in one semester. These preliminary results suggest that the proposed strategy can be used to effectively integrate LLMs to support students in code generation for open-ended projects in CEM. Further research was, however, found to be necessary to ensure the sustainable revision and redesign of the problems as LLM capabilities evolve.

1. Introduction

1.1. Problem Statement

The Institute of Technology and Management in Construction (TMB) at the Karlsruhe Institute of Technology (KIT) launched a new Master of Science (M.Sc.) program [1] in the winter semester of 2022 with five distinct profiles (i.e., specialization areas). One of these profiles is called “Digital Technologies in Construction”, offered by the department of Digital Engineering and Construction (DEC) [2]. Within this profile, two new mandatory modules, namely, “DEC” [3], and “Digital Technologies in Field Information Modeling (FIM)” [4,5], were designed and offered by the Chairholder of DEC [2]. The intended outcome of this profile is to prepare graduates to not only become familiar with the most up-to-date technological advancements pertaining to construction engineering and management (CEM) [6] but also to implement (or lead the implementation) of these technologies within their prospective careers and organizations. In fact, many successful organizations in the construction industry [7] are rapidly evolving to incorporate digitization- and technology-related divisions to capitalize on the competitive advantages gained through the fast adoption of reliable technologies [8]. Future leaders must therefore acquire knowledge in basic CEM, as well as recent technological advancements, and their application in construction, otherwise their ability to lead teams, articulate a vision, and clearly define and communicate necessary tasks will be diminished.

Therefore, graduates must achieve competencies in: (i) CEM concepts; (ii) digital technologies applicable to construction, including artificial intelligence (AI); and (iii) the implementation of these technologies to address real-world CEM problems. Given that the area of CEM is well established with many available online modules and open access resources [9], basic knowledge of these topics was expected before enrollment in the module. The course material was, therefore, mainly designed to bridge the learners’ knowledge gap in the broad and demanding topics of digital technologies in construction and best practices to apply this knowledge within real-world construction scenarios. Furthermore, to lead by example (that is, to demonstrate timely technology adoption), the modules were designed to consistently improve with the advent of new digital technologies. One example of this was the integration of large language models (LLMs) into the curriculum as soon as ChatGPT 3.0 [10] became available to the public in late 2022 [11].

To this end, this study focuses on integrating LLMs into the curriculum of the two aforementioned mandatory modules within the DEC profile of the TMB M.Sc. program [1,3,4] to enable CEM students to accomplish tasks that would otherwise require additional formal training. Examples of such tasks include brainstorming to summarize existing solutions to solved problems and translating logic models, pseudocodes, and algorithms from various human languages into a computer programming language of choice, thus supporting automation and streamlining tasks. More specifically, this study discusses and presents preliminary findings on integrating LLMs into the content, individual assignments, and final group term projects of the two modules [3,4].

1.2. Module Structure Overview

The two modules, DEC and FIM, combined various pedagogical concepts, such as the flipped classroom model [12,13,14], blended learning [15], and integrated engineering design projects [16,17], to accommodate active [18,19] and experiential learning [20] for a variety of student learning styles in a hybrid format [21], while fostering an autonomy-supporting learning [22] environment in alignment with the basics of self-determination theory (SDT) [23]. These two modules were carefully crafted with constructive alignment in mind [24]. As such, the objectives of the modules, learning outcomes, lecture content, assignments, and term project [25] were designed and developed wholistically. The basic principles of the flipped classroom model for blended learning together with the fundamentals of applied constructive alignment in teaching and learning for integrated engineering design [26] were acquired, implemented, and practiced through the Instructional Skills Workshop (ISW) in Calgary, Canada [27] and further refined within the Baden-Württemberg Certificate for University Didactics (HDZ) [28]. More specifically, the two modules were designed to achieve the following:

• Foster active participation [18,29,30] in the class by employing various teaching learning activities (TLAs) employed within “Module I-Fit—Teaching I and II” of the HDZ [28];
• Support research-oriented learning [31] using individual assignments by employing strategies presented and applied in “Module II—Research-based Learning” of the HDZ [28]; and
• Incorporate the effectiveness of practical problem-based learning (PBL) [32] while promoting collaborative learning in groups using a final group term project by utilizing knowledge shared and adopted within the “Module II—Crash Course: Teaching and Learning in Higher Education” of the HDZ [28].

In connection with the flipped classroom model, the lecture content was pre-recorded and shared with the students for review as pre-lecture activities (PLAs) using the ILIAS open source learning management system [33]. The PLAs are also publicly available and accessible on YouTube [34]. The lecture period, typically two 90 min time slots per week, involved discussions and participatory activities through real-world problem solving. The activities, structured into 20–30 min mini-activities, ranged from data collection using mobile phones to programming relevant AI-based solutions for the construction industry. The goal of the in-class activities was to gradually transition the students from merely describing and regurgitating the content of the PLAs (surface-level learning) towards the implementation and application of the knowledge in a real-world scenario (deep-level active learning [35]) using TLAs that support active and practical approaches [18,20].

Furthermore, the state-of-the-art recommendations on the proportion of collaborative learning [36] to individual learning in STEM-related flipped, blended, and PBL courses often emphasize higher proportions of collaborative learning compared to classical approaches, ranging anywhere from 30% to 60% [14,37,38]. To this end, the designed modules included four individual assignments, followed by a large PBL group term project, constituting approximately 40% and 60% of the final grade, representing the level of complexity and time required for completion, respectively. The assignments were carefully designed to solve real-world CEM problems by applying the concepts taught in the class and to further support the objectives of the program/profile. The final project was designed as a PBL project that again builds on top of the competencies gained by all members of the groups from the lecture contents and assignments to solve a larger problem related to digitization and automation in CEM.

Finally, the final project (and to a reasonable extent the assignments) was generally open ended by design while requiring the students to achieve a minimum set of competencies and learning outcomes to achieve their final grade. This student-centered (student-lead) freedom and flexibility enabled the students to also be an active participant [18,19,30] in their learning progress and focus on the competencies that would support them in their future career prospects and interests. This individualized choice and autonomy in learning in connection with SDT was shown to be effective in learner motivation and success within the contexts of PBL [39], flipped classroom models [40], and personalized learning [41]. Furthermore, independent thought with a growth mindset that deems challenges as opportunities is shown to improve problem-solving abilities [42]. In the developed modules, the students were merely encouraged to challenge themselves by showcasing the value and possibilities of what can be achieved and allowing them the option to choose rather than through obligation (e.g., requirement for pass or grading). One example of the successful implementation of this latter strategy was observed within the final term project of the FIM module in summer 2024, where the students were only required to generate a programming script to achieve a top grade. Despite this, all teams decided—of their own accord—to develop a standalone computer app with a user interface as their final deliverable product. Based on the author’s observations and discussions with the students, the students simply deemed the challenge of creating a new standalone app as an opportunity to learn, grow, and excel.

1.3. Professionalism

Established accreditation bodies, such as the Project Management Institute (PMI) Global Accreditation Center [43], commonly evaluate program curricula based on learning outcomes that not only strengthen learner direct management skills but also professional behavior. Given that the presented M.Sc. program is in TMB, with prospects to hold leadership roles in the CEM industry after graduation, two important aspects pertaining to professionalism and professional conduct were also incorporated into the curriculum:

• The students had to adhere to strict deadlines, which is normally less strict in typical student-centered teaching, given the inherent student-paced nature of the approach [44]. As such, strict deadlines were set for the delivery of the assignments as well as the final project.
• The students had to practice their communications skills and their ability to articulate their vision into simple, understandable, and meaningful “activities” so as to satisfy the first steps towards project success.

These two aspects are particularly crucial since the influence of leaders within project teams increases with the clarity of communication (conveying) of expectations, ideas, vision, and direction [45].

1.4. Relevance of LLM Interaction to Project Management

The latter competency, facilitating the students with articulating their vision, was further divided into the following two categories:

• Oral and written presentation and reporting skills, which were practiced within TLAs [18], such as think–pair–share, and were required for each of the assignments as well as the final project;
• Departmentalization and work breakdown structure along with project monitoring and control skills and the definition of metrics for the validation of success [9], which were taught with examples in the module content and evaluated throughout the assignments and final project as deliverables.

The latter skills are instrumental for the project’s success; however, they are generally acquired through experience. It turns out, as will be discussed later, that interaction with LLMs to solve open-ended problems requires the user to perform these tasks effectively, as otherwise the LLM generally cannot provide desirable solutions to complex problems without clear guidance from the user. This is due to the fact that LLMs in their current form are prone to extrapolation errors, as well as a phenomenon, called, hallucination, which will be further discussed in the next section. As such, LLMs can indirectly support effectively learning fundamental project leadership and management skills by simulating the “concrete experience” setting within established experiential learning methods [20].

2. Integration of LLMs Within the DEC Curriculum

2.1. Introduction to Generative Adversarial Networks

Neural networks (NNs) [46], such as artificial neural networks and convolutional neural networks, generally consist of a set of neurons which perform some affine transformation (tunable parameter) within different layers of the network (depending on the architecture) to map a set of input data to a set of outputs. The training of these networks generally involves the use of high-quality ground truth input and output data to find the optimum tunable parameters for the given network architecture. Once trained, they can be used to predict outputs given new input data. They are inherently suitable for interpolation, and as such large amounts of diverse data are required to train these networks. As an example, if a network designed to detect dogs in images is trained only on images of chihuahuas, it will most likely be unable to classify an image of a husky as a dog.

Generative adversarial networks (GANs), first introduced in [47], were designed to further extend the abilities of NNs by generating completely new data that resemble real data. These data can be images, or in the case of LLMs, sentences. GANs generally comprise two main NNs, the generator network and the critic (or discriminant) network. Only the critic network is trained on real input data. The generator network is then designed to generate a large set of possible input data. The generator then sends these data to the critic to receive feedback. The generator network is then trained through a reinforcement learning scheme that rewards the generated data receiving positive feedback and penalizes those receiving negative feedback. In other words, the trained generator, while not observing the original data, generates input data that resemble real data so much so that the critic cannot distinguish between them (i.e., the critic is tricked). An example of this is the generation of a human-like image by a GAN, which resembles a human but does not depict a human that actually exists.

GANs are therefore capable of extrapolating so far as the critic can tell. This requires the critic to be trained on very large and diverse datasets, which is the case for the common LLMs such as ChatGPT [10], Claude [48], and Gemini [49]. Another benefit of the reinforcement learning scheme is that LLMs can be trained progressively by receiving feedback from the users directly. However, the extent of their extrapolation (i.e., generating new text and reasoning for problems that have not been solved) is very much limited. Another shortcoming of LLMs is a phenomenon called hallucination, which involves responses that (in the context of LLMs) are grammatically correct and at first glance might feel true but are factually or logically incorrect. Hallucination is an inherent property of GANs and can occur due to many factors during the training and tuning of the network, such as incompleteness in the training data and vague/unclear questions and expectations by the user. As such, LLMs in their current form can only provide solutions to problems that are either solved or require negligible scientific discoveries/reasoning.

2.2. LLMs as Translation Tools for Computer Code Generation

Despite the limitations of LLMs in extrapolation and hallucination, they are competent in translation, and in the case of computer-code generation, can act as useful translating tools to generate computer code from normal human conversation. Therefore, students can use LLMs effectively to generate computer code to solve relevant problems if they clearly communicate the problem, expectations, and requirements. Otherwise, the provided responses from the LLM runs the risk of hallucination. Furthermore, even if hallucination does not occur, LLMs may still generate erroneous code, which requires additional debugging. Given that finding errors in codes with more lines is more challenging than in those with less lines, the students must utilize the basic principles of chain of thought reasoning to departmentalize (or modularize) the problem into smaller and more meaningful sub-problems/modules. Finally, a set of metrics must be defined and consequently measured at each step to evaluate the performance of each of the modules. In other words, key performance indicators (KPIs) must be defined so that the successful implementation of each module and function can be effectively evaluated and verified.

To this end, the effective utilization of LLMs for code generation requires the following:

• Clarity of problem definition, expectations, requirements, and constraints;
• Strategies to departmentalize and ideally modularize the problem into smaller problems and functions to be combined to solve the original problem;
• Definition of metrics for the evaluation of successful code generation for each of the sub-problems.

Incidentally, this process resembles the steps required for basic project management, including, planning, monitoring, and control. Furthermore, for many tasks that may seem trivial/experiential, the process of departmentalization and modularization can allow the students to become self-aware of their thought, reasoning, and cognitive process, fostering metacognitive awareness [50], which has been shown to be effective in problem solving. Modularization, departmentalization, and activity delegation can also be combined with the best practices in machine-learning-based computer networks [51] to allow multiple LLM chatbots to collaborate to complete a more significant task.

As such, if the students can effectively carry out the aforementioned processes by assigning clearly defined tasks to the LLM and overseeing the successful implementation and execution of the problem, the students in effect can strengthen their metacognitive awareness while practicing their fundamental project management skills. The latter supports simulating an environment for experiential learning [20] of fundamental project management skills, which is otherwise challenging to acquire without real-world exposure.

2.3. Concerns over Plagiarism

Two types of plagiarism were identified as possible risks when introducing LLMs within the curriculum:

• LLMs writing the code and solution to the problem with negligible interaction or guidance from the user;
• Students copying their code/solutions from each other.

With due consideration of the shortcomings of LLMs, concerns over possible plagiarism of the first type can be eliminated as long as LLMs are correctly integrated within the curriculum; that is, integrating LLMs as a tool to assist with solving open-ended problems within PBL. Furthermore, given that LLMs also generate unique code and sentences, it would be nearly impossible for any two students to submit the exact same assignment/code. This holds true even if the exact same questions are asked. If there are concerns over the students copying each other’s questions, this can be easily detected by requesting the students to submit all interactions with the LLMs together with the final submission of their work and feeding the submissions into a plagiarism scanning tool, such as Plagiarism Checker X [52], for further evaluation of possible similarities. This provides the opportunity to detect any possible plagiarism of the second type as well. Therefore, for the case of the present study, the effective integration of LLMs in the curriculum can not only eliminate the possibility of plagiarism of the first type—courtesy of the inherent properties of LLMs—but also identify plagiarism of the second type by design.

2.4. Advantages of LLMs’ Integration in the DEC Curriculum

To address limitations in technological; literacy among CEM students, the integration of LLMs as a bridge to generate code to solve complex problems within the DEC and FIM modules delivers the following advantages:

• It offers a platform for CEM students to realize their vision, particularly in streamlining and automating existing project planning and controlling best practices without explicit knowledge of coding. Furthermore, given that an important aspect of project management is to improve efficiency in construction, automating and streamlining repeatable tasks becomes inherently valuable. This allows students with little knowledge of coding to communicate their visions and solutions by generating computer code in the programming language of choice, which would have otherwise required considerable formal training on software programming. This is particularly useful for students in CEM, who are not classically trained in software programming but might be required (or wish) to lead (or procure) a team to develop customized software for the construction industry in their future career prospects.
• It allows international students with varying skills in the main language of education to communicate their visions with the LLM chatbots in other languages, which can support a more inclusive and equitable learning environment, particularly by removing language and communication barriers. In other words, the language skill level of the students will have a negligible impact on their learning quality and ability to complete the tasks/problems. This is also particularly useful in offering modules for an international audience globally or through remote/online means.
• It enables the construction management students to practice and strengthen their basic planning and control skills, such as WBS, resource allocation, and metrics to measure and report the project’s success, throughout the completion of their assignment/project. This will also include practicing their vision planning, task division and delegation, and clarification of KPIs through integration with LLM chatbots. This is due to the fact that LLMs can be considered as agents, and one agent (in one chat instance) can only be given limited information and requests before the phenomenon of hallucination occurs. This can simulate the dynamics with typical crews in construction projects, limited in time as well as capacity. As such, every part of the cooperation with the LLMs must be divided into clear activities and communicated with the chatbot. Finally, the whole process of creating the WBS and its departmentalization and modularization will support the students in becoming more aware of their own cognitive processes, indirectly fostering metacognitive awareness and problem-solving skills [50].

3. Module Design Methodology

Recent studies that have aimed to integrate LLMs, such as ChatGPT, to (i) directly teach science, engineering, technology and mathematics (STEM) concepts [53,54]; (ii) summarize reports (research or practice) related to specialized fields, such as off-shore construction [55]; (iii) produce academic reports [56]; and (iv) support the translation of specialized/domain specific terminology [57]. In this study, the curriculum involved the structured integration of LLMs to foster the effective interaction of the student with LLMs to bridge the technology literacy gap—as a translation tool—in computer code generation. This is since LLMs, due to their architecture and nature, are prone to errors and inherent biases, such as extrapolation errors and hallucination. As such, if the students rely solely on the LLM to complete tasks without prior knowledge and allow the LLM to make decisions, the likelihood of incorrect results increases [58]. Furthermore, the reliance on LLMs to make decisions will create dependency and decrease the possibility of independent critical thinking and learning [56].

Promoting critical [59] and independent thinking [23,39,41] together with life-long learning [60] is particularly important to prepare graduates to (i) uphold the integrity of academic institutions as well as democratic values, a pillar for a free society [61] and (ii) achieve proficiency and competence in self-regulation [62], creative problem solving [42], and rational decision making under uncertainty [63,64], particularly in light of the growing uncertainties regarding the rapid advancement of technology. As such, collaboration is necessary where the LLM provides solutions and information for informed decision making, but ultimately, the human (in our case student) orchestrates, oversees, directs, and evaluates the solutions, and ultimately decides on the best course of action.

Considering the modules’ overall structure discussed in Section 1.2, LLMs were integrated into the curriculum at three levels: (i) the lecture content to teach effective interaction with LLMs to achieve desired results; (ii) individual assignments to solve practical problems; and (iii) a group term project as a part of a PBL. The design of each is provided with more detail in the following.

3.1. Lecture Content Design: Teaching Effective Interaction with LLM Chatbot

To teach the LLM chatbot interaction, with due consideration of the overall structure of the module, two stages (consisting of three steps in total) were devised, namely (i) PLA as a part of a flipped classroom model and (ii) two live class activities in hybrid form, which gradually grow in difficulty. These are further explained in the following.

3.1.1. Flipped Classroom Pre-Lecture Activity

The students are given a 30 min PLA to watch before the in-person lecture. This lecture is available online and distributed through YouTube [11]. The lecture provides various steps required to complete a successful interaction with ChatGPT 3.0 to generate a computer program in Matlab 2022b programming software that (i) plots activity on arrow (AoA) networks; (ii) performs the critical path method (CPM) to find the duration of the project as well as the activities on the critical path; (iii) performs Monte Carlo simulation to incorporate the uncertainty and variability in the activity duration within the CPM; and (iv) plots the final project duration S-curve and the percentage of time each activity takes on the critical path (the criticality index). The lecture demonstrates many examples of unsuccessful interaction with the chatbot and strategies to avoid hallucination to generate a workable code with only a few interactions.

3.1.2. Live Hybrid Class Participatory Activities

Two live activities (in hybrid format), namely fitting a line to two-dimensional (2D) points and Monte Carlo simulation scheduling, were provided in two 90 min lectures. These activities are discussed in the following:

• In-class participatory activity #1 (90 min): The students start with writing a code to find the best-fit line parameters using the popular least squares adjustment method. This example was chosen since it is expected that the enrolled students have the basic mathematical skills to formulate the problem with ease. The lecture is structured as follows:
  • Formulating the mathematical basis (30 min): The students must use their basic mathematical knowledge to develop the closed formulations required for the estimation of the best fit line parameters (i.e., slope and intercept). It is important to mention that throughout the class, the students are provided guidance from the teaching assistant and the professor where appropriate to facilitate the completion of the tasks.
  • Step-by-step interaction with their LLM chatbot of choice (45 min): The students must use the strategies discussed in the pre-lecture recording to formulate the information provided to the chatbot step-by-step for successful code generation. Given that they have already formulated the problem, the method of communication and division/departmentalization of the equations becomes very important. Once the formulations are conveyed, the students must check whether the generated code in fact accomplishes the intended task of line fitting to points. Here, metrics for evaluation of success through defining effective KPIs become vital. The students are hence guided to prepare a few trivial examples to help validate the success of the code—for instance, a set of points on the x-axis (e.g., (0,0); (1,0); (2,0)), a set of points on the y-axis, or points following the equation x = y, where the slope and the intercept are known a priori. This simple planning process is a fundamental step that must be commonly carried out in all projects, including construction projects. The students are then given a set of points for the final assessment of completion. In other words, if the code outputs the correct slope and intercept, they can move to the next activity.
  • Reflection through think–pair–share TLA (15 min): The students reflect on the challenges, lessons learnt, number of interactions with the LLM chatbot, and the successful strategies employed to improve collaboration with the LLM.
• In-class participatory activity #2 (90 min): The students write a code to complete the Monte Carlo simulation scheduling method, shown in the pre-lecture recording. In this interaction, due to the inherent complexity of the problem, the students quickly notice that the LLM will start hallucinating unless the tasks are clearly defined and divided into small and modular sub-tasks, and the inputs and outputs for each sub-task are communicated in advance. The students must then determine the KPIs for each of the delegated sub-tasks. In other words, the students cannot complete the task by asking ChatGPT to write a Matlab code that completes all the tasks required to develop a program for the Monte Carlo simulation from scratch. The structure of this lecture follows that of the first activity, mentioned above.

3.2. Assignment Design

Three individual assignments in total were designed to directly utilize the LLM chatbot of choice (e.g., ChatGPT or Gemini) to solve a real-world and practical CEM problem. The designed problems were complex and open-ended, and as such the LLMs of choice cannot solve them directly without exhibiting considerable artifacts, such as hallucinations. It is important to mention that the problems were also tested every semester using the top five chatbot LLMs, found in the LMSYS Chatbot Arena Leaderboard [65], to ensure that the LLMs cannot simply solve the problem without the necessary interactions from the user. As such, the students must devise a workable solution to the problem and guide the LLM of choice to translate their proposed methodology from written human language to the software language, Matlab. The three assignments are described in the following.

3.2.1. Solar Panel Installation Waste Minimization

Solar panel installation waste minimization, shown in Figure 1, was given to the students enrolled in the DEC module. This assignment was used to support the students in defining an objective function, decision variables, and constraints to solve a linear combinatorial optimization problem within the one-dimensional (1D) cutting-stock problem. The successful completion of this assignment was crucial in completing an important (but more complex) part of the final term project for the DEC module, the robotic arm optimization. The latter will be discussed in Section 3.3.

3.2.2. Floor Object Detection from Laser Scanner Point Clouds

Floor object detection from laser scanner point clouds, shown in Figure 2, was given to the students enrolled in the FIM module. In this assignment, the students were allowed to become familiarized with an important topic regarding big data analytics, namely dimensionality reduction. Furthermore, due to the many parts involved in the methodology, including histogram smoothing, outlier detection, and unsupervised clustering, many variations of the same general methodology could be generated. The solution hence required effective modularization of the problem’s methodology into smaller sub-functions/modules. Due to the existence of different variations, once an effective modularization was achieved, the LLM served to generate not only the code for the basic methodology, but also many variations for each separate part of the solution (i.e., function/module). The latter is important for fast and effective comparative evaluation and analysis to achieve the best solution.

3.2.3. App Development for Point Cloud Object Detection

App development for point cloud object detection in building information modeling (BIM) projects, shown in Figure 3, was given to the students enrolled in the FIM module. In this assignment, the students were tasked to direct LLMs to generate an app with a user interface for an algorithm they had proposed and implemented in a previous assignment. In particular, the app design needed to simulate a typical custom software development project in real-world construction projects, which have grown in prevalence markedly in recent years [8].

In summary, the three assignments were designed to, respectively, target (i) the effective mathematical formulation of problems for use in common optimization frameworks; (ii) the modularization and departmentalization of serial processes in computer programming; and (iii) the design of user interfaces and standalone app development which simulates a software development project.

3.3. Group Project Problem-Based Learning

In the final project for FIM, the interaction with LLMs was elective. However, as will be discussed in Section 4.1, the students in fact chose to utilize LLMs in various areas related to the final project of FIM, within the design and development of a standalone app. Given the elective nature of the utilization of LLMs in the final project of the FIM module, in this study, only the portion of the final project of the DEC module which required interaction with LLMs is explained. The final-term project of DEC, shown in Figure 4, involved the following:

• The point-cloud as-built modeling of a column;
• Design topology optimization of the column to minimize weight of structure; and
• AI-based robotic arm collaborative optimization within digital fabrication.

For the first two, the students were given a choice to utilize existing software together with manual intervention to complete the tasks. The latter robotic arm collaboration, however, was required to be automated. This requirement necessitated the students to formulate the objective function, decision variables, and constraints. The objective function and the constraints were, however, highly non-linear, which required the utilization of a meta-heuristic AI-based gradient free optimization strategy to approximate the optimal decision variables (i.e., the number of robotic arms required on each given day). The solution required interaction with LLMs to develop an AI-based optimization strategy [66], such as a genetic algorithm (GA) [67], in a software programming platform such as Matlab.

4. Results and Discussions

Two categories of findings are discussed, namely the successful completion of the task as a quantitative evaluation and anonymous student rating as a qualitative evaluation. Within the first category, the percentage of participants “completing”, “partially completing”, and “not completing” the task in each semester for each assignment was calculated. In the second category, the Teaching Quality Index (LQI) [68] as well as written comments on instruction, related to LLMs within the curricula, were presented and discussed.

4.1. Successful Completion of the Tasks

Table 1. Summary of percentage of students for each level of completion, specifically “Full” “Partial” and “Incomplete”.

PBL Project	Semester	Level of Completion (%)
		Full	Partial	Incomplete
Solar Panel Installation Waste Minimization (Figure 1)	Winter 2023	83.33%	16.67%	0.00%
Floor Object Detection from Laser Scanner Point Clouds (Figure 2)	Summer 2023	87.50%	12.50%	0.00%
	Summer 2024	81.25%	18.75%	0.00%
App Development for (BIM) Object Detection from Point Clouds (Figure 3)	Summer 2024	75.00%	25.00%	0.00%
Robotic Arm Collaboration Optimization (Figure 4)	Winter 2023	100.00%	0.00%	0.00%
Total on average		86.36%	13.64%	0.00%

shows the results of the percentage of participants for each semester completing the assignments. Partial completion is defined as completion of at least 50% of the grades associated with the task. Overall, it was observed that 86.4%, 13.6%, and 0.0% of students completed, partially completed, and did complete the task, respectively. To further elaborate on the effectiveness of the integration of LLMs within the curriculum, an example “control” group from the winter 2022 semester is given. In this semester, a simpler robotic arm design collaboration than that presented in the term project for DEC in winter 2023 was given. The simpler version could be solved using simple linear programming, whereas the more complex robotic arm collaboration could only be solved using AI-based metaheuristic algorithms due to the problem’s inherent non-linearity. Yet, no group in the control group solved the simpler problem using available automation and numerical optimization methods (they utilized trial and error with hand calculations), while all groups in winter 2023 were able to solve the more complex problem using a metaheuristic algorithm, such as GA, through collaboration with LLMs.

One final observation was regarding the final-term project of the FIM module. The students were generally flexible in their use of LLMs to complete this challenging problem; however, app development was not a part of the required deliverables, but highly encouraged. Given that the students learnt app development during their fourth individual assignment (Figure 3), all teams developed an app with a user interface with the help of LLM chatbots. This was a very important observation, since all three groups went above the basic requirements (on their own accord) to provide a final product as their deliverable output, which was not required to receive a top grade. Therefore, all teams found the fourth assignment not only useful for creating an app for the final project, but also impactful for their future career prospects given its elective nature.

4.2. Student Evaluation Results

LQI is the quality assurance index developed by the Executive and Strategy office of KIT. As per the teaching quality assurance standards, an LQI over 75% is considered within the uncritical greenlight rating and represents success [68]. In terms of LQI, the DEC module received three LQIs of 100% in winter 2021/22, 2022/23, and 2023/24. The FIM module received an LQI of 100% in summer 2021 and summer 2023, but received an LQI of 96.7% in summer 2024 (FIM was not evaluated in summer 2022 due to insufficient participation). All six LQIs are well above the standards for quality assurance and are thus considered successful.

Furthermore, the students also provided written comments within the module evaluation surveys (Appendix A). The outcomes of these comments were summarized within three key themes, namely the integration of AI and LLMs, the structure and content, and the character of the instructor, as follows:

• Integration of AI and LLMs:
  • Students found adopting LLMs, such as ChatGPT, beneficial for learning.
  • The use of LLM chatbots enabled improved efficiency in acquiring knowledge.
  • Coding sessions, supported by LLMs, enhanced programming skills in a comfortable and inclusive learning environment.
• Structure and Content of the Modules:
  • Students commented on the high quality of the module materials and documents.
  • The engaging PBL assignments made the learning process enjoyable.
  • The unique and innovative structure of the module expanded students’ horizons and stimulated interest in the DEC topic.
  • The balanced fusion of individual assignments and group projects promoted personal growth, accountability, and collaboration.
• Character of the Instructor:
  • Students valued the instructor’s extensive knowledge and practical experience.
  • The instructor’s enthusiasm, dedication, and motivation contributed to a positive learning atmosphere.
  • The supportive nature and patient teaching style fostered personal development and accessibility to module material.
  • The friendly and encouraging demeanor of the instructor created an inclusive learning environment.

I am convinced that the words taken directly from the anonymous student surveys attest to the level of positive influence the adopted teaching strategy and module content had on the learners throughout the six occasions that the modules were offered.

5. Conclusions and Future Developments

This study showcases one of the first attempts globally to integrate LLM chatbots within the module curriculum of DEC students. First, the interaction with LLM chatbots was taught through a PLA, made available online by the author of this manuscript [11], within a flipped classroom model. The learning process was then reinforced through two live lectures to solve two problems which gradually increased in complexity. To evaluate this competency and to help graduates of the program achieve a competitive advantage in their prospective careers, three individual assignments and group collaborative term projects were designed to help students directly interact with LLM chatbots. Finally, the preliminary results in terms of the completion rate of tasks (that would otherwise be almost impossible without formal training on software programming) along with positive student feedback on instruction showed promise for the successful integration of LLM chatbots into the CEM curriculum.

In terms of future developments, as explained in Section 3.2, the problems must be revisited every semester due to the rapidly growing capabilities of LLMs. This is generally expected based on the scientific improvements in LLMs (e.g., architecture, tunable parameters, etc.) and new data and interactions with the users through reinforcement learning. In fact, it was found to be necessary to change Assignment 4 of the DEC module, presented in Figure 1, since the most recent updates of ChatGPT 4o and o1 solved the problem with only a few interactions. Therefore, a more complicated cutting-stock problem was replaced to ensure that the students had the chance to effectively interact with the LLM by directing the LLM as an active decision maker, rather than as a passive observer.

Furthermore, it is also possible for the instructional team to interact with the LLM chatbots to generate a set of possible complex problems that are not easily solvable using different LLM chatbots. Here, it may even be advantageous to allow different LLM chatbots to solve and critique the problems and solutions generated by other chatbots. Currently, the DEC team is working to propose a sustainable framework to define unique problems that account for the rapid growth in LLM chatbots’ capabilities.

In conclusion, advancements in LLMs pose new challenges, particularly for the sustainable utilization of these models in DEC education. To address these limitations, the instructional team must leave sufficient contingency and capacity to achieve the following:

• Adjust the curriculum to directly teach advanced topics, enabling the learners to independently formulate solutions to complex problems while using LLMs solely as translation tools in computer code generation.
• Design and test new problems with the latest LLMs to ensure they cannot be easily solved by the models, thereby reducing the learner’s dependency on LLMs for problem-solving, and consequently fostering the growth of critical thinking in learners.
• Integrating LLM output validation methodologies, such as those used in NVIDIA Nemo and Guardrail AI, to control the structure, context and overall quality of the outputs for each component of the program, thereby supporting the students with further automation and streamlining.
• Evaluating LLM capabilities to customize specialized CEM software, supporting students to streamline and automate existing software in unconventional programming languages, such as F# (in Microsoft Project), Smalltalk (in VisualWorks), and Lua (in Trimble SketchUp).