2022 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)

Learning and Teaching Fluid Dynamics using Augmented and Mixed Reality
2022 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct) | 978-1-6654-5365-3/22/$31.00 ©2022 IEEE | DOI: 10.1109/ISMAR-ADJUNCT57072.2022.00186

Nitesh Bhatia* Omar K. Matar†

Department of Chemical Engineering, Imperial College London, South Kensington, London SW7 2AZ, UK

A BSTRACT to the pandemic, offering a VR-based course delivery model was

formidable. We diversified our efforts into exploring Augmented
We have been developing an interactive and multimodal platform
reality (AR) and Mixed Reality (MR) for teaching and learning.
to facilitate learning fluid dynamics with the rationale of using an
AR can be explained as an interactive experience involving virtual
immersive environment as a visualisation medium. Before the pan-
computer-generated information in a real-world environment [3, 9].
demic, we used our in-house virtual reality app to teach fluid dy-
However, MR is very similar to AR but differs in that MR users can
namics (FD), significantly enhancing student engagement. Since the
simultaneously interact with virtual and real-world elements [28,30].
COVID-19 pandemic struck, we have explored AR and MR appli-
While AR relies on a handheld device such as a mobile phone, MR
cations for scaling our remote online and hybrid teaching efforts.
requires custom headsets such as Microsoft Hololens [21]. MR
The work presented in this paper has two objectives. (i) Provide an
users can also use gestures and eye movements to interact with
AR learning medium for remotely located students. (ii) Provide a
the interface, making the MR experience more accurately emulates
student-paced instructional learning medium using MR for the hy-
real-life interactions [17].
brid or onsite students. To achieve this, we describe a methodology
in four parts. (i) A computational fluid dynamics data processing and The pandemic challenged us to continue our teaching efforts
distribution pipeline for generating 3D models for AR and MR. (ii) using immersive technologies. In the design of the new platform,
A platform-independent FD learning platform that uses WebXR for we faced a few challenges at different stages of the pandemic. The
rendering models in AR. (iii) Hololens-based instructional medium challenges and solutions are described below:
in MR for learning FD.(iv) A pedagogy design. We discuss the
1. Early pandemic: For effective learning, remote students using
results of a feasibility study on 18 hybrid learning students to assess
the platform must be able to visualize these models without
the effectiveness of the pedagogy design using MR. We conclude
requiring any app installs. Moreover, since the CFD models
that by using our platform, students can interactively visualise our
were generated in-house alongside the FD course, we needed
in-house fluid dynamics models aligned with the course work and
flexibility in updating the dataset. Hence, we opted for WebXR
acquire knowledge naturally and intuitively.
AR application built for the internet browser. Any headset or
Index Terms: Learning and Teaching—; Fluid Dynamics—; Aug- handheld device with an AR-compatible browser could run our
mented Reality—; Mixed Reality—; Remote Learning—; Hybrid app.
Teaching—;
2. Late pandemic: Due to social distancing restrictions due to
1 I NTRODUCTION the pandemic, on-campus students using the platform must
be able to visualize and interact with the teaching resources
In most engineering universities and colleges, such as Imperial Col- independently, i.e. in the absence of an instructor. We devel-
lege London, fluid dynamics is typically taught as a compulsory oped FD instructional teaching material in MR using Hololens,
course at the undergraduate and graduate levels [2]. It provides which any student could use flexibility on their availability.
students with their first experience with the continuum description
of fluid, mass, pressure, velocity, force, and other quantities in a
familiar, mechanical context that often requires three-dimensional 2 M ETHODOLOGY
visualization [24]. The fundamental basis of the subject is reason-
ably easy to understand. Still, the mathematical statement and the Figure 1 shows a schematic of this platform which includes four
consequences of Newtonian principles for continuously deforming components.
media are hard to imagine and often daunting to the student [32].
Educators have always explored new ways of teaching hard-to-grab 1. Custom CFD data processing pipeline:Due to hardware and
concepts [13, 16, 29]. With improvements in computational power software fragmentation, a platform-neutral application must
and advances in extended reality technologies, virtual learning plat- support a 3D model in multiple formats. E.g., glTF (GL Trans-
forms can now offer potential benefits for training and education in mission Format) is supported by Android [27], USDZ (uni-
a simulated setting [7, 15, 19, 20]. versal scene description) is used in iOS [10], and Hololens
Before the pandemic, we used our in-house virtual reality app to supports glB, a binary glTF [26]. Hence, we developed a cus-
teach fluid dynamics [33]. This learning platform has been designed tom pipeline to convert CFD models into platform-specific AR
to provide accurate and interactive models for students based on and MR Models.
computational fluid dynamics (CFD) data assets generated in-house
by our researchers [5, 6, 25]. The platform helped in significantly 2. Augmented Reality Application: To visualise the models
enhancing student engagement [22]. However, following the tran- in AR, we created a WebXR application and hosted it on the
sition to remote working, online learning and hybrid teaching due internet using Github Pages. During the coursework, we shared
the URL of the application with the students.
* e-mail: [email protected]
† e-mail:[email protected] 3. Mixed Reality Application: We hosted the MR Models on
the Microsoft Dynamics Cloud platform. Using Dynamics 365
Guides, we created several step-by-step instructional modules
for learning FD. The modules were aligned with the pedagogy
design explained in the next bullet.

2771-1110/22/$31.00 ©2022 IEEE 865

DOI 10.1109/ISMAR-Adjunct57072.2022.00186
Authorized licensed use limited to: UK ATOMIC ENERGY AUTHORITY. Downloaded on January 09,2024 at 11:23:22 UTC from IEEE Xplore. Restrictions apply.
 3 R ESULTS
3.1 Custom Data Processing Pipeline
Using Python and Jupyter Notebook, we developed a custom compu-
tational fluid dynamics data processing and distribution pipeline for
generating 3D models for AR and MR. Figure 1(I) shows pictorial
representation. The pipeline as Jupyter Notebook runs online and re-
motely, providing flexibility and support for collaborative work [8].
It expects CFD data to be in an open standard X3D format [14].
All the software commonly used for CFD data visualization and
processing, such as Paraview, support converting CFD models to
X3D [23]. Using meshlab, we created a custom filter to correct
the orientation, extract geometry and convert coloured vertices to
a texture [12]. The X3D model and the meshlab filter are inputted
in the python notebook to segment and store geometry as an OBJ
model and texture as a PNG file associated with the OBJ model.
In the next step, we convert the OBJ model to a GLTF file using
’obj2gltf’, a library based on nodejs [11]. For producing a USDZ file,
we have used ’usdzconvert windows’, a python library compiled for
windows based on open source implementation of ’usdzconvert’ by
Pixar and Apple [1, 31]. The advantage of the Jupyter Notebook
could be used online and remotely, providing flexibility and support
for collaborative work. Using this pipeline we created a repository
of AR and MR models listed in Table 1. Visualization of the models
are shown in Figure 2.
Note: We have provided the pipeline as a notebook hosted on
Github for broader adoption in developing and sharing AR and MR
models [4].
Figure 1: Schematic of the platform

4. Pedagogy Design: Our pedagogic approach involves three

steps. They are as follows:

(a) The first step involves a “Presentation” of module-

specific teaching materials in the form of a video lecture
in AR or a holographic video in MR.

(b) The second relates to the “Immersive Visualisation” of

FD models using AR and MR applications.

(c) The third step involves “Reflection”, which includes

questions and answers to evaluate the concepts learned.

For effective teaching and learning, we focused on developing a

fixed number of modules, e.g. flow in a pipe, past a bluff body, in
stirred vessels, in laminar, transitional, and turbulent regimes, and
some simulations of two-phase flows. As shown in Table 1, the
modules map directly onto the material taught in an FD module
offered to undergraduates and MSc students.

Table 1: List of FD teaching modules and the respective AR and MR

modules
FD Module AR Model MR Module
1 Stoke’s Flow SphereRe0, Inviscid Flow, Figure 2: Repository of AR and MR models generated using custom
SphereRe50, Creeping Flow data processing pipeline
SphereRe200
2 Two-Phase Aeration, Two-Phase Flow - 3.2 WebXR AR Application
Flow SmellBubble, Rising Bubble
For remote learning, we developed a WebXR app to view and interact
LargeBubble
with CFD Assets hosted on the web. The application uses Google’s
3 Taylor–Couette Taylor-Couette, Taylor–Couette model viewer library for visualizing models in AR [18]. In the
Flow Cavern Flow case of Android, the application uses the GLTF model, whereas,
4 Turbulent Flow PipeHB Turbulent Flow for IOS, it uses USDZ. The app seamlessly transitions to placing
and interacting with those FD models in AR using android or iOS

866

Authorized licensed use limited to: UK ATOMIC ENERGY AUTHORITY. Downloaded on January 09,2024 at 11:23:22 UTC from IEEE Xplore. Restrictions apply.
 devices. Figure 3 shows a screenshot of this platform-independent Table 2: Pedagogy design evaluation survey results
FD learning platform that uses WebXR to render AR models.
Min Max Mean Std
Note: We have provided the source code of our WebXR app on
HL was enjoyable to use. 2.0 5.0 4.3 0.8
Github for broader adoption in developing and sharing AR and MR
models [4]. HL was easy to use. 1.0 5.0 3.9 1.0
HL is quick to pick up. 1.0 5.0 4.2 1.1
3.3 Pedagogy Design Evaluation using MR Ease of Use Score 2.7 5.0 4.1 0.6
To assess the effectiveness of the pedagogy, we conducted a user HL would help me visualise dif- 1.0 5.0 4.2 1.1
study using our MR application. For this study, 18 students (56% ficult concepts easier/better than
males, 44% females) were selected, out of which half of the popu- other traditional methods like
lation was below 20 years and the remaining 20 to 30 years. The lectures.
students were primarily from Imperial College London. Most of HL is more engaging than 2.0 5.0 3.9 1.1
the students had no experience of using Hololens. 11% of partici- other traditional teaching meth-
pants had some prior experience in MR. The students had previous ods like lectures.
knowledge of FD and were also enrolled in the same FD course. HL makes learning more enjoy- 2.0 5.0 3.8 1.1
Before taking part in the study, participants first oriented detailing able/interesting.
the study. They were then instructed to complete a consent form. I found it helpful to see the 3D 1.0 5.0 4.4 1.0
Following this, participants put on the Hololens and carried out models.
the MR-based learning module. A play space of at least 2m x Engagement Score 1.8 5.0 4.1 0.9
2m was initially set up to allow the participants to carry out the HL worked well as a teaching 1.0 5.0 3.5 1.0
interaction activities without obstructions. After the study, a series resource.
of questions were asked about their experiences in the form of a HL would enhance my learning 1.0 5.0 3.5 1.2
Likert survey. The questions were based on five points Likert scale experience.
to assess the student agreement. The options included Strongly HL would help me visualise dif- 2.0 5.0 4.3 0.8
disagree, Disagree, Neutral, Agree, and Strongly agree, which were ficult concepts.
evaluated on scales ranging from 1 to 5. HL helped me to understand the 1.0 4.0 3.2 0.8
Table 2 shows the survey findings of the student responses on concept easier.
adopting the new MR-based teaching. For pedagogy assessment, we Hololens clarified fluid dynam- 2.0 5.0 3.7 0.8
divided the survey questions into three categories. The first three ics concepts.
questions in the survey evaluated the general usage of MR and were Hololens improved my fluid dy- 1.0 4.0 3.0 1.0
categorised to assess ease of use. From the survey, four questions namics understanding.
were grouped for measuring engagement in the immersive educa- Hololens motivated me to ex- 1.0 5.0 2.7 1.1
tion application. The survey included seven questions to determine plore fluid dynamics more.
the efficacy of learning. The survey data were processed and anal- Learning Score 2.3 4.1 3.4 0.5
ysed using SPSS. We computed the mean of the Likert scales of
the questions from the three categories and computed ease of use,
engagement and learning scores. As shown in Figure 4, ease of use the design of this pipeline, and we have provided source code
and engagement scores were above 4 (i.e. between agree to highly in our Github repository.
agree), and the learning score was above 3 (i.e. between neutral to
agree). 3. We have developed a platform-independent WebXR applica-
tion for visualizing CFD models in AR. The application was
mainly in use for teaching during the early pandemic. The
source code of the applicationcan be accessed in our Github
repository.
4. Using a pedagogic design approach, we have developed a
Hololens-based instructional application in MR for learning
FD. The application was mainly in use for hybrid learning
during the late pandemic stage. To evaluate pedagogic de-
sign’s effectiveness, we surveyed 18 hybrid learning students
using our MR-based FD learning application and discussed the
results.
We believe that AR and MR platform technologies can potentially
bridge the gap between remote, hybrid, and classroom education by
making the process of learning as exciting and engaging as possible.
We emphasise making students understand hard-to-visualise use
Figure 4: Visualization of responses for survey results categories cases by supplying the missing parts to produce a full spectrum of
multisensory, nonlinear, and immersive experiences.
4 D ISCUSSION AND C ONCLUSIONS Several possible future directions are speculated for further re-
There are four main contributions from this work. They are as search. Currently, we are working to improve the preparation and
follows: presentation of this platform by following pedagogical methods
such as behaviourism, cognitivism, constructivism and collaborative
1. We have presented a fluid dynamics educational platform that learning. Additionally, to understand students’ affective domain,
uses AR and MR visualization for learning in this paper. learning outcomes and engagement in this virtual learning environ-
ment, we are working to improve and automate behavioural data
2. We developed a CFD data processing and distribution pipeline collection and analysis. We conclude that by using our platform, stu-
to generate 3D models for AR and MR. The paper discusses dents can interactively visualise our in-house fluid dynamics models

867

Authorized licensed use limited to: UK ATOMIC ENERGY AUTHORITY. Downloaded on January 09,2024 at 11:23:22 UTC from IEEE Xplore. Restrictions apply.
 Figure 3: Augmented Reality and Mixed Reality Applications.(a) Screenshot of the interactive WebXR CFD model viewer app in Chrome Browser.
Please scan the QR code to view it. (b) Visualization of “Mixer with visoplastic fluid (Cavern)” CFD model in AR using a mobile phone. The arrows
in the model point in the direction of particle velocity, and the colour shows pressure distribution.(c) Hololens User Point of View Screenshots.
Pedagogy Stage 1: Presentation of the teaching materials in MR as holographic video and instructions.(d) Pedagogy Stage 2: Holographic
Visualisation of CFD models.Example in the figure shows a model of small rising bubble. (e) Pedagogy Stage 3: Reflection - Q&A feedback
activities.

aligned with the course work and acquire knowledge naturally and [4] N. Bhatia. Code for ismar2022 paper ’learning and teaching fluid dy-
intuitively. namics using augmented and mixed reality’. https://github.com/
ImperialCollegeLondon/mfg_fdxr_ismar2022, 2022 (accessed
ACKNOWLEDGMENTS August 22, 2022).
[5] N. Bhatia, K. Bryk, E. Salazar, C. Rigby, and O. Matar. Towards
The authors wish to thank Lyes Kahouadji, Ricardo Constante- gamified learning in immersive teaching of fluid mechanics. In APS
Amores, Gabriel N Farah Noroes Goncalves, Emily Xu, Fayyad Division of Fluid Dynamics Meeting Abstracts, pp. H29–002, 2019.
Uddin, and Mrudang Mathur. This work was supported in part by [6] N. Bhatia, G. Goncalves, L. Kahouadji, and O. Matar. Visualizing time
Imperial College London Pedagogy Transformation Programme and varying complex cfd flows in virtual reality. In APS Division of Fluid
the EPSRC PREMIERE Programme Grant (EP/T000414/1). Dynamics Meeting Abstracts, pp. Q01–001, 2020.
[7] S. Bryson. Virtual reality in scientific visualization. Communications
R EFERENCES of the ACM, 39(5):62–71, 1996.
[1] Apple. Ar creation tools. https://developer.apple.com/ [8] A. Cardoso, J. Leitão, and C. Teixeira. Using the jupyter notebook as
augmented-reality/tools/, 2022 (accessed August 22, 2022). a tool to support the teaching and learning processes in engineering
[2] K. Aung. Design and implementation of an undergraduate computa- courses. In International Conference on Interactive Collaborative
tional fluid dynamics (cfd) course. In 2003 Annual Conference, pp. Learning, pp. 227–236. Springer, 2018.
8–367, 2003. [9] J. Carmigniani and B. Furht. Augmented reality: an overview. Hand-
[3] R. T. Azuma. A survey of augmented reality. Presence: teleoperators book of augmented reality, pp. 3–46, 2011.
& virtual environments, 6(4):355–385, 1997. [10] B. J. Carvey. Developing ar for ios with usd and usdz. Technical report,

868

Authorized licensed use limited to: UK ATOMIC ENERGY AUTHORITY. Downloaded on January 09,2024 at 11:23:22 UTC from IEEE Xplore. Restrictions apply.
 Sandia National Lab.(SNL-NM), Albuquerque, NM (United States), developed to enhance fluid dynamics lectures.
2019. https://www.imperial.ac.uk/news/189866/
[11] CesiumGS. obj2gltf convert obj assets to gltf. https://github.com/ innovative-virtual-reality-software-developed-enhance/,
CesiumGS/obj2gltf, 2022 (accessed August 22, 2022). Jan. 2019 (accessed August 22, 2022).
[12] P. Cignoni, M. Callieri, M. Corsini, M. Dellepiane, F. Ganovelli,
G. Ranzuglia, et al. Meshlab: an open-source mesh processing tool.
In Eurographics Italian chapter conference, vol. 2008, pp. 129–136.
Salerno, Italy, 2008.
[13] J. Cimbala, S. Moeykens, A. Kulkarni, and A. Parihar. Using flowlab,
a computational fluid dynamics tool, to facilitate the teaching of fluid
mechanics. In ASME International Mechanical Engineering Congress
and Exposition, vol. 47233, pp. 17–27, 2004.
[14] L. Daly and D. Brutzman. X3d: Extensible 3d graphics standard
[standards in a nutshell]. IEEE Signal Processing Magazine, 24(6):130–
135, 2007.
[15] P. Dillenbourg, D. Schneider, P. Synteta, et al. Virtual learning environ-
ments. In Proceedings of the 3rd Hellenic conference information &
communication technologies in education, pp. 3–18. Archive Ouverte
HAL Rhodes, Greece, 2002.
[16] D. Fraser, R. Pillay, L. Tjatindi, and J. Case. Enhancing the learning of
fluid mechanics using computer simulations. Journal of Engineering
Education, 96(4):381–388, 2007.
[17] M. Funk, M. Kritzler, and F. Michahelles. Hololens is more than air
tap: natural and intuitive interaction with holograms. In Proceedings
of the seventh international conference on the internet of things, pp.
1–2, 2017.
[18] Google. Model viewer. https://modelviewer.dev/examples/
augmentedreality/index.html, Jan. (accessed August 22, 2022).
[19] M. Keppell, K. Souter, and M. Riddle. Physical and virtual learn-
ing spaces in higher education: Concepts for the modern learning
environment. IGI Publishing (IGI Global), 2011.
[20] M. Koskela, P. Kiltti, I. Vilpola, and J. Tervonen. Suitability of a
virtual learning environment for higher education. Electronic Journal
of e-Learning, 3(1):23–32, 2005.
[21] Y. Liu, H. Dong, L. Zhang, and A. El Saddik. Technical evaluation of
hololens for multimedia: A first look. IEEE MultiMedia, 25(4):8–18,
2018.
[22] K. Mahmoud, I. Harris, H. Yassin, T. J. Hurkxkens, O. K. Matar,
N. Bhatia, and I. Kalkanis. Does immersive vr increase learning
gain when compared to a non-immersive vr learning experience? In
International Conference on Human-Computer Interaction, pp. 480–
498. Springer, 2020.
[23] K. Moreland. The paraview tutorial. Sandia National Laboratories,
Tech. Rep. SAND, 2013.
[24] A. R. Paterson. A first course in fluid dynamics. Cambridge university
press, 1983.
[25] L. Picinali, L. Kahouadji, L. Mason, M. Sutton, N. Bhatia, A. Patapas,
and O. Matar. Use of multi-sensory immersive technologies in fluid
dynamics education. In APS Division of Fluid Dynamics Meeting
Abstracts, pp. H29–003, 2019.
[26] G. N. Rao, K. A. Kumari, D. R. Shankar, and K. Kharade. A com-
parative study of augmented reality-based head-worn display devices.
Materials Today: Proceedings, 2021.
[27] F. Robinet, R. Arnaud, T. Parisi, and P. Cozzi. gltf: Designing an
open-standard runtime asset format. GPU Pro, 5:375–392, 2014.
[28] S. Rokhsaritalemi, A. Sadeghi-Niaraki, and S.-M. Choi. A review
on mixed reality: Current trends, challenges and prospects. Applied
Sciences, 10(2):636, 2020.
[29] C. Sert and G. Nakiboglu. Use of computational fluid dynamics (cfd)
in teaching fluid mechanics. In 2007 Annual Conference & Exposition,
pp. 12–1527, 2007.
[30] M. Speicher, B. D. Hall, and M. Nebeling. What is mixed reality? In
Proceedings of the 2019 CHI conference on human factors in comput-
ing systems, pp. 1–15, 2019.
[31] tappi287. usdzconvert windows use apples usdzconvert on
windows/unix platform. https://github.com/tappi287/
usdzconvert_windows, 2022 (accessed August 22, 2022).
[32] J. Tu, G. H. Yeoh, and C. Liu. Computational fluid dynamics: a
practical approach. Butterworth-Heinemann, 2018.
[33] S. West. Innovative virtual reality software

869

Authorized licensed use limited to: UK ATOMIC ENERGY AUTHORITY. Downloaded on January 09,2024 at 11:23:22 UTC from IEEE Xplore. Restrictions apply.