EGI-InSPIRE

Fire Simulation using FDS

(Fire Dynamics Simulator)

EGI-InSPIRE RI-261323 www.egi.eu

 FDS Simulation

• Fire Dynamics Simulator (FDS)

• Widely used system for the simulation of fires in
different human structures, developed in National
Institute of Standards and Technology (NIST), USA
(fire.nist.gov/fds)
FDS simulates:
• Pyrolysis
• Low-speed heat and smoke transfer
• Thermal radiation
• Flame spread
• Fire suppression by sprinklers

Amsterdam, 24-26, 2012

EGI-InSPIRE RI-261323 www.egi.eu
 FDS Simulation
• FDS Physical Principles: Conservation of
Mass, Species, Momentum and Energy

Amsterdam, 24-26, 2012

EGI-InSPIRE RI-261323 www.egi.eu
 FDS Simulation

• FDS Physical Principles

• System of PDE and equation of the state for these
unknowns: density, velocity temperature, pressure and mass
fractions
• Divergence of the momentum equations gives Poisson
equation for H(u,p)

• Inputs : geometry of the problem, material properties and

other parameters. FDS reads input parameters from a single
text file which provides all the necessary information to
describe the fire scenario.

Amsterdam, 24-26, 2012

EGI-InSPIRE RI-261323 www.egi.eu
 FDS Simulation

• FDS Numerical method

• Explicit predictor-corrector finite difference
scheme, which is second order accurate in
space and time
• Poisson equation for modified pressure is
solved in every time step by a direct FFT-
based solver that is part of a library
CRAYFISHPAK

Amsterdam, 24-26, 2012

EGI-InSPIRE RI-261323 www.egi.eu
 FDS Simulation

• FDS Compiling and Building

• FDS version 5.3.3. is used
• Compilers used
• GNU 4.1.2 and 4.4.0 (gcc, gfortran, OpenMP)
• Open MPI 1.4
• 4 programming models are build
• sequential, MPI, OpenMP, and hybrid
MPI+OpenMP

Amsterdam, 24-26, 2012 6

EGI-InSPIRE RI-261323 www.egi.eu
 FDS Simulation

• FDS Support
• To automate the process of simulation runs, for
each configuration an fds-manager script was
created which provides for the following actions:
• it accepts and checks input arguments specifying the
FDS input file and cluster configuration (the number of
nodes, cores, and eventually the number of MPI
processes and OpenMP threads)
• it produces an fds-submission script which serves as
an input to the Portable Batch System (PBS)
• it realizes the execution of the FDS simulation using
the previously generated job-submission script

Amsterdam, 24-26, 2012 7

EGI-InSPIRE RI-261323 www.egi.eu
 FDS Simulation

• FDS Running
• Experiments are carried out on the HP blade
cluster of the configuration:
• 16 blade nodes, each comprising of 2 quad-core
CPU Intel Xeon X5570@ 2.9GHz, 24-48GB RAM,
500GB hard disk
• 6 nodes are interconnected by the Infiniband
network (40Gb/s)
• 10 nodes are interconnected by the Ethernet
network (4Gb/s)
• Disk storage: 48 TB disk array
Amsterdam, 24-26, 2012 8
EGI-InSPIRE RI-261323 www.egi.eu
 FDS Simulation

• FDS Input
• FDS reads input parameters from a single
text file which provides all the necessary
information to describe the fire scenario.
• The computational domain representing the
given fire scenario included 12-25 millions
rectangular mesh cells dependent on the
resolution.
• Simulations with the given fire scenario
consumed 8-48GB RAM.
Amsterdam, 24-26, 2012 9
EGI-InSPIRE RI-261323 www.egi.eu
 FDS Simulation

• FDS Output
• The numerical solution to the governing
equations is written to various types of files. All
output quantities must be explicitly declared (in
the input file) at the start of the calculation.
• Simulations with the given fire scenario
produced output data files of amount 7-20GB.
• The simulation result can be visualized using the
Smokeview program that reads FDS output files
and produces animations on the computer
screen.
Amsterdam, 24-26, 2012 10
EGI-InSPIRE RI-261323 www.egi.eu
 FDS Simulation

• Requirements for running FDS

simulations on Cluster/Grid
• FDS simulations with the real scenario represent
long-time, computational intensive and memory
consuming jobs
• Hardware requirements
• HP, reliable, and failure-free computing resources
involving a sufficient amount of memory
• Software requirements
• Fortran 90 and C compilers including OpenMP
library, MPI, MPI-Start
Amsterdam, 24-26, 2012 11
EGI-InSPIRE RI-261323 www.egi.eu
 FDS Simulation

• Requirements for running FDS

simulations on Grid
• Grid middleware functionalities
• the proxy credential delegation and renewal
• the submission of parallel MPI and OpenMP jobs
• monitoring the status of submitted jobs
• the job output retrieval
• optionally, the job perusal (the real time output
retrieval) enabling the inspection of the job output
in real time
• the basic data management services
Amsterdam, 24-26, 2012 12
EGI-InSPIRE RI-261323 www.egi.eu
 FDS Simulation

• Fire Scenario
• We have constructed a model of a two-lane road
tunnel with dimensions 10x180x7m with 2 fans
located on the tunnel ceiling at the distance 50m and
140m from the left entrance.
• The fire source was represented by burning of a
flammable liquid in a pool with dimensions 2x3m
placed in the distance 92m from the left entrance of
the tunnel, 1.1m above the floor level.
• The initial air temperature in the whole tunnel was
set to 20°C.
• The total duration of the simulation was 150s.
Amsterdam, 24-26, 2012 13
EGI-InSPIRE RI-261323 www.egi.eu
 FDS Simulation

• Scheme of the Tunnel

Wall Temp. Measurement
Fire
Jet Fans

4
3
2
1

Gas Phase Measurements Points

Amsterdam, 24-26, 2012

EGI-InSPIRE RI-261323 www.egi.eu
 FDS Simulation

• Computational Requirements
Resolution 20 cm 10 cm 5 cm
Cells Number 1,62 mil. 12,96 mil. 103,68 mil.

• Simulation time for 10 cm resolution is 377

hours. It increases approximately with the
fourth power of increasing resolution.

Amsterdam, 24-26, 2012

EGI-InSPIRE RI-261323 www.egi.eu
 FDS Simulation

• Parallel Processing and the Domain

Decomposition
• Computational domain is decomposed into
meshes
• Each mesh can be assigned to specific MPI
process
• Velocity values at the mesh boundaries are
then averaged in order to maintain stability

Amsterdam, 24-26, 2012 16

EGI-InSPIRE RI-261323 www.egi.eu
 FDS Simulation
• Simulation Results – Smoke Density at the
50th, 57th, 63th, 150th s

Amsterdam, 24-26, 2012

EGI-InSPIRE RI-261323 www.egi.eu
 FDS Simulation
• Simulation Results – Velocity at the
50th, 57th, 63th, 150th s

Amsterdam, 24-26, 2012

EGI-InSPIRE RI-261323 www.egi.eu
 FDS Simulation
• Simulation Variants and their Performance
Sim. Mesh Division MPI Cells Max Cells Time Wall Max/ c(1M)/c
Description process [mil.] Per Mesh Steps Clock Min
[mil.] Time Mesh
[hrs] CPU
[hrs]
1M 1M: 180m, Seq. 12.96 12.96 33041 377.2 375.8 1.00
10cm 375.8
3M 3M: 60m, 10cm 3 12.96 4.32 31905 172.4 169.5 0.73
165.6
10M 1M: 60m, 10cm 10 43.20 4.32 60727 313.6 309.6 0.80
8M: 7.5m, 5cm 289.4
1M: 60m, 10cm
24M 24M: 7.5m, 24 12.96 0.54 31206 32.6 32.0 0.48
10cm 26.9
48M 48M: 3.75m, 48 25.92 0.54 68759 63.2 61.5 0.50
10x 5x10cm 53.8

Amsterdam, 24-26, 2012

EGI-InSPIRE RI-261323 www.egi.eu
 EGI-InSPIRE

Thank you for your attention!

Amsterdam, 24-26, 2012 20

EGI-InSPIRE RI-261323 www.egi.eu