HPC ANSYS 2012

Running ANSYS in
NTU HPC Center

ANSYS
CFX
FLUENT
LS-DYNA
AUTODYN
Ma Shengwei
Assistant Technical Manager
CAD-IT Consultants
January 2010
© 2010 CAD-IT Consultants and ANSYS, Inc. All rights reserved. 1 ANSYS, Inc. Proprietary
.
Welcome
© 2010 CAD-IT Consultants and ANSYS, Inc. All rights reserved. 2

Contents
• Objectives
• Why ANSYS HPC?
• An introduction to FEA
• Using ANSYS in HPC Center
• Using ANSYS LS-DYNA in HPC Center
• Using AUTODYN in HPC Center
• An introduction to CFD simulations
• Using FLUENT in HPC Center
• Using CFX in HPC Center
•Q&A
Objectives
• To run ANSYS (FEA and CFD) on the HPC’s

cluster
• Not an ANSYS product training, which will be
conducted regularly in subsequent months
ANSYS is more than a solver

ANSYS is more than one solver
Implicit solver: ANSYS
CFD solvers: FLUENT and CFX
Explicit solvers: Autodyn and ANSYS LS-DYNA

Why ANSYS HPC?

Why ANSYS HPC?
• Need for speed and accuracy

– Runtimes can be hours, days or even weeks
• Lots of computations!!
• Mesh fidelity continues to increase
– More equations to solve
• More computations!!
• More complex physics being analyzed
– More nonlinear solutions
• More computations!!

Why ANSYS HPC?
Past Computer Speed:

• Clock Speed
• Execution Optimization
Future Computer Speed:

• Multicore
• Parallel operations
Source: http://www.lanl.gov/news/index.php/fuseaction/1663.article/d/20085/id/13277
Why ANSYS HPC?
• 2 Types of memory systems

– Shared memory (SMP)  single box, workstation/server
– Distributed memory (DMP)  multiple boxes, cluster
HP Z800 Workstation
NCSA TeraGrid IA-64 Linux Cluster
Why ANSYS HPC?
• Domain decomposition approach

– Break problem into N pieces (domains)
– “Solve” the global problem independently within each
domain
– Communicate information across the boundaries as
necessary
Processor 3
Processor 1
Processor 4
Processor 2

Why ANSYS HPC?
2009
► 15 % spent on R&D ►Ideal scaling to 2048 cores (fluids)
► 570 software developers

►Teraflop performance at 512 core (structures) 2010
►Parallel I/O (fluids)
► Strategic partner ►Domain Decomposition introduced (HFSS 12) 2007 - 2008
relationships ► Optimized performance on multicore
2005 - 2006 processors
►Parallel meshing (fluids) ►1st One Billion cell fluids simulation
►Support for clusters using Windows HPC
2005 -2007
2001 - 2003
►Distributed sparse solver
►Parallel dynamic moving/deforming mesh
►Distributed PCG solver
►Distributed memory particle tracking
►Variational Technology
1998-1999 ►DANSYS released
►Integration with load management systems ►Distributed Solve (DSO) HFSS 10
►Support for Linux clusters, low latency interconnects
►10M cell fluids simulations, 128 processors
2000 2004
►1st company to solve 100M structural DOF
1994 - 1995
►Parallel dynamic mesh refinement and coarsening 1999 - 2000
►Dynamic load balancing ►64bit large memory addressing
►Shared memory multiprocessing (HFSS 7)
1993
st
►1 general-purpose parallel CFD with 1994
interactive client-server user environment ►Iterative PCG Solver Introduced for large structural analysis
1990 1990
► Shared Memory Multiprocessing for structural simulations
1980 1980’s
► Vector Processing on Mainframes
© 2010
2008 CAD-IT
ANSYS,Consultants
Inc. All rights
andreserved.
ANSYS, Inc. All rights reserved. 10 ANSYS, Inc. Proprietary
Why ANSYS HPC?
• ANSYS focus on HPC is ongoing…

– Ongoing optimization and performance tuning
• Dynamic load balancing; optimized resource mapping, compiler evaluation
– Architecting for next level scalability
• Performance at 10,000 cores or more; increased core density and GP-GPUs
• Innovative mechanical solvers: Multilevel PCG, 2D parallel DSPARSE fronts
• Hybrid distributed/shared memory and vector processing paradigms
– Scalability across all components and full simulation process
• Meshing, setup, solver, I/O, visualization, optimization…
• Parallel for linear dynamics, including mode superposition-based analyses
• Distributed domain solver, especially for contact nonlinearities
• Partial factorization (in-core substructuring) for localized nonlinearities
– Usability
• Multi-component parallel execution environment, job scheduler support
• Hardware fault tolerance, system performance tracking and debugging

An introduction to FEA

• Finite Element Analysis (FEA) is a way to simulate loading

conditions on a design and determine the design’s response
to those conditions.
• The design is modeled using discrete building blocks called
elements with solution variables at nodes
– Each element is comprised of nodes, Historical Note
with each node having degrees of
• The finite element method of
freedom (DOF). structural analysis was created
by academic and industrial
– FEA involves solving the DOF, such researchers during the 1940s
as translations, at each node through 1960s.
– The elements provide a relationship • The underlying theory is over
100 years old, and was the basis
of the DOF of each of its node for pen-and-paper calculations in
the evaluation of suspension
– The equations are coupled, and the bridges and steam boilers.
DOF (response) of the entire system
based on a loading is solved for.
• A mesh is generated from CAD geometry:

– the CAD model is an idealization of the physical system.
– the mesh is a mathematical discretization of this idealized
geometry to obtain nodes and elements.
– The finite element model, which has a finite number of
unknowns (DOF), can only approximate the response of
the physical system, which has infinite unknowns.

• Because a mathematical model is used to analyze the

physical system, the following items are important to consider
before getting started:
– Element behavior
• Geometric and spatial dimensionality
• Shape functions
– Analysis type
• Linear or nonlinear
• Frequency or time domain
– Domain to model
• Analysis of component, sub-assembly, or entire assembly

• There are four items that are controlled by elements:

– DOF: for example, a thermal element has temperature DOF (solve for
temperature field) whereas a structural element may have
translational DOF (displacements in x, y, and z)
– Element shape: 2D elements are typically triangular or quadrilateral,
whereas 3D elements may be tetrahedral, hexahedral, pentahedral, or
pyramid in form.
– Dimensionality: an element has both geometric and spatial
dimensionality. For example, a quad element is geometrically 2D;
however, if it is spatially 2D, it represents a planar element whereas if
it is spatially 3D, it represents a shell element
– Assumed shape functions: the choice of element also implies the
shape function, or the variation of DOF along the edge of the element.
Typically, lower-order elements only have nodes at the corners
whereas higher-order elements have nodes along the edge.

• There are many types of elements, some of which will be reviewed

next, based on topology:
– Point elements
– Line elements (beam, link, or 2D shell elements)
– 2D planar elements
– 3D shell elements
– 3D solid elements
• One may view 3D solid elements as the most general with other
elements including idealizations/assumptions but being more
computationally efficient.

Analysis Types (examples)
K x  F 
M x C x K x  F 
M x K  S x  0

Beyond structural analysis

• Heat transfer analyses allow users to calculate the temperature
distribution in the system
– Steady-state or transient analyses available
– Analysis is conduction-based (conduction in solid is solved)
• Hence, radiation, convection, heat flux are treated as boundary conditions
– Special provisions for thermal contact, phase change, etc.
– Composite shells with through-thickness temperature variation
– Easily couple with structural analysis for thermal-stress calculations


Acoustic analyses are used to solve wave propagation
– Harmonic, modal, or transient analyses available
– Can be coupled with structure for vibro-acoustic analyses
– Not to be confused with CFD, which is typically meant for noise generation.
Linear wave equation (Helmholtz) used to solve propagation of sound
in a medium
– Wave-absorption elements available for acoustic radiation
– Animation of sound waves in a room


• Low-Frequency Electromagnetics involve solving for electric
and magnetic fields
– Scalar, vector potential along with edge-based elements available
– Static (magnetostatic, electrostatic), harmonic (AC), transient analyses
available
– Circuit coupling possible
– Flux lines shown in animation on right of moving stator
– Used for motors, solenoids actuators, transformers, MEMS devices,
etc.
• High-Frequency Electromagnetics solves the vector Helmholtz
equation (acoustics is scalar Helmholtz equation)
– Modal or harmonic analyses available
– Used for RF switches, antennae design, EMI/EMC, etc.
– Animation of waves in RF connector below
Multiphysics: direct coupling
A finite element that couples the effects of interrelated physics

within the element matrices or load vectors, which contain all
necessary terms required for the coupled physics solution.
• Direct coupling for multiphysics solutions
• Strong (matrix) or weak (sequential) coupling
PLANE223 SOLID226 SOLID227

2-D 8-node 3-D 20-node brick 3-D 10-node
quadrilateral tetrahedron

Multiphysics: sequential coupling

A robust, automated method for solving sequentially coupled physics
problems. Sequential coupling uses iterative coupling where each
physics is solved sequentially and each matrix equation solved
separately. The solver iterates between each physics field until the
loads transferred across physics interfaces converge.
Time Loop
• Implicit sequential coupling Stagger Loop
• Explicit sequential coupling Field Loop ( i=1,n)
Field Solutions
Solution Options
Loads Send/Receive
End Field Loop

End Stagger Loop
© 2010 CAD-IT Consultants and ANSYS, Inc. All rights reserved. 23 End Time Loop
Multiphysics: one-way load transfer
• Transfer results data from one simulation and apply the results
as loads in another
• Available in the Workbench environment
• Automatic load mapping
– Solution quantities mapped to new model
• Fluid pressures, temperatures, Joule heat, etc.
• Many applications:
– Thermal-stress, fluid-structure interaction with small displacements,
Joule heating with constant properties, etc.

Param- Meshing
terization
MULTIPHYSICS
Fluids
BREADTH Structural Thermal
CAD In-house
DEPTH Emag Workflow
Import Solution
ANSYS’ technical depth and

breadth, provides the foundation
for true multiphysics simulation. Post-
processing

• “Implicit” and “Explicit” refer to two types of time integration

methods used to perform dynamic simulations
• Explicit time integration is more accurate and efficient for
simulations involving
– Shock wave propagation
– Large deformations and strains
– Non-linear material behavior
– Complex contact
– Fragmentation
– Non-linear buckling
• Typical applications
– Drop tests
– Impact and Penetration

Impact Response of Materials

Solution Impact Velocity Strain Rate (/s) Effect
(m/s)
Implicit <10-5 Static / Creep
< 50 10-5 - 10-1 Elastic
Elastic-Plastic (material
50 -1000 10-1 - 101
strength significant)
Primarily Plastic (pressure

1000 - 3000 105 - 106 equals or exceeds material
strength)
Hydrodynamic (pressure
3000 - 12000 106 - 108 many times material
strength)
Vaporization of colliding
Explicit > 12000 > 108
solids
Typical Values for Solid Impacts

VELOCITY LOW HIGH
Deformation Global Local
Response Time ms - s µs - ms
Strain <10% >50%
Strain Rate < 10 s -1 > 10000 s -1
Pressure < Yield Stress 10-100 x Yield Stress

STATIC “QUASI” STATIC DYNAMIC
PUNCH
BLANK
DIE
Structural problems Metal forming Impact problems
SF=0 SF» 0 S F = ma
IMPLICIT METHOD
EXPLICIT METHOD

Implicit Time Integration Explicit Time Integration

• For linear problems, the time • Only stable if time step size is
step can be arbitrarily large smaller than critical time step
(always stable) size:
• For nonlinear problems, time
step size may become small
due to convergence difficulties
• Where w max = largest natural

circular frequency
• Due to this very small time
step size, explicit is useful
only for very short transients

Using ANSYS

Using ANSYS
ANSYS, Inc. Family of Products
• ANSYS Workbench – Complete environment for geometry
modeling, mesh manipulation, structural/thermal analysis, and
optimization, which is tightly integrated with CAD packages.
• ANSYS Mechanical APDL – Advanced mechanical and
multiphysics FEA solution capabilities utilizing the traditional ANSYS
user interface.
• ANSYS CFD – State-of-the-art CFD solvers, including CFX and
FLUENT.
• ANSYS AUTODYN – Explicit dynamic solver for transient non-
linear simulations involving large deformations and strains, non-
linear material behavior, non-linear buckling, complex contact,
fragmentation, and shock wave propagation.
• ANSYS LS-DYNA – LSTC’s LS-DYNA explicit dynamic solver
technology with the pre-/post-processing power of ANSYS software.
This powerful pairing can be used to simulate crash tests, metal
forging, stamping, and catastrophic failures.
• ANSYS ICEM CFD – Powerful meshing tools with general pre-
and post-processing features, including ICEM CFD for generating
complex CFD grids and AI*Environment for creating sophisticated
structural FEA meshes.
• ANSYS EKM – Engineering Knowledge Manager

Using ANSYS
Every analysis involves four main steps: Preliminary

• Preliminary Decisions Decisions
– Which analysis type?
– What to model?
– Which element type?
• Preprocessing Preprocessing
– Define Material
– Create or import the model geometry
– Mesh the geometry
• Solution
Solution
– Apply loads
– Solve
• Postprocessing
– Review results
– Check the validity of the solution Postprocessing

Using ANSYS
Interactive and Batch modes
Two ways of working with ANSYS: Interactive and Batch Modes

• Interactive mode allows you to interact “live” with ANSYS, reviewing
each operation as you go.
– Of the three main phases of an analysis — preprocessing, solution,
postprocessing — the preprocessing and postprocessing phases are best suited
for interactive mode.
• Batch mode allows you to submit a batch file of commands which
ANSYS runs in the background.
• In HPC center, you will mainly be in batch mode. You need prepare your
input files in interactive mode, usually on your workstations.

Using ANSYS
Starting ANSYS
Command Line Start-up –essential for this cluster
• Allows you to start ANSYS by entering a command at the
system level.
• /ansys_inc/v110/ansys/bin/ansys110 -[b|g] [options] [input]
/ansys_inc/v121/ansys/bin/ansys121 -[b|g] [options] [input]
Product Launcher
• Allows you to select and start ANSYS products and utilities.
• On Unix/Linux systems, issue: launcher110/121
• On Windows systems:
Start > Programs > ANSYS 11.0 (or 12.1) > ANSYS Product
Launcher

Using ANSYS
• Launcher has four tabs: File Management,

Customization/Preferences, Distributed Solver
Setup, and MFX-ANSYS/CFX Setup.
• Launcher is used to
select the Simulation
Environment (i.e.:
ANSYS, ANSYS Batch,
ANSYS Workbench, MFX
- ANSYS/CFX, LS-DYNA
Solver.)
• Available license and

Add-on Modules are
selected here.

Using ANSYS
• File Management Tab is

used to specify the
Working Directory (where
all of your files will
reside) and a Job Name
of your choosing. The
default Job Name is
“file”.

Using ANSYS
• Customization/Preferences
tab allows you to set
memory options, run a
custom version of ANSYS,
define parameters, set the
language to be used by the
Graphical User Interface
(GUI), and specify a
Graphics Device (2D or 3D).
• A customizable file called

start100.ans can also be
read at start-up.

Using ANSYS
• Distributed Solver Setup

tab is used to set up
Parallel Performance
solving options.
• Other, more advanced

options such as
Distributed Processing
are also selected here..

Using ANSYS
• MFX - ANSYS/CFX
Setup tab is used to
set up ANSYS Multi-
field solving options
for ANSYS and CFX.

Using ANSYS
Utility Menu
Icon Toolbar Menu Input Line Raise/Hidden Icon
Output Contact Manager Icon

Window
Command Window Icon
Abbreviation Toolbar Menu

Model Control
Toolbar
Main Menu
Graphics Area
User Prompt Info

Current Settings

Using ANSYS
ANSYS Mechanical in Workbench

Using ANSYS
• Two ways of running Mechanical:
– Configured from within ANSYS Workbench …
– … or from a supported CAD system

Using ANSYS
Toolbars
Menus
Graphics Window
Tree Outline
Mechanical
Application Wizard
Details View Message Window
Status Bar

Using ANSYS
Demo:
1. Starting ANSYS
2. Example
3. Generate input
file

Using ANSYS
Demo:
1. Edit the job script
2. Submit a job
3. Monitor status

Using ANSYS
• Files you need know:

jobname.log: Log file, ASCII.
• Contains a log of every command issued during the session.
• If you start a second session with the same jobname in the same
working directory, ANSYS will append to the previous log file
(with a time stamp).
jobname.err: Error file, ASCII.
• Contains all errors and warnings encountered during the session.
ANSYS will also append to an existing error file.
jobname.db, .dbb: Database file, binary.
• Compatible across all supported platforms.
jobname.rst, .rth, .rmg, .rfl: Results files, binary.
• Contains results data calculated by ANSYS during solution.
• Compatible across all supported platforms.

Using AUTODYN

Using AUTODYN
AUTODYN® - Focus
• Transient dynamic wave propagation
– Gases, Liquids, Solids + their
Interaction (FSI)
• Non-linear
– Material behaviour
– Structural behaviour
– Contact/Interaction
– Fluid Structure Interaction
• Large/extreme deformation
• Large/extreme strain and
fragmentation

Using AUTODYN
• Typical Applications
– Impact
– Ballistic Protection
• Low => Hypervelocity
– Energetic Systems
– Blast Propagation
– Blast Effects on Structures
– Materials Research

Using AUTODYN
Coupling
Euler (CFD) Solvers
Lagrange (FE) Solvers (Multi-material, Blast)
(Solids, shells & beams)
Solid / Gas / Fluid Flow
Structural Response
Complex Materials
Interactions Blast Waves
Solid Impact
FSI (Deforming Structures)
Bonding / Combined Blast &

Coupling
Contact Impact Loading
Mesh Free Solver

ALE Solver
Hypervelocity Impact
Fluid-Structure Interaction
Brittle Material Fracture / with Strong Structures
Fragmentation
Bonding / Contact
Using AUTODYN
• Material models
Equations of State Strength Models Failure Models
(Hydrodynamic) (Plasticity)
Models Also Available in

Explicit Dynamics (ANSYS)
Using AUTODYN
Context-sensitive Help Set Up Model Import Models

LS-DYNA
Linked to extensive Interactively Nastran
and readable HTML Manuals
ICEM-CFD
Create and View

Animations Access Extensive
GIF, MPEG, AVI, GFA Material Library
View Results Open Architecture

User Subroutines
Advanced FE & CFD Visualization
User Variables
Time Histories
Set Up
Run Analysis Interactively Parallel Processing
Visually Refreshed Load / Save Model Host Configuration
Stop / Start at any time Startup Files Decomposition
Interrogate progress Restart Files Load Balancing

Using AUTODYN
Notes:
• Linux---solver only till current release (R12.1)
• Windows--- standalone or within Workbench

Using AUTODYN
View panel

Using AUTODYN
Dialog Panel
Dialog Window
Button
opens
window

Using AUTODYN
• Files associated with a particular model use the model Ident

In the filename
Ident_0.ad Cycle 0 Save file
Ident_428.ad Cycle 428 Save file
Ident_0_428.adres Cycle 428 Results file
Ident.his History file
Ident.sum Summary file
Ident.prt Print file
Ident.log Log file
Ident.uhs User History file
Ident.fil Remap file

Using AUTODYN
• Other AUTODYN files

autodyn.ini AUTODYN initialization file
matlib_name.mlb Material library file
partlib_name.slb Part library file
setting_name.set Plot settings file
slide_name.seq Sequence file for slideshow
slide_name_nn.gif Slideshow files
animation_name.gif GIF animation file
animation_name.gfa GFA animation file

Using AUTODYN
Demo
1. Autodyn interface
2. Parallel settings
3. Edit job script
4. Submit a job
5. Useful resources

Using ANSYS LS-DYNA

Using ANSYS LS-DYNA
Restart file (d3dump)

ANSYS /PREP7 written at frequency
Preprocessing (database) specified by EDDUMP.
Creates Jobname.DB
ANSYS /SOLU
-mesh, materials, loads, etc. EDSTART continues
LS-DYNA solver task
Writes and submits Jobname.K analysis from specified
- standard LS-DYNA ASCII input file d3dump (restart) file.
ANSYS /POST1 LS-PREPOST (phase 1)

LS-PREPOST (phase 3) & ANSYS /POST26
General postprocessing Postprocess binary files
Postprocess ASCII output files
Reads Jobname.RST - d3plot
- GLSTAT, MATSUM, SPCFORC, etc.
- general binary result data Similar to Jobname.RST
EDOUT,File and EDREAD, ,File
EDRST,Freq EDRST,Freq
ANSYS /POST26
LS-PREPOST (phase 2)
Time history postprocessing
Postprocess time history binary results files
Reads Jobname.HIS
- d3thdt
- selective binary results data
Similar to Jobname.HIS
EDHIST,Comp and EDHTIME,Freq
EDHIST,Comp and EDHTIME,Freq

Using ANSYS LS-DYNA
ANSYS files generated during an ANSYS LS-DYNA run:

Jobname.K
• LS-DYNA input file that is automatically generated upon execution of the
ANSYS SOLVE command.
• Contains geometry, load, and material data that exists in ANSYS database
• ASCII input file 100% compatible with LS-DYNA version 971
• File can also be manually generated using the EDWRITE command:
Solution > Write Jobname.K
Jobname.RST
• Explicit dynamics results file that is nearly identical to standard ANSYS .RST
• Primarily used to review results in the general ANSYS postprocessor (POST1)
• Contains results at a relatively small number of time steps (e.g., 10 - 100)
Jobname.HIS
• Explicit dynamics time history results files used in POST26
• Contains results for a subset of nodes and/or elements of the model
• Typically contains results at significantly more time steps than Jobname.RST
Use /CONFIG, NRES to write more than 1000 results sets to .HIS and .RST files

Using ANSYS LS-DYNA
ANSYS LS-DYNA solvers
• Serial: lsdyna110 (lsdyna121 for R12.1)

• SMP solver: lsdyna110 –NCPU=
• MPP solver: lsdyna110 -dis -machines
Possible options (LS-DYNA Keyword User’s Manual, Version 971):
I=inf O=otf G=ptf D=dpf F=thf U=xtf T=tpf A=rrd M=sif J=jif S=iff Z=isf1
L=isf2 B=rlf W=root E=efl X=scl C=cpu K=kill V=vda Y=c3d BEM=bof
{KEYWORD} {THERMAL} {COUPLE} {INIT} MEMORY=nwds NCPU= ncpu
PARA=para ENDTIME=time NCYCLE=ncycle JOBID=jobid
D3PROP=d3prop

Using ANSYS LS-DYNA
Demo
1. On keyword file;
2. Edit job script
3. Submit a job;
4. Useful resources

Break and Discussion

Introduction of CFD
• Computational fluid dynamics (CFD) is the science of predicting fluid flow,
heat and mass transfer, chemical reactions, and related phenomena by
solving numerically the set of governing mathematical equations
– Conservation of mass
– Conservation of momentum
– Conservation of energy
– Conservation of species
– Effects of body forces
– Etc.
• The results of CFD analyses are relevant in:

– Conceptual studies of new designs
– Detailed product development
– Troubleshooting
– Redesign
• CFD analysis complements testing and experimentation by reducing total

effort and cost required for experimentation and data acquisition.
Introduction of CFD
• ANSYS CFD solvers are based on the

finite volume method
– Domain is discretized into a finite set of Control
control volumes Volume*
– General conservation (transport) equations for

mass, momentum, energy, species, etc. are
solved on this set of control volumes
Fluid region of pipe flow is
discretized into a finite set of
control volumes.
Unsteady Convection Diffusion Generation Equation Variable

– Partial differential equations are discretized Continuity 1
X momentum u
into a system of algebraic equations Y momentum v
– All algebraic equations are then solved Z momentum w
numerically to render the solution field Energy h
* FLUENT control volumes are cell-centered (i.e. they correspond

directly with the mesh) while CFX control volumes are node-centered

Introduction of CFD
• Problem Identification
Problem Identification 1. Define your modeling goals
1. Define goals 2. Identify the domain you will model
2. Identify domain
• PreProcessing and Solver Execution
Pre-Processing 3. Create a solid model to represent the
domain
3. Geometry
4. Design and create the mesh (grid)
4. Mesh
5. Set up the physics (physical models,
5. Physics material properties, domain
6. Solver Settings properties, boundary conditions, …)
6. Define solver settings (numerical
Update Model
schemes, convergence controls, …)

Solve 7. Compute and monitor the solution
7. Compute solution
• Post-Processing
9.
Post Processing 8. Examine the results.

9. Consider revisions to the model.
8. Examine results
Introduction of CFD
Problem Identification
1. Define goals
2. Identify domain
• What results are you looking for (i.e. pressure drop, mass flow rate),
and how will they be used?
– What are your modeling options?
• What physical models will need to be included in your analysis (i.e.
turbulence, compressibility, radiation)?
• What simplifying assumptions do you have to make?
• What simplifying assumptions can you make (i.e. symmetry,
periodicity)?
• Do you require a unique modeling capability?
– User-defined functions (written in C) in FLUENT or User FORTRAN
functions in CFX
• What degree of accuracy is required?
• How quickly do you need the results?
• Is CFD an appropriate tool?

Introduction of CFD
Problem Identification
1. Define goals
2. Identify domain
• How will you isolate a piece of the

complete physical system?
• Where will the computational

domain begin and end?
Domain of Interest
– Do you have boundary condition
as Part of a Larger
information at these boundaries? System (not modeled)
– Can the boundary condition types
accommodate that information?
– Can you extend the domain to a point
where reasonable data exists?
Domain of interest
isolated and meshed
for CFD simulation.
• Can it be simplified or
approximated as a 2D or
axisymmetric problem?
Introduction of CFD
Pre-Processing
3. Geometry
4. Mesh
5. Physics
6. Solver Settings
• How will you obtain a solid model of the

fluid region?
– Make use of existing CAD models?
• Extract the fluid region from a solid part?
– Create from scratch?
• Can you simplify the geometry?
– Remove unnecessary features that would
complicate meshing (fillets, bolts…)?
– Make use of symmetry or periodicity?
• Are both the solution and boundary
conditions symmetric / periodic?
• Do you need to split the model so that
boundary conditions or domains can be Solid model of a
Headlight Assembly
created?
Introduction of CFD
3.
Pre-Processing
Geometry
Hex or Tet?
4. Meshing • For flow-aligned geometries,
5. Physics
quad/hex meshes can provide
6. Solver Settings
higher-quality solutions with fewer
• For complex geometries, cells/nodes than a comparable tri/tet
quad/hex meshes show no mesh
numerical advantage, and you
– Quad/Hex meshes show reduced
can save meshing effort by numerical diffusion when the mesh
using a tri/tet mesh or hybrid is aligned with the flow.
mesh – It does require more effort to
– Quick to generate generate a quad/hex mesh
– Flow is generally not aligned with • Meshing tools designed for a
the mesh
specific application can streamline
• Hybrid meshes typically the process of creating a quad/hex
combine tri/tet elements with mesh for some geometries.
other elements in selected • GAMBIT, TGrid, ICEM CFD, ANSYS Meshing
regions
Introduction of CFD
Pre-Processing • For a given problem, you will need to:

3. Geometry
4. Mesh
– Define material properties
5. Physics • Fluid
6. Solver Settings
• Solid
For complex problems • Mixture
solving a simplified or 2D
problem will provide – Select appropriate physical models
valuable experience with the • Turbulence, combustion, multiphase,
models and solver settings
for your problem in a short
etc.
amount of time. – Prescribe operating conditions
– Prescribe boundary conditions at all
boundary zones
– Provide initial values or a previous
solution
– Set up solver controls
– Set up convergence monitors
Introduction of CFD
Solve • The discretized conservation equations are

7. Compute solution solved iteratively until convergence.
• Convergence is reached when:

– Changes in solution variables from one iteration
to the next are negligible.
• Residuals provide a mechanism to help
monitor this trend.
– Overall property conservation is achieved
• Imbalances measure global conservation
– Quantities of interest (e.g. drag, pressure drop) have
reach steady values.
• Monitor points track quantities of interest.
• The accuracy of a converged solution is

dependent upon:
– Appropriateness and accuracy of physical models.
A converged and mesh-
– Mesh resolution and independence
independent solution on a well-
posed problem will provide useful – Numerical errors
engineering results!
Introduction of CFD
• Examine the results to review solution
9. Update Model
and extract useful data
– Visualization Tools can be used to
Post Processing
8. Examine results
answer such questions as:
• What is the overall flow pattern?
• Is there separation?
• Where do shocks, shear layers, etc.
form?
• Are key flow features being
resolved?
– Numerical Reporting Tools can be used

to calculate quantitative results:
• Forces and Moments
• Average heat transfer coefficients
Examine results to ensure property conservation and
correct physical behavior. High residuals may be • Surface and Volume integrated
caused by just a few poor quality cells. quantities
• Flux Balances

Introduction of CFD
Geometry Meshing Setup Solve Post-

CAD
Processing
ANSYS
CAD CFX-Pre CFX-Solver
Plug-in ANSYS CFD-Post
DesignModeler
CAD
Meshing FLUENT
System Setup/Solve/Post
ANSYS Workbench

Introduction of CFD
Standalone or Inside Workbench?
• Running ANSYS CFD inside the Workbench

– Simplifies the workflow
• Geometry, Mesh, Setup, Solution and Results steps shown on the Project
Schematic
• Easier to update a project when a change is made
– E.g. after a geometry change a single click updates the Mesh, Setup, Solution
and Results
– Allows you to easily link to other Analysis Systems and Components
– Is necessary when performing DesignExploration (parametric studies)

Introduction of CFD
Standalone or Inside Workbench?

• Running ANSYS CFD standalone:
– Less computational overhead
– Produces a simpler directory / file structure on disk
– But no direct association between geometry, mesh, setup and results
files
• Each component must be updated
• No built-in automation for parametric studies
– Less automation
• E.g mesh needs to be manually imported into CFX-Pre
For this cluster, you can prepare the files either in

standalone or in WB in your own PC. Then submit your
job in this cluster to the batch solver.
Introduction of CFD
FLUENT or CFX?

Using FLUENT

Using FLUENT
Procedures
Prepare the geometry– DesignModeler,GAMBIT, ICEM CFD,CAD software
Meshing– GAMBIT, TGrid, ICEM CFD, ANSYS Meshing
Setup the problem—FLUENT
Solving –FLUENT (batch or interactive)
Postprocessing—FLUENT, CFD POST
FLUENT within ANSYS Workbench

(DM—AMP/ICEM CFD—FLUENT—CFD POST)
For large problems that need this HPC cluster, you can
prepare the case and data files in WB or FLUENT
standalone.

Using FLUENT
• Fluid flow and heat transfer • Multiphase flows
– Momentum, continuity, energy – Discrete Phase Model (DPM)
equations – Volume of Fluid (VOF) model for
immiscible fluids
– Radiation
– Mixtures
• Turbulence – Eulerian-Eulerian and Eulerian-granular
– RANS-based models (Spalart- – Liquid/Solid and cavitation phase
Allmaras, k–ε, k–ω, Reynolds stress) change
– Large-eddy simulation (LES) and
detached eddy simulation (DES) • Moving and deforming mesh
• Species transport – Moving zones
• Volumetric reactions • Single and multiple reference frames
(MRF)
– Arrhenius finite-rate chemistry • Mixing plane model
– Turbulent fast chemistry • Sliding mesh model
• Eddy Dissipation, non-Premixed, – Moving and deforming (dynamic) mesh
premixed, partially premixed (MDM)
– Turbulent finite-rate chemistry
• EDC, laminar flamelet, composition • User-defined scalar transport
PDF transport equations
– Surface Reactions
82
© 2010 CAD-IT Consultants and ANSYS, Inc. All rights reserved.
Using FLUENT

Using FLUENT

Using FLUENT

Using FLUENT
• FLUENT can be run in batch mode using
journal files.
• A journal file is a text file which contains
TUI commands which FLUENT will execute
sequentially. Sample Journal File
• The FLUENT TUI accepts abbreviations of ; Read case file
the commands; for example, rc example.cas.gz
– ls Lists the files in the working folder ; Initialize the solution
– rcd Reads case and data files /solve/initialize/initialize-flow
; Calculate 50 iterations
– wcd Writes case and data files
it 50
– rc/wc Reads/writes case file ; Write data file
– rd/wd Reads/writes data file wd example50.dat.gz
– it Iterate ; Calculate another 50 iterations
• TUI commands in a batch file can be used it 50
to automate operations in a non- ; Write another data file
interactive mode. wd example100.dat.gz
– The TUI commands file/read-bc and ; Exit FLUENT
file/write-bc can be used for reading and exit
writing the settings for a FLUENT session to yes
and from a file, respectively.
– A web-based training module is available which
explains this process

Using FLUENT
– scalable > 1024!!

– No other software has
shown this!

Using FLUENT
• Solution parameters Set the solution parameters
– Choosing the solver

– Discretization schemes Initialize the solution
• Initialization Enable the solution monitors of interest
• Convergence Calculate a solution

Modify solution
parameters or grid
– Monitoring convergence
– Stability
Check for convergence
• Setting Under-relaxation
• Setting Courant number
Yes No
– Accelerating convergence
• Accuracy Check for accuracy No

– Grid Independence
– Adaption Yes
Stop

Using FLUENT
• There are two kinds of solvers

available in FLUENT – Pressure Pressure-Based Pressure-Based Density-Based
based and Density based. (segregated) (coupled) (coupled)
Solve U-Momentum
• The pressure-based solvers take
momentum and pressure (or Solve V-Momentum Solve Mass,
pressure correction) as the primary Solve Mass Momentum,
Energy,
variables. Solve W-Momentum & Momentum
Species
– Pressure-velocity coupling algorithms
Solve Mass
are derived by reformatting the Continuity;
continuity equation Update Velocity
• Two algorithms are available with
the pressure-based solvers:
Solve Energy
– Segregated solver – Solves for
pressure correction and momentum
sequentially. Solve Species
– Coupled Solver (PBCS) – Solves
pressure and momentum Solve Turbulence Equation(s)
simultaneously.
Solve Other Transport Equations as required

Using FLUENT
• Density-Based Coupled Solver

– Equations for continuity, momentum, energy and species
(if required) are solved in vector form.
– Pressure is obtained through an equation of state.
– Additional scalar equations are solved in a segregated
fashion.
• The DBCS can be run either explicit or implicit.

– Implicit – Uses a point-implicit Gauss-Seidel / symmetric
block Gauss-Seidel / ILU method to solve for variables.
– Explicit: uses a multi-step Runge-Kutta explicit time
integration method

Using FLUENT
• The pressure-based solver is applicable for a wide range of flow regimes

from low speed incompressible flow to high-speed compressible flow.
– Requires less memory (storage).
– Allows flexibility in the solution procedure.
• The pressure-based coupled solver (PBCS) is applicable for most single
phase flows, and yields superior performance to the standard pressure-
based solver.
– Not available for multiphase (Eulerian), periodic mass-flow and NITA cases.
– Requires 1.5–2 times more memory than the segregated solver.
• The density-based coupled solver (DBCS) is applicable when there is a
strong coupling, or interdependence, between density, energy, momentum,
and/or species.
– Examples: High speed compressible flow with combustion, hypersonic flows, shock
interactions.
• The implicit option is generally preferred over explicit since it has a very
strict limit on time step size
• The explicit approach is used for cases where the characteristic time scale
of the flow is on the same order as the acoustic time scale. (e.g.:
propagation of high-Ma shock waves).

Using FLUENT
• Field variables (stored at cell centers) must be interpolated to the faces of

the control volumes.
• Interpolation schemes for the convection term:

– First-Order Upwind – Easiest to converge, only first-order accurate.
– Power Law – More accurate than first-order for flows when Recell < 5 (typ. low Re flows)
– Second-Order Upwind – Uses larger stencils for 2nd order accuracy, essential with tri/tet
mesh or when flow is not aligned with grid; convergence may be slower.
– Monotone Upstream-Centered Schemes for Conservation Laws (MUSCL) – Locally 3rd
order convection discretization scheme for unstructured meshes; more accurate in
predicting secondary flows, vortices, forces, etc.
– Quadratic Upwind Interpolation (QUICK) – Applies to quad/hex and hybrid meshes,
useful for rotating/swirling flows, 3rd-order accurate on uniform mesh.

Using FLUENT
Interpolation Methods for Pressure

• Interpolation schemes for calculating cell-face pressures when using
the segregated solver in FLUENT are available as follows:
– Standard – The default scheme; reduced accuracy for flows exhibiting large
surface-normal pressure gradients near boundaries (but should not be used
when steep pressure changes are present in the flow – PRESTO! scheme
should be used instead.)
– PRESTO! – Use for highly swirling flows, flows involving steep pressure
gradients (porous media, fan model, etc.), or in strongly curved domains
– Linear – Use when other options result in convergence difficulties or
unphysical behavior
– Second-Order – Use for compressible flows; not to be used with porous
media, jump, fans, etc. or VOF/Mixture multiphase models
– Body Force Weighted – Use when body forces are large, e.g., high Ra natural
convection or highly swirling flows

Using FLUENT
• Pressure-velocity coupling refers to the numerical algorithm which

uses a combination of continuity and momentum equations to derive
an equation for pressure (or pressure correction) when using the
pressure-based solver.
• Four algorithms are available in FLUENT

– Semi-Implicit Method for Pressure-Linked Equations (SIMPLE)
• The default scheme, robust
– SIMPLE-Consistent (SIMPLEC)
• Allows faster convergence for simple problems (e.g., laminar flows with no
physical models employed).
– Pressure-Implicit with Splitting of Operators (PISO)
• Useful for unsteady flow problems or for meshes containing cells with
higher than average skewness
– Fractional Step Method (FSM) for unsteady flows.
• Used with the NITA scheme; similar characteristics as PISO.

Using FLUENT
Demo 1 Demo 2
1. GAMBIT FLUENT in WB
2. FLUENT standalone
3. Solving
4. Post-processing Demo 3
1. Prepare case file
2. Prepare input file
3. Modify job script
4. Submit a job

Using CFX

Using CFX
Meshing CFX-Pre CFX-Solver CFD-Post
GAMBIT Interactive Interactive or Interactive

ICEM CFD session, on batch. Test session, on
TGrid your run on your your
ANSYS workstation workstation. workstation
meshing Submit your
job to this
cluster

Using CFX

Using CFX
• The Advection Scheme refers to the way the

advection term in the transport equations is
modeled numerically
– i.e. the term that accounts for bulk fluid motion
– Often the dominant term
Unsteady Advection Diffusion Generation
• Three schemes are available, High

Resolution, Upwind and Specified Blend
– Discussed in more detail next
• There is rarely any reason to change from

the default High Resolution scheme

Using CFX
• Solution data is stored at nodes, but variable values are required

at the control volume faces to calculate fluxes
• The upstream nodal values (up) are interpolated to the integration

points (ip) on the control volume faces using:
ip =  up +    r
– Where  is the variable gradient and r is the vector between the

upstream node and the integration point
– In other words, the ip value is equal to the upstream value plus a correction
due to the gradient
–  can have values between 0 and 1 …
Using CFX
Using CFX
Using CFX
• When multiple files

are loaded you can
select Case
Comparison from
the Outline tree
– Automatically
generates
difference variables SST k-
and plots
Difference Plot
• Expression syntax:
– function()@CASE:#.Location
– E.g: areaAve(Pressure)@CASE:1.Inlet
Using CFX
• .cfx files contain mesh and Import Mesh

physics data and can be opened .cmdb, .cfx5, .def, .res, …
by CFX-Pre
Open .cfx
.cfx, .def, .res
CFX-Pre (CFX-Pre Database)
• .def files contain mesh and
.def (Solver Input or Definition File)
physics data and can be run in
the Solver .out
CFX-Solver (Solver Output File)
.res (Results File)
• .res files contain mesh, physics
and results data and can be .def, .cmdb
CFD-Post
.cst
(Mesh Files) .cse
opened in CFD-Post
• CFX-Pre can also open .def and .res files to recover the mesh and physics. Some
unused data (e.g. unused Materials) is not written to the .def / .res files and is
therefore not recovered.
• The CFX-Solver can use .res files to continue a run or as an initial guess
• CFD-Post can also open .def and .cmdb files to view the mesh
Using CFX
Demo 1 Demo 2
1. DM 1. Prepare definition file
2. Meshing 2. Modify job script
3. CFX-Pre 3. Submit a job
4. CFX-Solver
5. CFD-POST
Questions?

HPC ANSYS 2012

Uploaded by

Copyright:

Available Formats

HPC ANSYS 2012

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

HPC ANSYS 2012

Uploaded by

Copyright:

Available Formats

Running ANSYS in

NTU HPC Center

© 2010 CAD-IT Consultants and ANSYS, Inc. All rights reserved. 2

• To run ANSYS (FEA and CFD) on the HPC’s

ANSYS is more than a solver

© 2010 CAD-IT Consultants and ANSYS, Inc. All rights reserved. 4

© 2010 CAD-IT Consultants and ANSYS, Inc. All rights reserved. 5

• Need for speed and accuracy

© 2010 CAD-IT Consultants and ANSYS, Inc. All rights reserved. 6

Past Computer Speed:

Future Computer Speed:

• 2 Types of memory systems

• Domain decomposition approach

© 2010 CAD-IT Consultants and ANSYS, Inc. All rights reserved. 9

► 570 software developers

• ANSYS focus on HPC is ongoing…

© 2010 CAD-IT Consultants and ANSYS, Inc. All rights reserved. 11

© 2010 CAD-IT Consultants and ANSYS, Inc. All rights reserved. 12

• Finite Element Analysis (FEA) is a way to simulate loading

• A mesh is generated from CAD geometry:

© 2010 CAD-IT Consultants and ANSYS, Inc. All rights reserved. 14

• Because a mathematical model is used to analyze the

© 2010 CAD-IT Consultants and ANSYS, Inc. All rights reserved. 15

• There are four items that are controlled by elements:

© 2010 CAD-IT Consultants and ANSYS, Inc. All rights reserved. 16

• There are many types of elements, some of which will be reviewed

© 2010 CAD-IT Consultants and ANSYS, Inc. All rights reserved. 17

Analysis Types (examples)

© 2010 CAD-IT Consultants and ANSYS, Inc. All rights reserved. 18

Beyond structural analysis

© 2010 CAD-IT Consultants and ANSYS, Inc. All rights reserved. 19

Beyond structural analysis

© 2010 CAD-IT Consultants and ANSYS, Inc. All rights reserved. 20

Beyond structural analysis

Multiphysics: direct coupling

A finite element that couples the effects of interrelated physics

PLANE223 SOLID226 SOLID227

© 2010 CAD-IT Consultants and ANSYS, Inc. All rights reserved. 22

Multiphysics: sequential coupling

End Field Loop

Multiphysics: one-way load transfer

© 2010 CAD-IT Consultants and ANSYS, Inc. All rights reserved. 24

BREADTH Structural Thermal

ANSYS’ technical depth and

© 2010 CAD-IT Consultants and ANSYS, Inc. All rights reserved. 25

• “Implicit” and “Explicit” refer to two types of time integration

© 2010 CAD-IT Consultants and ANSYS, Inc. All rights reserved. 26

Impact Response of Materials

< 50 10-5 - 10-1 Elastic

Primarily Plastic (pressure

Typical Values for Solid Impacts

Deformation Global Local

Strain <10% >50%

Strain Rate < 10 s -1 > 10000 s -1

Pressure < Yield Stress 10-100 x Yield Stress

© 2010 CAD-IT Consultants and ANSYS, Inc. All rights reserved. 28

STATIC “QUASI” STATIC DYNAMIC

Structural problems Metal forming Impact problems

© 2010 CAD-IT Consultants and ANSYS, Inc. All rights reserved. 29