8/15/2012

What is CSC?
A 3-day course
Founded in 1971 as a technical support
unit for Univac 1108
Elmer FEM Connected Finland to the Internet in
from PCs to supercomputers 1988
Reorganized as a company, CSC –
Scientific Computing Ltd. in 1993
Peter Råback, Thomas Zwinger, All shares to the Ministry of Education
Jonas Thies, Mikko Byckling and Culture of Finland in 1997
Operates on a non-profit principle
Facilities in Espoo, close to Otaniemi
campus and Kajaani
Staff ~200
Turnover 2009 21,9 million euros
Currently official name is:
”CSC – IT Center for Science Ltd.”

CSC as a Finnish IT Infrastructure for Research

PRACE in a nutshell
The volume of data is growing exponentially. To exploit the data for,
e.g., drug design, a global, constantly updating IT infrastructure is
needed (programs, DBs)
PRACE is building a permanent, pan-European
research infrastructure for high-end scientific
DB
DB computing
SW DB
DB
Astronomy Chemistry – Deploy systems of the highest performance level
CELL
GbE (Tier-0) with ensuring diversity of architectures
Biology PRACE training activities
Physics
– Face-to-face events: Workshops and Seasonal
Schools
Data and services over the Internet – Training Portal www.prace-ri.eu/training
– PRACE Advanced Training Centers

PRACE Advanced Training Centres Elmer Course – Day 1

Mandate 9.00 Registration & Coffee
9.30 Welcome & practical information
“The PRACE Advanced Training Centres will serve as
9.40 Overview of Elmer (PR)
European hubs of advanced, world-class training
10.30 Demonstration of ElmerGUI (TZ)
for researchers working in the computational
11.00 Guided exercises with simple walk-through examples in
sciences.” heat transfer, fluid dynamics, structural mechanics and
Vision electromagnetics. (PR, all)
“When all centres are functional, the PATC network is 12.00 Lunch
a visible and important part of the electronic ~13.00 Exercises continue
research infrastructure in Europe.” 13.30 Structure of Elmer simulation, MATC language (TZ)
14.30 Coffee
Six centers for 2012-2014
~14.50 Strategies for pre- and postprocessing (PR)
– Finland (CSC), France (MdS/GENCI), Germany 15.30 Exercises continue (include Gmsh & Paraview)
(GSC), Italy (CINECA), Spain (BSC), UK (EPCC) 16.30 End of Day
– Coordinated operation and curriculum contents
planning

1
 8/15/2012

Elmer Course – Day 2 Elmer Course – Day 3

9.00 Basic programming of Elmer; APIs & DefUtils (PR) 9.00 Exercises, Parallel Computing
10.00 Coffee 10.00 Coffee
10.30 Solution strategies with Elmer (TZ) 10.30 Parallel preconditioning, Trilinos, Hypre (JT)
11.00 Linear Algebra: Direct & Iterative Methods (JT) 12.00 Lunch
12.00 Lunch 13.00 Strategies for parallel pre- and postprocessing (PR)
13.00 Exercises on linear algebra, UDFs, MATC etc. 13.30 Exercises
14.30 Coffee 14.30 Coffee
14.50 Parallel computing with Elmer (PR) 15.00 Future developments (MB,PR?)
15.50 Demonstration of parallel use (TZ) 16.00 Concluding remarks, discussion (all)
16.30 End of day 16.30 End of course

~19 Dinner, if enough interest

Elmer – A finite element software

for multiphysical problems

Elmer
Open Source Finite Element Software
for Multiphysical Problems

ElmerTeam

CSC – IT Center for Science Ltd. ElmerGUI + ElmerSolver + ElmerPost

ElmerGrid
PATC Elmer course
CSC, August 2012
ElmerFront

ElmerGUI ElmerSolver
Graphical user interface of Elmer
– Based on the Qt library (GPL) Assembly and solution of the finite element equations
– Developed at CSC since 2/2008 Many auxiliary routines
Mesh generation
Good support for parallellism
– Plugins for Tetgen, Netgen, and
ElmerGrid Note: When we talk of Elmer we mainly mean ElmerSolver
– CAD interface based on
OpenCascade
Easiest tool for case specification > ElmerSolver StepFlow.sif
– Even educational use MAIN: ==========================================
MAIN: E L M E R S O L V E R S T A R T I N G
– Parallel computation
MAIN: Library version: 5.3.2
New solvers easily supported MAIN: ==========================================
through GUI MAIN:
MAIN: -----------------------
– XML based menu definition MAIN: Reading Model ...
Also postprocessing with VTK ...
...
SolveEquations: (NRM,RELC): ( 0.34864185 0.88621713E-06 ) :: navier-stokes
: *** Elmer Solver: ALL DONE ***
SOLVER TOTAL TIME(CPU,REAL): 1.54 1.58
ELMER SOLVER FINISHED AT: 2007/10/31 13:36:30

2
 8/15/2012

ElmerPost ElmerGrid
Based on the FUNCS program Creation of 2D and 3D structured meshes
– Rectangular basic topology
– written in late 80’s and early 90’s by Juha Ruokolainen
– Extrusion, rotation
All basic presentation types – Simple mapping algorhitms
– Colored surfaces Mesh Import
and meshes – About ten different formats:
– Contours, isosurfaces, Ansys, Abaqus, Fidap, Comsol, Gmsh,…
vectors, particles Mesh manipulation
– Animations – Increase/decrease order
Includes MATC language – Scale, rotate, translate
– Data manipulation Partitioning
– Derived quantities – Simple geometry based partitioning
– Metis partitioning
Output formats Example: > ElmerGrid 1 2 step –metis 10
– ps, ppm, jpg, mpg Usable via ElmerGUI
– animations – All features not accessible (partitioning, discont.
BC,…)

Elmer – Numerical Methods Elmer - Physical Models

Heat transfer Electromagnetics
Time-dependency
– Heat equation – Emphasis on steady-state and
– Static, transient, eigenmode, scanning – Radiation with view factors harmonic analysis
Discretization – convection and phase change – New Whitney element formulation
for magnetic fields
– Element families: nodal, edge, face, and p-elements, DG Fluid mechanics
Mesh movement (Lagrangian)
– Test functions: Galerkin, Stabilization, Bubbles – Navier-Stokes (2D & 3D)
– RANS: SST k-, k-, v2-f – Extending displacements in free
Linear system solvers surface problems
– LES: VMS
– Direct: Lapack, Umfpack, (SuperLU, Mumps, Pardiso) – ALE formulation
– Thin films: Reynolds (1D & 2D)
– Iterative Krylov space methods (HutIter & Hypre) Level set method (Eulerian)
Structural mechanics
– multigrid solvers (GMG & AMG) for “easy” equations (own & Hypre) – Free surface defined by a function
– General Elasticity
– Preconditioners: ILU, BILU, Parasails, multigrid, SGS, Jacobi,… (unisotropic, lin & nonlin) Electrokinetics
Parallellism – Plate, Shell – Poisson-Boltzmann

– Parallel assembly Acoustics Thermoelectricity

– Solution with selected methods – Helmholtz Quantum mechanics
– Linearized time-harmonic N-S – DFT (Kohn Scham)
Adaptivity Particle Tracker
Species transport
– For selected equations, works well in 2D
– Generic convection-diffusion ….
equation

Application Fields – Poll (Status 5/2012) Elmer – Selected multiphysics features

Solver is an asbtract dynamically loaded object
– Solver may be developed and compiled without touching the main library
– No upper limit to the number of Solvers
– Currently ~50 different Solvers, roughly half presenting physical
phenomena
Solvers may be active in different domains, and even meshes
– Automatic mapping of field values
Parameters of the equations are fetched using an overloaded
function allowing
– Constant value
– Linear or cubic dependence via table
– Effective command language (MATC)
– User defined functions with arbitrary dependencies
– As a result solvers may be weakly coupled without any a priori defined
manner
Tailored methods for some difficult strongly coupled problems
– Consistant modification of equations (e.g. artificial compressibility in FSI,
pull-in analysis)
– Monolitic solvers (e.g. Linearized time-harmonic Navier-Stokes)

3
 8/15/2012

Solution strategies for coupled problems

Solution strategies for coupled problems
Hierarchical solution

lterative solution

Monolithic solution

Short history of Elmer Elmer - Developers

1995 Elmer development was started as part of a national Current main developers at CSC
CFD program – CSC: Mika Malinen, Juha Ruokolainen, Peter Råback,
– Collaboration with TKK, VTT, JyU, and Okmetic Ltd. Sampo Sillanpää, Thomas Zwinger
After the initial phase the development driven by number of application Other/past developers & contributors
projects – CSC: Mikko Lyly, Erik Edelmann, Jussi Heikonen, Esko Järvinen, Jari
– MEMS, Microfluidics, Acoustics, Crystal Growth, Hemodynamics, Glaciology, …
Järvinen, Antti Pursula, Ville Savolainen,…
2005 Elmer published under GPL-license
– VTT: Martti Verho
2007 Elmer version control put under sourceforge.net
– Roughly 400 000 lines of code – TKK: Jouni Malinen, Harri Hakula, Mika Juntunen, Mikko Byckling
2010 Used wordwide by thousands (?) of researchers – Trueflaw: Iikka Virkkunen
– About 1500 downloads of the Windows binary each month
– Open Innovation: Adam Powell
– ~50000 visits to community forum from ~120 countries during last year
Readily available in major Linux systems – LGGE: Olivier Gagliardini, Fabien Gillet-Chaulet
Application projects are nowadays mainly international – University of Uppsala: Jonas Thies
– Used in a number of EU-projects – etc… (if your name is missing, please ask it to be added)
– Central tool in computational glaciology
May 2012 ElmerSolver library to be published under LGPL

16k Windows downloads at sf.net in a year

Elmer Simulations

Figures by Esko Järvinen, Mikko

Lyly, Peter Råback, Timo Veijola
(TKK) & Thomas Zwinger

4
 8/15/2012

CZ-growth: Transient simulation

Czockralski Crystal Growth Parallel simulation of silicon meltflows using stabilized finite
element method (5.4 million elements).
Most crystalline silicon is grown by the
Czhockralski (CZ) method
Simulation Juha Ruokolainen, animation Matti Gröhn, CSC
One of the key application when Elmer
development was started in 1995 Figures by Okmetic Ltd.

V. Savolainen et al., Simulation of large-scale

silicon melt flow in magnetic Czochralski growth,
J. Crystal Growth 243 (2002), 243-260.

MEMS: Inertial sensor MEMS: Microphone membrane

 MEMS provides an ideal field for multi-  MEMS includes often geometrical features that
physical simulation software
may be modeled with homogenization
 Electrostatics, elasticity and fluid flow are techniques
often inherently coupled
 Example shows the effect of holes in the
 Simulation shows the damping oscillations of a
motion of an accelerometer prototype perforated micromechnical membrane
Figure by VTI Technologies

A. Pursula, P. Råback, S. Lähteenmäki and J. Lahdenperä, Coupled FEM simulations of accelerometers

P. Råback et al., Hierarchial finite element simulation of perforated plates with
including nonlinear gas damping with comparison to measurements,
arbitrary hole geometries, MSM 2003.
J. Micromech. Microeng. 16 (2006), 2345-2354.

Microfluidics: Flow and heat transfer in a microchip Computational Hemodynamics

Cardiovascular diseases are the
leading cause of deaths in western
countries
Calcification reduces elasticity of
arteries
Modeling of blood flow poses a
challenging case of fluid-structure-
interaction
Artificial compressibility is used to
enhance the convergence of FSI
Electrokinetically driven flow coupling
Joule heating
Heat Transfer influences
performance
Elmer as a tool for prototyping
Complex geometry E. Järvinen, P. Råback, M. Lyly, J. Salonius. A
Complex simulation setup method for partitioned fluid-structure interaction Blood flow in carotid artery
T. Sikanen, T. Zwinger, S. Tuomikoski, S. Franssila, R. Lehtiniemi, C.- computation of flow in arteries. Medical Eng. & accounting FSI interaction
M. Fager, T. Kotiaho and A. Pursula, Physics, 30 (2008), 917-923
Microfluidics and Nanofluidics (2008)

5
 8/15/2012

RANS turbulence modeling VMS turbulence modeling

Comparison of k- vs. v2-f –turbulence models (red Large eddy simulation (LES)
provides the most accurate
& green line) presentation of turbulence without
the cost of DNS
Requires transient simulation
where physical quantities are
Plane flow with Re=395
averaged over a period of time
Variational multiscale method
Simulation J. Ruokolainen, CSC (VMS) by Hughes et al. Is a variant
of LES particularly suitable for
FEM
Interation between fine
(unresolved) and coarse (resolved)
scales is estimated numerically
No ad’hoc parameters Simulation
J. Ruokolainen, CSC

Thermal creep in light mills

Glaciology: Grand challenges
Glass container in a very low
Elmer is the leading code for
pressure < 10 Pa
3D ice flow simulation even
internationally Each ving has a black and
silver side
Elmer uses full Stokes
equation to model the flow When hit by light the light mill
rotates with silver side ahead
Currently the mostly used tool
in the area
– British Antarctic Survey
The physical explanation of the
light mills requires
– University of Grenoble
consideration of rarefied gases
– University of Sapporo
and thermal creep
Simulations of continental ice
These were studied in the
sheets very demanding
thesis project of Moritz Nadler,
Climate change makes the University of Tubingen, 2008
simulations very important Simulation T. Zwinger, CSC

Thermal creep in light mills EHDL of patterned surfaces

Solution of Reynolds &
nonlinear elasticity
equations
Simulation Bengt
Wennehorst,
Univ. Of Hannover,
2011

Simulation Moritz Nadler, 2008

6
 8/15/2012

Whitney element Solver Optimization in FSI

Elmer includes some tools that help in
the solution of optimization problems
Profile of the beam is optimized so
that the beam bends as little as
possible under flow forces Optimized profiles for Re={0,10,50,100,200}

Simulation by ”madstamm” Pressure and velocity distribution with Re=10

In elmerfem.org/forum
Simulation Peter Råback

Particle tracker - Granular flow

Quantum Mechanics
 Finite element method is used to solve the Kohn-Sham equations of
density functional theory (DFT)

 Charge density and wave function of the 61st eigenmode of fullerine C60
 All electron computations using 300 000 quadratic tets and 400 000 dofs

Simulation Mikko Lyly, CSC

Reasons to use open source software Ecosystem of Elmer

free as in ”beer” vs. free as in ”speech”

OS software Basic development of Elmer

New algorithms development CSC

New equations Large scale

Finnish university groups
comp. science Collaboration with selected companies
Collaborators, EU-projects: MMM@HPC, PRACE,…
Open Collaboration Computational glaciology,
Other developers
Linux packagers
…
Parallelism
1000’s of users around
the world (basic & advanced)
Open source users
Lisense costs

Free

7
 8/15/2012

Elmer – Infrastructure for Open Innovation Most important Elmer resources

University http://www.csc.fi/elmer
D – Official Homepage of Elmer
– Overview, examples, compilation, …
Institute Company – pointers to other sources of information
C A http://sourceforge.net/projects/elmerfem/
Company – Version control system: svn
B – Binaries
www.elmerfem.org
– Discussion forum, wiki & doxygen
Elmer As propriety Further information: [email protected]
GPL
Infrastructure modules modules
Elmer
Courses
HPC Thank you for your attention!
Elmer Elmer
Support
Library