See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/233758608

Using symbolic computation software packages in production of

multidimensional finite volume-based large eddy simulation codes

Article  in  International Journal for Numerical Methods in Fluids · February 2013

DOI: 10.1002/fld.3673

CITATIONS READS

3 609

2 authors:

Andrea Aprovitola Filippo Maria Denaro

Università della Campania "Luigi Vanvitelli" Università degli Studi della Campania "Luigi Vanvitelli
44 PUBLICATIONS   154 CITATIONS    51 PUBLICATIONS   363 CITATIONS   

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

New modelling in the integral-based LES View project

Conceptual design of Winged Re-Entry Vehicles View project

All content following this page was uploaded by Filippo Maria Denaro on 10 October 2017.

The user has requested enhancement of the downloaded file.

INTERNATIONAL JOURNAL FOR NUMERICAL METHODS IN FLUIDS
Int. J. Numer. Meth. Fluids (2012)
Published online in Wiley Online Library (wileyonlinelibrary.com/journal/nmf). DOI: 10.1002/fld.3673

Using symbolic computation software packages in production of

multidimensional finite volume-based large eddy simulation codes

A. Aprovitola1 and F. M. Denaro2, * ,†

1 Istituto Ricerche sulla Combustione, CNR Napoli, Italy
2 Dipartimento di Ingegneria Aerospaziale e Meccanica, Seconda Universita degli Studi di Napoli, Italy

SUMMARY
The numerical simulation of turbulence is one of the most challenging tasks in the field of the modern
computational science. At present, the most advanced approach is the large eddy simulation (LES) tech-
nique wherein a formal separation between resolved (large) and unresolved (small) scales of the motion is in
effect by means of a filtering operation applied onto the governing equations. However, LES requires very
sophisticated numerical discretizations in terms of both accuracy and efficiency. Often, the modelling of the
unresolved subgrid scale terms adds further computational complexities. This paper illustrates the suitabil-
ity in using software packages for symbolic computation (in the present case, Maple© for helping in the
production of subroutines for a new multidimensional, high-order accurate finite volume-based LES code.
Specifically, it will be detailed how producing, rapidly and efficiently, the routines for computing convective,
diffusive as well as subgrid scale modelling fluxes. It is particularly detailed how exploiting the package for
differential calculus and linear algebra for the analytical integration of the flux polynomials over the finite
volume faces. The structure of the LES code is illustrated, and an accuracy analysis of the local truncation
errors is performed comparing the third-order accurate multidimensional upwind and the classical second-
order centred reconstruction in the wavenumbers space. Then, some numerical results for the turbulent plane
channel and some brief points concerning the parallelization issue are addressed. Copyright © 2012 John
Wiley & Sons, Ltd.

Received 6 August 2011; Revised 17 November 2011; Accepted 13 March 2012

KEY WORDS: finite volume methods; multidimensional upwind; large eddy simulation; symbolic
language

1. INTRODUCTION

The numerical simulation of incompressible turbulent flows is one of the most challenging issues
in the field of the modern computational sciences. Often, the hyperbolic–parabolic mathematical
model governing such flows, the so-called Navier–Stokes (NS) equations system, is based on the
low-Mach hypothesis that leads to solve a parabolic–elliptic system of equations. Owing to the
consequent onerous need of CPU time for solving the elliptic part as well as to the necessity of
adopting very accurate formulations and turbulence modelling, the solution of such equations sys-
tem is still a challenging task for many practical flow problems such as external flows or internal
combustion. The presence of very different time-length scales increases further the computational
onerous. Actually, until when the computational performances will become powerful enough to
allow us performing the direct numerical simulation (DNS) also at high Reynolds number, that is,
a model-free simulation obtained resolving up to the smallest turbulent scales, it remains necessary
to work with reduced-scales engineering problems. This is obtained by means of statistical tools

*Correspondence to: Filippo Maria Denaro, Dipartimento di Ingegneria Aerospaziale e Meccanica, Real Casa
dell’Annunziata, Via Roma 29, 81031, Aversa (CE), Italy.
† E-mail: [email protected]

Copyright © 2012 John Wiley & Sons, Ltd.

 A. APROVITOLA AND F. M. DENARO

that drive us to resolve the flow motion only in terms of some averaged quantities. Compared with
other low-order statistical approaches, such as Reynolds average ones (RANS, URANS), large eddy
simulation (LES) is considered the most powerful tool for studying certain details of turbulence
because it resolves the large anisotropic flow scales while modelling only the more homogeneous
subgrid scale (SGS) small ones. The LES technique is based on a formal preliminary decomposition
of the velocity field between resolved (large), let us say vQ .x, t /, and unresolved (small) scales of the
motion. This separation is performed by means of a filtering operation applied onto the NS equa-
tions system and a consequent suitable closure modelling of the filtered equations. Now, advances
in the numerical analysis as well as in SGS modelling allow us thinking of LES as a promising tool
also for some engineering problems, for example, see [1–3].
However, LES requires sophisticated numerical discretization, in terms of both accuracy and
efficiency, and it is necessary to use the best-of-the-art SGS modelling. These tasks add further
difficulties because LES is much more sensitive than DNS to the accuracy in the discretization of
both domain and equations. Indeed, in DNS, the whole frequencies spectrum, ranging from the low-
est corresponding to the integral scale up to the highest lying in the dissipative subrange, must be
resolved. Thus, the numerical errors become prevalently manifest at high frequencies, close to the
Nyquist frequency (i.e. the maximum resolvable frequency associated to a domain discretization of
measure h) wherein the energy content is low. Hence, the intermediate inertial subrange, which is
significantly energetic because it contains the large structures of the flow, is accurately resolved. The
only problem in DNS consists in the capability of using very fine grids.
In LES, the filtering frequency (i.e. the maximum frequency that one chooses to take into account
in the computation by means of the filter width ) is often practically coincident with the Nyquist
frequency, particularly when implicit filtering is used. Because the filtering frequency is generally
chosen lying in the inertial subrange, the resolved scales could be strongly contaminated by numer-
ical errors (dissipative or dispersive behaving). Therefore, a high-order discretization is one of the
guarantees that an SGS model could efficiently represent the global effect of the unresolved scales.
Although spectral methods may represent the best choice to accomplish this requirement, they are
limited to special cases, often only of academic interest. Limitations in spectral methods are due
mainly to the use of spectral polynomials on irregular grids. Hence, owing to their wider feasibility,
finite volume (FV) methods are still of plenty interest in the LES field [4].
The discretization of the integral form of the NS equations appears opportune from both physical
and mathematical points of view, implying the use of discrete models, which allow the balanced
quantities, that is, mass and momentum, to be a priori conserved. This fact is particularly important
because it is known that the discrete conservation property ensures the correct wave velocity propa-
gation.‡ Furthermore, a supplementary appealing feature of the FV discretization consists in the fact
that the volume average represents a natural link with the smooth spatial filtering (the so-called top-
hat) in LES. However, for regular solutions, a volume-averaged field represents only a second-order
approximation of the pointwise unfiltered filed; therefore, the accuracy of FV methods is defined in
terms of the local truncation error (LTE) associated to the flux discretization [5]. More specifically,
an FV method that is accurate to order p means that the error in fluxes discretization behaves as
O.hp /, whereas the evaluation of the discretization error in terms of the difference between local
averaged-filtered and unfiltered pointwise solutions cannot be better than O.2 /. Because the filter
width and the grid measure could also be very different, owing to the subfilter resolution parameter
Q D = h > 1, convergence and order of accuracy of FV-based LES approach must be carefully
examined, for example, see [6]. A possible technique for getting higher-order filtered values, that is
O.p /, involves using the deconvolution approaches, as shown in [7, 8].
An original high-order multidimensional upwind FV integration, suitable for LES, is here pro-
posed, extending the one-dimensional (1D) methodology analysed in [7] and the deconvolution

‡
In case of non-regular solutions, the integral formulation remains the only one mathematically admissible and, from the
well-known Lax–Wendroff theorem, if the FV-based solution converges, it does so towards a physical solution of the
conservation law, namely the so-called weak solution [5]. Correct shock-wave propagation in inviscid flows is the most
intuitive case wherein conservation property is fundamental, but turbulent flows represent themselves a case of strong
interaction between waves at high wavenumbers that must be correctly described.

Copyright © 2012 John Wiley & Sons, Ltd. Int. J. Numer. Meth. Fluids (2012)
DOI: 10.1002/fld
 USING SYMBOLIC COMPUTATION SOFTWARE PACKAGES IN FV-BASED LES

technique illustrated in [8]. In this paper, we limit the presentation of the method to incompressible
flows. For vanishing viscosity, the hyperbolic character of the equations is better suited by upwind
schemes that reputed more capable than central discretization in reducing the aliasing error due
to the numerical dissipation. Unlike central discretizations, upwind schemes are characterized by
the presence of numerical diffusion, its relevance increasing via the wavenumbers tend to the filter
frequency. Consequently, upwind schemes were considered responsible of failing in LES and judged
inappropriate by the turbulence community, for example, see [9]. However, more recently, a new
impulse in the research of upwind schemes for LES has been provided by further studies; see [7,10].
In this study, we illustrate that a proper use of multidimensional upwind discretization allows us to
minimize the impact of the LTE.
The construction of a specific FV method for LES according to the present state-of-the-art of LES
is the main issue analysed in this article. The flux integration is here obtained by defining explicit
second-degree polynomials on three-dimensional (3D) 10-node Lagrangian simplex, then the ana-
lytical evaluation of the convective, diffusive and SGS flux integrals is performed. Owing to its
greater simplicity in being extended to real multidimensional cases, this method is preferred over an
implicit compact discretization; for example, see [10]. The use of symbolic computation software
packages (herein is Maple©) for performing operations such as multidimensional interpolation and
integral flux reconstruction is introduced. The Fortran code is particularly suitable to be imple-
mented in a straightforward way by means of the symbolic software package for linear algebra and
differential calculus present in Maple as it will be shown in details. To simplify the procedure, the
variables are co-located in the same grid nodes, instead of adopting a staggered grid. Therefore, an
approximate projection method for the velocity–pressure decoupling, for example, see [11–13], is
also used and is briefly addressed how to treat the discrete Poisson problem with a dedicated parallel
successive overrelaxation (SOR) procedure.

2. GOVERNING EQUATIONS FOR FINITE VOLUME-BASED LARGE EDDY SIMULATION

The motion of incompressible, isothermal Newtonian viscous flows in a domain V , with sufficiently
smooth boundary @V , is governed by the continuity and momentum equations that represent the NS
equations system. Herein, the governing equations are prefiltered according to a local average over
a finite volume .x/  V leading to the LES equations
Z
1
vQ  n dS D 0, (1)
j.x/j
@.x/

h i1 @Qv
A.m/
x D fconv C fdiff C fpress C fsgs , (2)
@t
1
R
representing v.x, t/  j.x/j
v.x0 , t/ dx0 the filtered volume average-based velocity field, the
.x/
local deconvolved volume-averaged velocity vQ being defined as
2 3
Z
.m/ 6 1 7
vQ .x/  A.m/
x v D Ax 4 v.x0 , t / dx0 5 (3)
j.x/j
.x/

and A.m/
x standing for the mth order truncated deconvolution differential operator [7, 8]. Let us note
that the adoption of truncated series expansion isa well-known
 procedure in regularizing ill-posed
problems. This way, because v.x/ D vQ .x/ C O mC1 ,  being the linear extension of the FV
(e.g.  D j.x/j1=3 /, the deconvolved velocity field vQ represents an mth order approximation to v,
whereas the volume-averaged
 velocity would be v.x/D v.x/ C O.2 /. In the following, m D 2 is
2 2 y 2 @2 ´2 @2
fixed, and A.2/ x @
x D I  24 @ 2  24 @ 2  24 @ 2 , A is denoted.

Copyright © 2012 John Wiley & Sons, Ltd. Int. J. Numer. Meth. Fluids (2012)
DOI: 10.1002/fld
 A. APROVITOLA AND F. M. DENARO

The fluxes in (2) are defined according to

Z
1
fconv D  vQ vQ  n dS , (4)
j.x/j
@.x/

Z
2 1
fdiff D .r s vQ /  n dS  D vQ , (5)
Rej.x/j Re
@.x/

Z
1
fpres D  pn dS , (6)
j.x/j
@.x/

Z  
1 2 s s
fsgs D .r v  r ev / C . vQ vQ  vv/  n dS , (7)
j.x/j Re
@.x/

D being an integro-differential operator, j.x/j the FV measure, n the local outward-oriented unit
vector normal to the boundary @.x/ and Re the Reynolds number. In case of presence of source
terms (e.g. reactive flows at low Mach number), it can be taken into account by simply adding a
further volume integral contribution in Equation (2) and the equation for the filtered internal energy;
see [15]. Equations (1) and (2) are coupled by means of the continuity equation that represents a con-
straint on the (deconvolved) filtered velocity field leading the pressure p to result only a Lagrangian
multiplier. Proper boundary conditions on @V must be prescribed to integrate the LES equations
system starting from a divergence-free initial velocity; for example, see [14, 16].
The definition T  .2=Re/ .r s v  r s vQ /C. vQ vQ  vv/ can be used for the exact unresolved tensor
in (7). This tensor can be approximated according to the eddy viscosity Bousinnesq assumption
(thus, being modelled only the deviatoric part, the complementary spherical tensor is added to the
filtered static pressure) [1-3, 11, 15]
Z
1
Q
fsgs Š fev D 2LE S .r s vQ /  n dS . (8)
j.x/j
@.x/

Anyway, the mathematical closure of the problem still requires a relation LE S D LE S . vQ /,
that is, the turbulence model. In particular, the Smagorinsky eddy viscosity model can be adopted.
 1=2
Hence, one has LE S D .C/2 jr s vQ j, being jr s vQ j D 2e S ij e
S ij , C the Smagorinsky constant
and  a suitable length related to the filter width, for example,  D jj1=3 . Note that the constant C
can be either prescribed a priori or computed dynamically according to the procedure of Germano
et al.; see [1–3]. Of course, many other SGS models, such as the scale similarity or mixed ones,
are applicable to the unknown tensor T. It is worthwhile addressing that, even setting fsgs D 0, the
use of the deconvolution procedure can be interpreted in terms of a generalized scale-similar SGS
model [1, 8, 17–19].
In case the presence of a source term, as in reactive flows, the energy production enters into the
SGS modelling.

3. THE NUMERICAL PROCEDURE

In this section, the numerical procedure for the time-dependent solution of the LES equations is
introduced, and an original multidimensional third-order accurate FV upwind discretization is pre-
sented. As the computational domain is concerned, a Cartesian volume V D Œ0, Lx Œ0, Ly Œ0, L´ 
is assumed to be partitioned by means of a structured grid. Along the x- (streamwise) and ´-
(spanwise) directions, where the flow is assumed to be homogeneous, the FV centres are uni-
formly distributed with mesh sizes x D Lx =Nx and ´ D L´ =N´ , Nx and N´ being the

Copyright © 2012 John Wiley & Sons, Ltd. Int. J. Numer. Meth. Fluids (2012)
DOI: 10.1002/fld
 USING SYMBOLIC COMPUTATION SOFTWARE PACKAGES IN FV-BASED LES

number of FVs in these directions. Along the y-direction, where the flow is assumed to be non-
homogeneous, a non-uniform grid is introduced by means of a 1D mapping y D Y./,  being
the independent variable in the transformed axis. The latter is uniformly partitioned in Ny C 1
nodes according to p D .p  1/h, .p D 1, : : : , Ny C 1/, h D Ly =Ny being the grid-spacing
and Ny the number of subdivisions in -direction. Thus, the coordinates of the FV faces in the
physical space are defined by yj D Y.j 1 / and yjC D Y.j /, .j D 2, : : : , Ny C 1/ and, sub-
sequently, the Ny FV nodes are suitably located at the cell barycentre. Fixing the boundary nodes
C
y1 D y2 D 0, yNy C2 D yN y C1
D Ly completes the domain partition. Moreover, the distance
 
between two adjacent FV centres is defined as yj  yj  yj 1 D yjC  yj1 =2, whereas the
FV width along the y-direction is hy .j / D yjC  yj D Y.j C1 /  Y.j /. Finally, one has the FV
   h i 
definition  xi , yj , ´k  ij k D xi , xiC  yj , yjC  ´ k
, ´C
k
, with jij k j D xhy .j /´.
Henceforth, the explicit dependence of hy from j is omitted, and the simplified index notation
xi˙ D i ˙ 1=2, yj˙ D j ˙ 1=2, ´˙ k
D k ˙ 1=2 is used, whenever possible, for defining the face coor-
dinates. The subscripts E, W , N , S , U , L will be also used when possible to indicate these face
locations, for example, xi  W , yj  S and ´k
 L. Further details can be found in [8, 15, 16].

3.1. Discrete time integration and pressure–velocity decoupling

The time integration of the momentum equation exploits the second-order accurate semi-implicit
Adams–Bashforth/Crank–Nicolson (AB/CN) scheme. In particular, for improving the numerical
stability of the method in case of confined viscous flows, the diffusive terms along the y-direction
are integrated according to the CN approximation, whereas the AB time-extrapolation is adopted for
all the others. The velocity–pressure decoupling is obtained by means of the pressure-free approx-
imate projection method (e.g. see [8–11, 13–17]). Specifically, one proceeds by first considering a
prediction step, where a non-solenoidal vector field v .x, t / is computed by solving the equation


 ² 
1 t  1 t n t 1
A  D2 v D A C D2 vQ C 3 .D1 C D3 /Qvn
2Re 2Re 2 Re
 ³
n n
1 n1 n1 n1 (9)
C fconv C fsgs  .D1 C D3 / vQ C fconv C fsgs
Re
v D vb on @V .
obtained after Equation (2) is integrated in time, according to the AB/CN method, while disregard-
ing the pressure term. Note that the integro-differential operator D was split along the Cartesian
directions as D D D1 C D2 C D3 . Accurate intermediate boundary conditions vb are required, and
they can be deduced provided that the correct relations between the real filtered velocity, pressure
gradient and the auxiliary fields are adopted. The assignment of congruent numerical intermediate
boundary conditions is not a trivial task, and full details are illustrated in [13, 16, 20].
To obtain a final divergence-free velocity vector field, the intermediate velocity must be corrected
in the so-called projection step. That is, according to the Helmholtz–Hodge decomposition theorem
[11, 13, 21], v is expressed as the sum of the unknown divergence-free filtered velocity vQ nC1 and a
pure gradient field, that is,
vQ nC1 D v  r. (10)
Now, by imposing the integral continuity constraint (1), the unknown potential field .x, t / must
satisfy the elliptic equation associated with proper Neumann boundary conditions
Z Z
1 1
r  n dS D v  n dS , (11)
j.x/j j.x/j
@.x/ @.x/

@  
D n  vb  vQ nC1
b
on @V , (12)
@n

Copyright © 2012 John Wiley & Sons, Ltd. Int. J. Numer. Meth. Fluids (2012)
DOI: 10.1002/fld
 A. APROVITOLA AND F. M. DENARO

allowing us to fulfil the compatibility condition that ensures the existence of a solution (apart from
a constant). The procedure can be now repeated in successive time steps.
The illustrated procedure is very general, being adoptable in both DNS and LES approaches.
Apart from the computational onerous caused by the large number of unknowns needed for simulat-
ing turbulence, the major complexity consists in developing and validating the code. The framework
of this paper is focused on the following main computational tasks:
1) The implementation of the routines for computing the fluxes fconv , fdiff , fsgs . This task requires
an efficient way for performing both a local functional reconstruction from the set of cell-
centred variables and an analytical surface integral over the faces of the FV. This step is simply
parallelizable because each flux computation requires independent sets of values.
2) The solution of three linear algebraic systems for computing the field v . Depending on the
adopted solver, it can be parallelized, however, for a certain
 space-discretization.
 The struc-
ture of the matrix is generally strongly diagonal-dominant A1 D I C : : : , and also, a simple
iterative sequential SOR solver converges in few iterations.
3) The solution of the linear algebraic system coming from the discretization of the elliptic prob-
lem (11), (12). This task is generally the most expensive part in a computational DNS/LES
code on the basis of projection methods representing much more than the 50% of total CPU
time. This is the routine mainly requiring care in parallelization.

3.2. Implementation of the multidimensional upwind fluxes integration: from symbolic software to
Fortran subroutines
This section illustrates the technique adopted for performing task 1. According to [7, 22], the idea
guiding the upwind criterion will be to make the interpolation stencil depending on the vector veloc-
ity direction, in the present case, adopting a real multidimensional upwind criterion. Assuming that
for high Reynolds number the system of equations is prevalently hyperbolic, the present choice is
to construct an upwinded Lagrangian simplex. Such a choice is motivated by the physical fact that
the mass and momentum quantity depend on to the part of the flow in the dependence region that
is individuated by the Euler eigenvalues and, therefore, by the trajectories of the flow. Although in
finite difference the discretization is performed on the differential form of the governing equations
and a numerical derivative is performed on each direction, separately, in finite volume approach,
one needs to evaluate the integral of the (multidimensional) flux flowing in a certain time interval
through the section of the control volume. For multidimensional flows, it appears suitable to use
a full 3D polynomial rather than a factorized interpolation along 1D directions. Furthermore, the
interpolation over Lagrangian simplex ensures the uniqueness of the polynomial over each of its
face, such property being not ensured by classical Lagrangian interpolations.
Thus, the local Lagrangian polynomial approximation of each filtered velocity component
uQ i , with .uQ 1 D u,
Q uQ 2 D v,
Q uQ 3 D w/,
Q being i, j, k the unit vectors along the Cartesian directions,
writes as
2
X
u
Qi
e
ui ., , / Š cpqr  p q  r  pT ., , /  cuQ i , (13)
p, q, r D 0I
pCqCr 62
u
Q
i
wherein the coefficients cpqr are defined over a Lagrangian simplex, that is, a tetrahedron
having 10 nodes suitably individuated onto the computational grid for ensuring the existence
of
 the polynomial (13). 2In this compact
notation, the vector of the monomial basis pT D
2 2 101
1, , , ,  , , ,  , ,  , p 2 R (superscript T stands for transpose) and the vector
of the polynomial coefficients related to a flux section, cuQ i 2 R101 , are introduced. Once indi-
viduated a proper set of 10 grid nodes, say ¹p , q , r º, the corresponding 10 collocated values e
ui
constitute the components of the known term vectors, say quQ i 2 R101 , of the linear algebraic sys-
tem A  cuQ i D quQ i , A 2 R1010 whose solution cuQ i D A1  quQ i , having adopted a Lagrangian
simplex for which det.A/ ¤ 0, exists and is unique.

Copyright © 2012 John Wiley & Sons, Ltd. Int. J. Numer. Meth. Fluids (2012)
DOI: 10.1002/fld
 USING SYMBOLIC COMPUTATION SOFTWARE PACKAGES IN FV-BASED LES

To clarify the multidimensional upwinding criterion that drives us to choose the proper local set of
grid nodes forming the upwinded Lagrangian simplex, let us consider as an example the flux section
at xi  W having outward unit normal vector n D i, which is staggered of x=2 from the FV
centre (ijk). Depending on the direction of the velocity vector evaluated on the section mid-point,
that is, vQ i 1=2,j ,k D iuQ i 1=2,j ,k C jvQ i 1=2,j ,k C kwQ i 1=2,j ,k , one can build the set of 10 grid nodes
constituting that tetrahedron, among the eight possible ones, fulfils the criterion dictated by the sign
of the three velocity components according to
¹p , q , r ºW W
 
0 D xi 1CiW , 1 D xi iW , 2 D xi C13iW I iW D H uQ i 1=2,j ,k
  (14)
0 D yj , 1 D yj jW , 2 D yj 2jW I jW D sign vQ i 1=2,j ,k
 
0 D ´k , 1 D ´kkW , 2 D ´k2kW I kW D sign wQ i 1=2,j ,k
being H.x/ D 1, 8x > 0, H.x/ D 0, 8x < 0 the Heaviside function and sign.x/ the sign of the
variable.§ Equation (14) allows us to uniquely individuate the matrix associated to the flux section
Q
W , say AW , and the vector of the known terms, say qvW , in a straightforward way. In fact, because
the velocity components are collocated, they are easily written in terms of the computational grid
indices,
 T 
Q
qvW D vQ i 1CiW ,j ,k , vQ i iW ,j ,k , vQ i C13iW ,j ,k , vQ i 1CiW ,j jW ,k ,
vQ i 1CiW ,j 2jW ,k , vQ i iW ,j jW ,k , vQ i 1CiW ,j ,kkW , vQ i 1CiW ,j ,k2kW , (15)

vQ i iW ,j ,kkW , vQ i 1CiW ,j jW ,kkW .
For example, if the three velocity components are all negatives, then one has iW D 0, jW D 1,
kW D 1, and the Lagrangian simplex results that one sketched in Figure 1. More in general,
by performing a proper permutation in (14), it is simple to extend the upwinding criterion also to
the flux sections S and® L having¯outward ¯ vectors n D j and n D k, respectively,
® unit normal
and determine the sets p , q , r S and p , q , r L . Similarly, one determines the corresponding
vectors of the known terms, that are qvSQ and qvL Q
, and the matrices AS and AL .
Owing to the FV method peculiarity of being flux-conservative, the construction of the
Lagrangian simplexes over the other three sections at xiC , yjC , ´C k
(having outward unit normal
vectors i, j and k, respectively) does not require further manipulations.
For example, the simplex to be individuated at xiC (or E/ must be the same already built for the
flux section at xiC1 , that is, staggered of x=2 from the grid node (i C 1, j , k), centre of the
adjacent FV i C1,j ,k . Thus, one can use (14) by shifting the node from i to i C 1. The same feature
applies for the sections at yjC (or N ) and ´C k
(or U ). Hence, one has the corresponding discrete
conservation property expressed by the FV constraints
® ¯
ix C D ixiC1 , jx C D jxiC1 , kx C D kxiC1 I p , q , r x C D ¹p , q , r ºxiC1
i i i
® ¯i
iy C D iyjC1 , jy C D jyjC1 , ky C D kyjC1 I p , q , r y C D ¹p , q , r ºyjC1 (16)
j j j j
® ¯ ® ¯
i´C D i´kC1
, j´C D j´kC1
, k´C D k´ kC1
I p , q , r ´C D p , q , r ´ .
k k k k kC1

Therefore, in the practical code implementation, only the fluxes at the faces W , S , L are com-
puted over the computational domain, whereas those at E, N , U are simply assigned according to
the conservative constraints (16).
Finally, to perform the integration of the fluxes over the FV surfaces, it is opportune to assume
a local reference system for each one of the three flux sections W , S , L according to the upwind
criterion. For example, the local system for the section W accords to .x/ D x  x0 , .y/ D y  yj ,
.´/ D ´  ´k , being x0  xi 1CiW . Therefore, the node .0 , 0 , 0 / of the tetrahedron is always

§
If u
Q i1=2,j ,k D 0, then iW would be undefined, but the convective flux is assumed to vanish.

Copyright © 2012 John Wiley & Sons, Ltd. Int. J. Numer. Meth. Fluids (2012)
DOI: 10.1002/fld
 A. APROVITOLA AND F. M. DENARO

Figure 1. Sketch (left) of the multidimensional upwinding criterion applied to the flux section W :
Lagrangian simplex in case of negative velocity components. Sketch (right) of the upwind-based local
reference system applied to the flux section W : the system (, , ) is centred in i if u is positive or in
i  1 if u is negative.

associated to the
® origin, which
¯ is upwind-depending along the x-direction, as shown in Figure 1
(right). The set p , q , r W is rescaled in local coordinates by substituting (14) with
0 D 0, 1 D xi iW  x0 D .1  2iW /x, 2 D xi C13iW  x0 D .2  4iW /xI
0 D 0, 1 D yj jW  yj , 2 D yj 2jW  yj I (17)
0 D 0, 1 D ´kkW  ´k D kW ´, 2 D ´k2kW  ´k D kW 2´ .
 
In the local system, the flux section results located at W D xi  x0 D xi  x  x0 D
1  2
2
 iW x, and the polynomial (13), evaluated at this location, rewrites as
u
Q  uQ i
ui .W , , / Š pT .W , , /  cWi D pT .W , , /  A1
e W  qW
2
X
u
Q  ˇ
 .mW /T  qWi D mpqr .W , , / uQ i .p , q , r /ˇW , (18)
p, q, r D 0I
pCqCr 62
where the vector m of the second-degree shape functions is built, on any flux section, in a finite
element manner as
mT ., , /  pT ., , /  A1 , m 2 R101 . (19)
Now, we are able to illustrate the surface integration methodology. Let us start by considering an
ith component of the convective flux evaluated at the coordinate xi  W . By exploiting (17)–(19),
one rewrites
C C
Z´k Zyj  T
1 1 u
Q
d´ .uQ uQ i /jx  dy Š quQ  MW  qWi (20)
jij k j i xhy ´ W
´
k yj

Copyright © 2012 John Wiley & Sons, Ltd. Int. J. Numer. Meth. Fluids (2012)
DOI: 10.1002/fld
 USING SYMBOLIC COMPUTATION SOFTWARE PACKAGES IN FV-BASED LES

having defined the matrix

´=2
Z hZy =2

MW  d mW .mW /T d , MW 2 R1010 . (21)

´=2 hy =2

Accordingly to such procedure, one can apply the technique for approximating all the other
integrals and compute the total advective flux of a component uQ i as
   T
i 1 T
u
Q u
Q
fconv Š quEQ  ME  qEi  quW
Q
 MW  qWi C
xhy ´
 T  T  T
uQ u
Q u
Q
C qvNQ
 MN  qNi  qvSQ  MS  qSi C qwQ
U  MU  qUi C (22)
 T 
u
Q
 qwL
Q
 ML  qLi

being ME , MN , MS , MU , ML the matrices computed, according to (21), over the other flux sections
of the FV. Of course, only the matrices MW , MS , ML are really computed because, according to the
FV conservation constraints (16), the equalities .ME /ij k D .MW /i C1,j ,k , .MN /ij k D .MS /i ,j C1,k ,
.MU /ij k D .ML /i ,j ,kC1 are exploited.
The adopted matrix–vector form is particularly suitable to be implemented by means of the
Maple© software in order for the Fortran subroutines to be produced. For example, Worksheet 1
illustrates the instructions for implementing the routine for the convective flux (20).
The SGS fluxes integration will follow the same guidelines. Concerning the spatial discretiza-
tion of e fev , let us assume to have already computed the cell-centred eddy viscosity values
LE S .xi , yj , ´k / by means of either a static or a dynamic procedure. Then, a local second-degree
Lagrangian polynomial is used for approximating LE S ., , / Š pT ., , /  cLES over each FV
flux section. Thus, one has
2
X   
LE S ., , / Š .m/T  qLES D mpqr LE S p , q , r , (23)
p, q, r D 0I
pCqCr 62

being qLES 2 R101 the vector of the cell-centred eddy viscosity values individuated according
to the upwind criterion. For example, concerning the flux section located at xi , one has an ith
component of the SGS flux
C C
Z´k Zyj
ˇ
1 @uQ i ˇˇ 1  LES T u
Q
d´ LE S ˇ dy Š q  SW  qWi , (24)
jij k j @x x  xhy ´ W
´
k yj i

where the matrix

´=2
Z hZy =2 ˇ
@mT ˇˇ
SW  d mW d , SW 2 R1010 (25)
@ ˇW
´=2 hy =2

is defined and qWLES is individuated on the grid as in (15).

Following such procedure, one can apply the technique for approximating all the integrals and
compute the total flux
1 h T  T  T
u
Q uQ u
Q
fQev
i
Š qELES  SE  qEi  qWLES  SW  qWi C qNLES  SN  qNi
xhy ´ (26)
 LES T  LES T  LES T i
uQi u
Qi u
Qi
 qS  SS  qS C q U  SU  qU C  q L  S L  qL ,

Copyright © 2012 John Wiley & Sons, Ltd. Int. J. Numer. Meth. Fluids (2012)
DOI: 10.1002/fld
 A. APROVITOLA AND F. M. DENARO

being SE , SN , SS , SU , SL the matrices computed over the other flux sections. Again, accord-
ing to (16), the equalities .SE /ij k D .SW /i C1,j ,k , .SN /ij k D .SS /i ,j C1,k , .SU /ij k D .SL /i ,j ,kC1
are prescribed.
The previous relations are implemented by means of the Maple© software, for example, for the
SGS flux (24), one proceeds exactly as in Worksheet 1 until the definition of the known terms, then
one adds the instructions according to Worksheet 2.
Let us observe that the molecular diffusive flux is discretized similarly to the SGS one. Indeed,
it is just necessary to set in (23) the constant value of the kinematic viscosity. That is congruent
to the fact that one expects that the flux crossing the section comes from a material volume that
moves from an upwinded region. This is motivated by the fact that the diffusion acts in an isotropic
way around a material volume evolving along the streamlines; thus, it appears congruent to use an
upwind stencil also for performing the diffusive term calculation.

3.3. Solution of the prediction equation. Treatment of the implicit formulation

As addressed in Section 3.1, the time integration is here implemented within the fractional time-step
method that requires solving Equation (9) for the intermediate velocity field v . This equation con-
tains an implicit part coming from the vertical molecular diffusive flux. At second accuracy order,
the local average value is congruently approximated by the cell value at the cell circum-centre.¶
Therefore, for a symmetric cell construction, the LHS of Equation (9) rewrites as

ˇ !ˇ
t ˇ x 2 2
@ h2y @2 ´ 2 2
@ ˇ
ˇ
1
A  D2 ˇˇ v D Ix C

2
C 2
C ˇ v C
2 ˇ
2Re ij k 24 @ 24 @ 24 @
ij k
´C xiC ˇ ˇ !
Z k Z
t @v ˇˇ @v ˇˇ
 d  d
2Rejij k j @ ˇy C @ ˇy 
´
k xi j j

´=2
Z x=2
Z " ˇ
ˇ t @mT ˇˇ (27)
Š A b1 ˇˇ 
v  d  qv


ij k 2Rejij k j @ ˇN N
´=2 x=2
ˇ #
@mT ˇˇ v
 q d
@ ˇS S
ˇ t h i
ˇ  
D b
A1 ˇ v  .dN /T  qvN  .dS /T  qvS .
ij k 2Rexhy ´
Because no privileged directions for the filtering exist to justify using an upwinded stencil, the
deconvolution operator is discretized over a central stencil. Furthermore, it has been experienced [8]
that a fourth-order accurate discretization of the second derivatives is necessary to guaranty a good
spectral accuracy of the numerical solution. Accordingly, one has the discretization
!
ˇ 19 C h2y aj 1   
b 1 ˇ 
A ˇ v D vij k C vi 1,j ,k C vi C1,j ,k C vi ,j ,k1 C vi ,j ,kC1 C
ij k 24 18

1    h2 2
X
y
 vi 2,j ,k C vi C2,j ,k C vi ,j ,k2 C vi ,j ,kC2 C aj Cp vi ,j Cp,k ,
288 24
pD2I p¤0
(28)
where the coefficients for the second derivative on the non-uniform direction y are reported in
Table I being the coefficients expressed by d1 D yj C1  yj ; d2 D yj C2  yj ; d1 D yj 1  yj ;
d2 D yj 2  yj .

¶
Accuracy order greater than second requires using specific techniques such as the deconvolution ones that can be coupled
to the implicit diffusive part in a single step [8].

Copyright © 2012 John Wiley & Sons, Ltd. Int. J. Numer. Meth. Fluids (2012)
DOI: 10.1002/fld
 USING SYMBOLIC COMPUTATION SOFTWARE PACKAGES IN FV-BASED LES

Table I. Performances of the parallelized successive overrelaxation

for the channel-flow simulation.
Number of Elapsed time
processors (ms) Speed-up rate
1 1800
2 1033 1.74
4 759 2.37
6 531 3.39
8 528 3.4

Moreover, according to the equalities (16), .dN /ij k D .dS /i ,j C1,k , and from derivation of the
shape functions (19), one has the vectors
  ˇ ˇ ˇ 
.dS /T 2 j  1  2 ˇ 2  j  2 ˇˇ 1  2  j ˇˇ
D ˇ
x´ 1 2 ˇ , 0, 0, .  / ˇ , .  / ˇ , 0, 0, 0, 0, 0, ,
S 1 1 2 S 2 1 2 S
  ˇ  ˇ ˇ 
.dN /T 2 j C1  1  2 ˇ 2 j C1  2 ˇ 1  2  j C1 ˇˇ
D ˇ ˇ
x´ 1 2 ˇ , 0, 0, .  / ˇ , .  / ˇ , 0, 0, 0, 0, 0 .
N 1 1 2 N 2 1 2 N
(29)
Remembering that the subscripts stand for the section locations, that is, S  yj and N  yjC1 ,
one has 1 jS D yj jS  y0 , 1 jS D yj C13jS  y0 , 1 jN D yj C1jN  yj CjN 2 jN D
hy .j / hy .j C1/
yj C23jN  yj CjN . Furthermore, it is  j D yj  2
 y0 ,  j C1 D yj C1  2
 yj CjN ,
v v
being y0 D yj 1CjS and jN  jyjC1 D .jS /i ,j C1,k . The vectors qN , qS can be suitably
written as
  T h i
qvS D vi ,j 1CjS ,k , 0, 0, vi ,j jS ,k , vi ,j C13jS ,k , 0, 0, 0, 0, 0
  T h i (30)
qvN D vi ,j CjN ,k , 0, 0, vi ,j C1jN ,k , vi ,j C23jN ,k , 0, 0, 0, 0, 0

because, owing to the zeros in (29), only the corresponding three non-vanishing components
 are

 
necessary. Let us observe that one should more correctly consider jS D jS D H vi ,j 1=2,k ,
 
jN D jN D H vi,j C1=2,k ; therefore, because the upwinded stencil depends on the unknown
solution v , the resulting algebraic system has matrix coefficients that depend on the solution.
However,
 without loss of accuracy,
 it is suitable to assume a time-linearization
 in the form jS D
H 2vin,j 1=2,k  vin1
,j 1=2,k
, jN D H 2vin,j C1=2,k  vin1
,j C1=2,k
that fixes the stencil.
It must be considered the handling of the boundary conditions is necessary; therefore, (27) needs
to be modified near a boundary along the vertical direction. This is accomplished by changing the
stencil of the derivatives along y from centred to a suitable asymmetric one, as long as conserving
the same accuracy. Owing to the strong diagonal dominance, the three algebraic systems are then
solved with a sequential SOR procedure.

3.4. Solution of the projection step. Discretization and parallelization strategies

This subsection is focused onto the discretization of the projection step (10)–(12) that allows us
to obtain the final divergence-free filtered velocity vQ nC1 . From a general point of view, the dis-
cretization of the elliptic operator can be obtained while adopting the same discretization technique
already illustrated in the previous sections. On the other hand, from a rigorous point of view, there is
no mathematical or physical considerations that can support the use of an upwinding-based stencil
for an elliptic problem. In the incompressible fluid model, acoustic waves spread at infinite veloc-
ity propagation in isotropic way; therefore, a centred stencil is more opportune. According to [8],
a fourth-order accurate discretization of the integral form (15) is possible, what had driven us to
produce a local stencil of 25 computational nodes in two dimensions. Consequently, one would get

Copyright © 2012 John Wiley & Sons, Ltd. Int. J. Numer. Meth. Fluids (2012)
DOI: 10.1002/fld
 A. APROVITOLA AND F. M. DENARO

a 125-node computational stencil in three dimensions. Actually, this leads to a heavy computational
effort and deserves a deeper consideration as well as future analyses that are out of the aims of the
present study.
Following such considerations, the discretization of the projection step is now simplified. Hence,
according to an approximate projection method [12, 15], a second-order FV discretization of
Equation (11) on a compact seven-node stencil is used. Thus, one has the discrete equation
2  3
nord sud
N C N  nC1 C inC1
4 1 C 1 C
j C1 j 1
 2inC1 5 C i C1,j ,k 1,j ,k
,j ,k
x 2 ´2 h2y x 2
 
inC1 C  nC1 2 inC1 N nord
,j C1,k j C1
C  nC1
N sud
i ,j 1,k j 1 ui C1,j ,k  ui 1,j ,k
,j ,kC1 i ,j ,k1 (31)
C C D C
´2 h2y 2t x
  3
Njnord 
C1 vi ,j C1,k C Nj
nord
 Njsud vi,j ,k  Njsud 
1 vi ,j 1,k wi,j ,kC1  wi,j ,k1
C C 5
hy t 2t ´

having defined the linear shape functions along the non-uniform direction as

yjC  yj hy yj C1  yj hy
C
Njnord
C1 D D , Njnord D D1 ,
yj C1  yj 2yj C1 yj C1  yj 2yj C1
(32)
yj  yj 1 hy yj  yj hy
Njsud D D1 , Njsud
1 D D ,
yj  yj 1 2yj yj  yj 1 2yj

being Njnord  Njsud D hy .yj C1  yj /=.2yj yj C1 /.

Regarding the implementation of the non-homogenous Neumann boundary conditions (12) at
nodes jbnd D 1 and jbnd D Ny C 2  m, let us assume the normal velocity component vQ i nC1 ,jbnd ,k
to be a known value. For example, if one considers the upper frontier located
 at jbnd D m,
ˇnC1 ˇnC1
then while exploiting Equation (11), that is, y ˇi ,j C1=2,k D y ˇi ,m,k D vi,m,k  vinC1
,m,k
=t,
Equation (31) is rewritten for each node having j D m  1 (and for nodes not involving periodic
boundary conditions) according to


2 2 1 inC1
C1,m1,k
C inC1
1,m1,k
 inC1
,m1,k 2
C 2
C C
x ´ hy ym1 x 2
inC1
,m1,kC1
C inC1
,m1,k1
inC1
,m2,k
ui C1,m1,k  ui 1,m1,k
C C D (33)
´2 hy ym1 2t x
wi,m1,kC1  wi,m1,k1 vinC1
,m,k
sud 
 Nm1 sud 
vi ,m1,k  Nm2 vi ,m2,k
C C ,
2t ´ hy t
h nord h
having considered that Nmnord D 2yym D 1 and Nm1 D 1  2yym D 0. Similarly, one pro-
ceeds in prescribing the boundary condition at node jbnd D 1 that enters into the equation written
for each node having j D 2. The resulting Poisson problem becomes equivalent to that one with
homogeneous Neumann boundary conditions and a modified source term [13, 15].
The implementation of the boundary conditions along the directions of periodicity is performed
by linking the values at i D 1 with those at i D Nx and the values at i D Nx C 2 with those at i D 2.
Analogously, one proceeds for the direction k.
This way, the discrete counterpart of the compatibility condition is verified, and a solution (apart
from a constant) of the pressure problem is ensured. However, it is well known that, although using
the approximate projection method, the continuity equation cannot be driven exactly to zero up to
machine accuracy, but it vanishes according to the magnitude of the LTE [12, 15]. The presence

Copyright © 2012 John Wiley & Sons, Ltd. Int. J. Numer. Meth. Fluids (2012)
DOI: 10.1002/fld
 USING SYMBOLIC COMPUTATION SOFTWARE PACKAGES IN FV-BASED LES

of such local errors must be taken into account, and eventually, the presence of spurious non-
divergent modes can be controlled by means of some specific methods [15]. Some comments about
the performances of the parallel solver used in the projection steps are discussed in Section 5.

4. RESULTS

The following results are focused more onto the theoretical accuracy analysis rather than to appli-
cations. In fact, such a complex formulation requires a careful validation phase for assessing the
validity of the computational code. On the other hand, LES formulations are much more sensi-
tive than DNS ones to the spectral distribution of the errors because the flow scales across the
filter frequency are still energetic. Therefore, in order for an SGS model to be ideally well suited
to the filtering and to efficiently reproduce the effects of the unresolved flow scales, the resolved
spectrum must not be excessively contaminated by numerical errors [1–3]. In contrast to central
discretizations, the upwind ones are characterized by the additional presence of a non-vanishing
imaginary part in the modified wavenumber (associated to the numerical diffusion) increasing via
the wavenumbers tend to the LES filter frequency. Consequently, they were considered responsi-
ble of failing in LES [9], but more recently, a new impulse in the research of upwind schemes for
LES has been provided by further studies [7, 10, 23]. Actually, provided that the artificial viscosity
does not affect the spectral content, upwind schemes appear appropriate owing to the respect of the
hyperbolic (for vanishing viscosity) character of the equations. Because of the numerical dissipa-
tion, they are also more capable of reducing the aliasing error than central discretization. Now, we
show that our formulation accomplishes such requirements.

4.1. Analytical determination of the local truncation errors in the wavenumbers space
According to the local volume average over an FV, the filtered velocity is obtained by means of a fil-
ter width of measure  (for the sake of simplicity now supposed uniform in all directions), which is
linked to the grid size by means of the subfilter resolution parameter Q. A distinctive feature exists
from what is generally performed by FD-based LES approaches. For example, the second-order FD
discretization is generally used in LES being it considered an implicit application of the top-hat
filtered velocity definition. For example, in one dimension, the central FD derivative exactly writes
xZi Ch
.ui C1  ui 1 / 1 du
as D dx, that is, the filtering is defined on a measure 2h that implies
2h 2h dx
xi h
Q D 2. Comparing the transfer functions for the FD-based top-hat and the FV-based one, in the
first case, the transfer function has the first zero at
= h that exactly coincides with the Nyquist fre-
quency, whereas in the second case, the first zero happens at 2
= h. Owing to this difference, in FV
methods, the Nyquist frequency acts implicitly as a cut-off of the components k >
= h, those more
smoothed by the filter. Of course, this aspect is deduced while analysing the operators in continuous
form. The framework becomes much more complex in the discrete form and for 3D schemes, as
detailed in [4].
The spectral accuracy analysis of the multidimensional upwind flux discretization is now per-
formed. First, it is necessary to develop an exact reference expression for the flux integrals, so that
the LTE can be determined. For obtaining this goal, let us assume the filtered velocity component e u
expressed in terms of the discrete Fourier series
N´ =21 Ny =21 =21
NxX
X X
e
u.x/ D b
u.w/e i wx
e
rDN´ =2 qDNy =2 pDNx =2 (34)
X
D u.p, q, r/e i.pxCqyCr´/  eT .x/  cb
b
e u
Q
,
p,q,r

being w D iwx C jwy C kw´ the wavenumbers vector, Nx , Ny , N´ even positive integers and
wx D p2
=Lx , wy D q2
=Ly , w´ D r2
=L´ . For the sake of simplicity, it was assumed that the

Copyright © 2012 John Wiley & Sons, Ltd. Int. J. Numer. Meth. Fluids (2012)
DOI: 10.1002/fld
 A. APROVITOLA AND F. M. DENARO

function is periodic over Lx D Ly D L´ D 2

, hence wx D p, wy D q, w´ D r. Thus, summations
extend until to the values pmax D
=x, qmax D
=y, rmax D
=´ corresponding to the Nyquist
frequencies along x, y and ´ directions, the mesh sizes being uniform. Further, we define also the
even integer N D Nx Ny N´ .
Similarly to the approach adopted in the previous sections, it was convenient to exploit in (34)
b
the vector notation wherein the vectors e.x/ 2 C N 1 , ceu
2 C N 1 are organized collecting the N
elements e i.pxCqyCr´/ and e b
u.p, q, r/, respectively, according to the column-oriented Fortran lan-
guage. Hence, one writes two vectors in which an element in the entry irow  D irow.p,
 q,
 r/
Nx Ny
is linked to the wavenumbers according to irow.p, q, r/ D 2 C 1 C p C Nx 2 C q C
 
Nx Ny N2´ C r , that is,

2   3 2 3
N
bu  N2x ,  2y ,  N2´
.Nx xCNy yCN´ ´/
e e i  2
6   7 6    7
6 N 7 6 .N yCN ´/ 7
6 b
eu  N2x C 1,  2y ,  N2´ 7 6
i Nx
2 1 xC y 2 ´
7
6 7 6 e 7
6 .. 7 6 .. 7
6 . 7 6 7
6 7 6 . 7
6 b
e u.1, 0, 0/ 7 6 7
6 7 e ix
cb
u
Q
D6 b 7 , e.x/ D 6
6
7
7 . (35)
6 eu.0, 0, 0/ 7 6 1 7
6 b 7 6 7
6 eu.1, 0, 0/ 7 6 e ix 7
6 .. 7 6 .. 7
6 7 6 7
6 . 7 6 . 7
6 7 4 5
4 b
e
u .pmax  1, qmax , rmax / 5 e i Œ.pmax 1/xCqmax yCrmax ´
b
e
u.pmax , qmax , rmax / e i.pmax xCqmax yCrmax ´/

Hence, for given values (p, q, r/, one fixes both the row position in (35) and the correspond-
ing values b
eu.p, q, r/ and e i.pxCqyCr´/ . Let us show the analysis for the component i D 1 of the
convective flux (20).
u2 at the
While exploiting the local reference system, the exact integral of the convective term e
flux section located at W D .1=2  iW /x writes as

C C
Z´k Zyj
T
1 ˇ
d´ u 2 ˇx 
e dy D cb
u
Q
 EW  cb
u
Q
, (36)
j@y´ j i
´
k yj

having defined the area measure j@y´ j D y´ and the matrix EW 2 C N N

´=2
Z y=2
Z
1
EW  d e .W , , / eT .W , , / d , (37)
y´
´=2 y=2

b y .q C m/G
its entries being EW .irow, icol/ D G b ´ .r C n/e i.pCl/W , with the transfer functions
b y .q C m/  sinŒ.q C m/y=2=Œ.q C m/y=2 and G
G b ´ .r C n/  sinŒ.r C n/´=2=Œ.r C n/
 
Nx
´=2. The entries in the columns are individuated by icol.l, m, n/ D 2
C1Cl C
   
Ny N´
Nx 2 C m CNx Ny 2 C n . The matrix (37) is symmetric and represents the exact convective
flux expressed in wavenumbers space.
Then, the numerical surface integral (20) can be similarly expressed in the wavenumbers space by
substituting the N harmonics e u.x/ D beu.p, q, r/e i .pxCqyCr´/ in the 10 components of the vector
u
Q
qW . This is performed by substituting in each one of the 10 entries (15) the Fourier series evaluated

Copyright © 2012 John Wiley & Sons, Ltd. Int. J. Numer. Meth. Fluids (2012)
DOI: 10.1002/fld
 USING SYMBOLIC COMPUTATION SOFTWARE PACKAGES IN FV-BASED LES

at the corresponding location and by reexpressing quW Q

according to
2 3
Œe.0, 0, 0/T  cb
u
Q
6 7
6
6 Œe..1  2iW /x, 0, 0/T  cb u
Q 7
7
6 7
6
6 Œe..2  4iW /x, 0, 0/T  cb u
Q 7
7
6 Œe.0, jW y, 0/T  cb u
Q 7
6 7
6 7
6 Œe.0, 2jW y, 0/T  cb u
Q 7
7  QW  cb
qe
u
W D6
6
7
u
Q
, (38)
6 Œe..1  2iW /x, jW y, 0/T  cb uQ
7
6 7
6
6 Œe.0, 0, kW ´/T  cb u
Q 7
7
6 7
6
6 Œe.0, 0, 2kW ´/T  cb u
Q 7
7
6 Œe..1  2iW /x, 0, kW ´/T  cb uQ 7
4 5
Œe..1  2iW /x, jW y, kW ´/T  cb u
Q

having defined the rectangular matrix QW 2 C 10N . This way, one can reformulate the numerical
integral (20) according to

T

1  e u
T
e
u 1 b
e
u b
e
u
q  MW  qW D QW  c  MW  QW  c
y´ W y´

T
T
1 b  b b b
D ce
u
 .QW /T  MW  .QW /  ce u
 ceu
EW  ce
e u
,
y´
(39)

having introduced the matrix eEW 2 C N N corresponding to the adopted numerical approximation.
Now, while exploiting (36) and (39), it is a simple task to express the LTE in wavenumbers space
according to the difference between the exact and the numerical flux
C C
Z´k Zyj
T
1 ˇ 1  e T b b
d´ u 2 ˇx 
e dy  q u
 MW  qW D ce
e
u u
LW  ce
b u
(40)
j@y´ j i y´ W
´
k yj

by means of the matrix b

LW 2 C N N ,
b
LW .!p , !q , !r , !l , !m , !n /  EW  e
EW , (41)
wherein it has been posed !p D px, !q D qy, !r D r´ and !l D lx, !m D my,
!n D n´.
The complete analysis of (41) is rather extensive owing to the number of independent variables
(of course, the six wavenumbers are not all independent) as well as because one should differentiate
between the effective contribution due to numerical discretization and that one deriving from the
aliasing errors. That is, one must correctly decompose (41) in two matrices b LW  b LdWi s C b
Lal
W , then
owing to the adopted matrix organization, one can identify the elements of such decomposition. For
example, one can start from the entry of the matrix b LW corresponding to the 0 wavenumber vector,
located at the position irow.0, 0, 0/ D icol.0, 0, 0/ on the main diagonal. Thus, one obtains that
b
LdWi s is composed solely by those elements of b LW at entries irow.p, q, r/, icol.l, m, n/ such that
N
they correspond to the conditions jp C lj 6 N2x , jq C mj 6 2y , jr C nj 6 N2´ . As an example of
the potentialities of this spectral analysis, the elements in the entries of the main diagonal are easily
individuated by the conditions p D l, q D m, r D n. Of course, this means that wavenumbers
j2pj > pmax , j2qj > qmax , j2rj > rmax are associated to components corresponding to the aliasing
error. Again, it is very straightforward to perform the analysis while using symbolic software pack-
ages; indeed, it is sufficient to implement the matrix QW and substitute the Fourier components. For

Copyright © 2012 John Wiley & Sons, Ltd. Int. J. Numer. Meth. Fluids (2012)
DOI: 10.1002/fld
 A. APROVITOLA AND F. M. DENARO

Third order multidimensional upwind Second order central

Third order multidimensional upwind Second order central

Figure 2. Real (upper) and imaginary (lower) parts of the local truncation error at (a) !p D 0 in the
0 6 .!q , !r / 6
plane for the third-order multidimensional upwind (supposing iW D jW D kW D 1)
and central convective flux discretizations; (b) !q D 0 in the 0 6 .!p , !r / 6
plane for the third-order
multidimensional upwind (supposing iW D jW D kW D 1) and central convective flux discretizations;
(c) !r D 0 in the 0 6 .!p , !q / 6
plane for the third-order multidimensional upwind (supposing
iW D jW D kW D 1) and central convective flux discretizations.

Copyright © 2012 John Wiley & Sons, Ltd. Int. J. Numer. Meth. Fluids (2012)
DOI: 10.1002/fld
 USING SYMBOLIC COMPUTATION SOFTWARE PACKAGES IN FV-BASED LES

Third order multidimensional upwind Second order central

Figure 2. Continued

example, while assuming to start from the last instruction of the worksheet Maple that computed
Int_flux_x_meno, this can be implemented as illustrated in the instructions in Worksheet 3.
Now the results are illustrated, in terms of the representation of the real and imaginary parts of
the LTE (thus, without addressing the dissipative and dispersive characters) focusing on the magni-
tude of the errors. It is supposed that the three velocity components are positive on the flux section,
thus iW D jW D kW D 1. Figure 2a, b and c illustrates the real and imaginary parts of the
LTE for !p D 0, !q D 0 and !r D 0, respectively. For the sake of completeness, the LTE of
the standard second-order centred discretization is reported. The figures clearly demonstrate that
the convective flux is very well represented by the multidimensional upwind discretization up to the
wavenumbers pmax , qmax , rmax , that is, in the first half of the reported intervals. The spectral analy-
sis makes clear that the multidimensional upwind is very accurate also in the most critical case of
strongly multidimensional effects, that is when !p D !q D !r . Comparing the LTE resulting from
the second-order central discretization, then one sees that such discretization is not able to capture
anisotropic effects. This is an indication that LES formulations with second-order FV central dis-
cretization can produce strong errors for flows not aligned with the grid. A similar spectral analysis
can be performed to evaluate the performances of the SGS fluxes.

4.2. Numerical accuracy analysis

Here, the space–time accuracy analysis of the presented FV method is numerically examined. For
this goal, an exact periodical solution of the time-dependent NS equations is exploited because this
kind of flow is often considered for validating unsteady solvers (e.g. [8, 21, 24]). The adopted flow
is governed by the following two-dimensional (2D) Taylor solution

u.x, y, t / D  cos x sin y e 2t ,

v.x, y, t / D sin x cos y e 2t , (42)
p.x, y, t / D 0.25.cos 2x C cos 2y/e 4t

depicting a decaying vortex interaction. Actually, owing to the 2D character, this solution is here
exploited twice, the first performing the computations in the (x, y) plane while repeating the same
profiles along the ´ planes and assessing that the velocity component w remains zero. Then, the

Copyright © 2012 John Wiley & Sons, Ltd. Int. J. Numer. Meth. Fluids (2012)
DOI: 10.1002/fld
 A. APROVITOLA AND F. M. DENARO

computations are repeated in the (´, y) plane, interchanging the role between x and ´ velocity com-
ponents. As the boundary conditions are concerned, the periodicity is imposed in both x and ´
directions while Dirichlet boundary conditions are always prescribed in vertical (y) one. Clearly,
this test cannot be considered fully exhaustive of the performances for 3D codes. On the other hand,
3D solutions are available, but similarly to (42), they describe an energy decaying flow that, in line
of principle, is not best suited for assessing the absence of numerical dissipation. We accept solu-
tion (42) to be a compromise and a valid starting point for our code validation. The velocity field is
initialised in the domain V D Œ0, 2
/  Œ0, 1, being this latter the most critical one because of the
lack of orthogonality in the Hodge decomposition [13].
Hence, a convergence study on the discretization error is performed. The simulations were per-
formed while taking constant the rate t= h D 103 and diminishing the mesh size, running the
code until T D 0.3. The results, obtained for the two velocity components, are reported in a double
logarithmic scale in Figure 3. As a comparison, the results obtained with the second-order accurate
central FV method [15] are also superimposed. The convergence shows a second-order accuracy
according to the LTE of the AB/CN integration and highlights smaller magnitude errors than those
obtained with the central discretization. Again, the same results were obtained by running the code
also for the case tested in the (´, y) plane.

4.3. Turbulent channel flow

The last test analyses the well-known turbulent flow in a plane periodic channel, which is a classical
test-case in confined turbulence [25]. Here, we consider first a channel of height 2ı, streamwise
length 4
ı and spanwise length 4
ı=3 at Re D u ı= D 180, being u the friction velocity. A
60  40  48 non-uniform (vertically) grid is used, and t D 2  104 . Consequently, the mesh reso-
lution in wall unit is x C D 37.3, ´C D 15.5 along the uniform directions, whereas the first grid
point in vertical direction is at y C D 0.28. Then, a second more challenging case with streamwise
length 2
ı and spanwise length
ı=2 at Re D u ı= D 590 is resolved over a 48  100  48 grid
for which the mesh resolution in wall unit is x C D 77, ´C D 39 along the uniform directions
whereas the first grid point in vertical direction is at y C D 0.15. For such a Reynolds number, the
LES database recently created by the LESinItaly group is also available [26].
At the present, our FV-based LES approach is used without supplying an explicit SGS model.
The action of the LTE along with the deconvolution procedure is in effect. To diminish the alias-
ing error deriving from the convective flux discretization in our code, the multidimensional upwind
interpolation is used only for the balanced variable uQ i , whereas a central interpolation is applied
for the normal velocity. Recent results illustrating the performance of an LES of such flow with a
third-order upwind discretization are also published in [23].

Figure 3. Convergence study of the discretization error at time T D 0.3. The representation for the two
velocity components is in a double logarithmic scale reporting the errors in the L1 norm. As comparison,
the convergence curves obtained with a second-order central FV code (uc and vc ) are also reported.

Copyright © 2012 John Wiley & Sons, Ltd. Int. J. Numer. Meth. Fluids (2012)
DOI: 10.1002/fld
 USING SYMBOLIC COMPUTATION SOFTWARE PACKAGES IN FV-BASED LES

a
10
DNS [23]
FV Upwind 3d
1 FV Central
x**(-5./3)
0.1

0.01

0.001

0.0001

1e-005

1e-006

1e-007

1e-008
0.1 1 10 100

1
DNS [23]
FV Upwind 3d
0.1 FV Central
x**(-5./3)

0.01

0.001

0.0001

1e-005

1e-006

1e-007

1e-008
1 10 100

b
Streamwise energy spectra of streamwise velocity component

0.1

0.01
DNS from Moser et al.
-5/3 slope
Present FV method
0.001
Fluent
Euu

OpenFOAM
TransAT
0.0001 Code_Saturne

1e-005

1e-006
0.1 1 10 100
Kx

Figure 4. (a) Channel flow at Re D 180: energy spectra of the streamwise velocity component along
the streamwise wavenumber (upper) and spanwise wavenumber (lower) for the central and upwind FV
discretization compared with the DNS database [25]; (b) Channel flow at Re D 590: energy spectra of
the streamwise velocity component along the streamwise wavenumber for the upwind FV discretization
compared with the DNS database [25] and the LESinItaly database [26].

Copyright © 2012 John Wiley & Sons, Ltd. Int. J. Numer. Meth. Fluids (2012)
DOI: 10.1002/fld
 A. APROVITOLA AND F. M. DENARO

In Figure 4a, the energy spectra of the streamwise velocity component along the streamwise
and spanwise wavenumbers for Re D 180 are reported. The results are compared with both the
DNS and a standard second-order centred scheme solutions. In Figure 4b, the energy spectra of
the streamwise velocity component along the streamwise wavenumbers for Re D 590 are reported,
the profile being compared with the DNS and some of the solutions in [26].
Two comments are worthwhile to note about the results. The first is that our upwind discretization,
unlike expected, is not more dissipative than central one, and the second is that the energy content
of the large flow scale appears better represented. However, a slight energy accumulation appears
along the spanwise direction. Clearly, the SGS model would act on this behaviour by dissipating
energy close to the filter cut-off. Furthermore, a better comparison between LES and DNS would be
obtained by filtering the DNS spectra with the corresponding top-hat.

5. PERSPECTIVES ON MESSAGE PASSING INTERFACE PARALLELISM AND

CONCLUDING REMARKS

The CPU time spent in solving the elliptic problem (11) and (12) represents, generally, the largest
part of the total computational time in an LES code; therefore, the algorithms for solving the
algebraic linear system must be suitably chosen. We started testing two sequential algorithms,
specifically the preconditioned GMRES solver, included in the SPARSKIT packet and an opti-
mized SOR procedure suitably developed by ourselves. We experienced that, although using a grid
with 106 unknowns, the SOR method performed much better than GMRES in terms of CPU time
(the memory occupation is not comparable). Of course, owing to the dependencies in the Seidel
process, the SOR implementation of (31) needs specific care in order for parallelization to be prac-
ticable. Thus, dependences were eliminated by choosing a ‘colouring technique’ on the basis of a
red–black colouring in the (i, j ) plane while alternating the cycles in the k plane. At present, the
parallelization was obtained by means of the OpenMP directives. This algorithm was successfully
tested on several Shared Memory Multiprocessing machines and performed sufficiently fine for our
aims on the IBM SP5 of the CINECA on four processors. As a realistic test-case on SP5, a 3D
channel flow simulation illustrated in the previous section is used. Specifically, a 60  40  48 non-
uniform grid is used, and the performances of the parallelized SOR solver is tested at the first time
step. Convergence is obtained in the Euclidean norm onto the residual (multiplied by the local mesh
size) with a tolerance of 1011 reached after 1050 iteration cycles. The following table illustrates
the behaviour of the solver in terms of the elapsed time expressed in milliseconds when the number
of processors increases (on a single eight-CPU node) (Table II).
Of course, the SOR method is not representative of the state-of-the-art; it was here presented
because the global LES code still represents a prototype. Owing to the restrictions in using the
OpenMP directives, which limit the code running mainly on symmetric multiprocessor architectures

Table II. Coefficients of the second derivative in the deconvolution operator discretized at the fourth-order
on non-uniform grid.
aj 2 2.d1 d1 C d1 d2 C d2 d1 /
 2  d3 d

d1 d1 d2  .d1 d1 C d1 d2 C d1 d2 /d2 C .d1 C d1 C d2 /d2 2 2

aj 1 2.d1 d2 C d1 d2 C d2 d2 /

 2  d3 d

d2 d1 d2  .d1 d2 C d1 d2 C d2 d2 /d1 C .d1 C d2 C d2 /d1 2 1

aj 2.d1 d1 C d1 d2 C d1 d2 C d1 d2 C d1 d2 C d2 d2 /

.d1 d2 d1 d2 /
aj C1 2.d1 d2 C d1 d2 C d2 d2 /

d2 d1 d2  .d1 d2 C d1 d2 C d2 d2 /d1 C .d1 C d2 C d2 /d12  d13 d1
aj C2 2.d1 d1 C d1 d2 C d2 d1 /

d1 d1 d2  .d1 d1 C d1 d2 C d1 d2 /d2 C .d1 C d1 C d2 /d22  d23 d2

Copyright © 2012 John Wiley & Sons, Ltd. Int. J. Numer. Meth. Fluids (2012)
DOI: 10.1002/fld
 USING SYMBOLIC COMPUTATION SOFTWARE PACKAGES IN FV-BASED LES

as well as the limited performances, we are also planning the parallelization strategies for using
Message Passing Interface directives in our code allowing us running on cluster architectures. The
initial strategy is simply based on a domain decomposition for allowing parallelism along the
streamwise direction x. Therefore, assuming even Nx FVs along x and a decomposition in even
Nproc slices, each subdomain will consist of Nxp  Ny  N´ , being Nxp D Nx =Nproc. Thus,
the iteration cycles remain unchanged in j and k indices while the i-cycle in each subdomain
spans between i_lef t D 1 C myrank  Nxp and i_right D i_lef t C Nxp  1. The variable
myrank D 0, 1, : : :Nxp  1 will identify the current process inside the communicator. As concerns
the fluxes computation, each subdomain needs communicating the nodal values close to the bound-
aries that are necessary for defining the complete Lagrangian simplex. Therefore, one must supply
a suitable extension of ghost cells.
In particular, two slices of ghost cells are necessary at left and right positions of each internal
boundary, whereas careful communication applies for prescribing the periodic streamwise condi-
tion, as illustrated in Figure 5. As concerns the algebraic systems, further work is required to assess
the better strategy between implementation of a dedicated SOR parallelism and adoption of more
sophisticated linear algebra packages. This has been recently performed in preliminary form, and
results are presented in [27].
This paper was mainly devoted to illustrate the strategy of development of an original third-
order accurate FV multidimensional upwind method on non-uniform Cartesian grids for simulat-
ing turbulence with the LES approach. A second-order time-accurate projection method for the
pressure–velocity decoupling of the NS equations system was exploited. Particularly, it was high-
lighted how to exploit symbolic software packages for working with linear algebra and differential
calculus algorithms that are necessary in developing an LES code. Details about the construction of
the routines for the convective fluxes as well as SGS ones are provided. Some comments about the
parallelization strategy are also addressed. An accuracy analysis both in the physical and spectral
spaces was reported. However, the high accuracy of the method was accomplished by exploiting
high-order flux integration that allows us to compute the top-hat filtered velocity field. This volume-
averaged filtered field can represent, at best, only up to second-order accuracy the DNS counterpart;
furthermore, the smoothing of high resolved frequencies is a possible source of modelling diffi-
culties. There are several reasons for requiring a modification of such filtered variable, basing on
both modelling and accuracy requirements. The general strategy for performing a modification of
the top-hat filter required using an approximate deconvolution procedure of the volume-averaged
velocity. This way, the deconvolved filtered velocity tends to approach the spectral-like resolution.
This way, unlike other classical high-order FV formulations (e.g. ENO and WENO), the resolved
field effectively stands for a high-order approximation to the pointwise solution. As FV simula-
tions are much more feasible than the spectral ones for simulating complex flows, the present
method, leading to a good resolution in spectral space, appears very efficient. As the deconvolution

Figure 5. One-dimensional (i-direction) domain decomposition for Message Passing Interface parallelism
with layers of ghost cells: blue for periodicity, green for i_right boundaries and red for i_lef t boundaries.

Copyright © 2012 John Wiley & Sons, Ltd. Int. J. Numer. Meth. Fluids (2012)
DOI: 10.1002/fld
 A. APROVITOLA AND F. M. DENARO

procedure allows us to have a good spectral resolution in the resolved wave numbers range, it
becomes possible to improve the performances of classical SGS models because they essentially
rely on the smallest resolved scales. On the other hand, it was recognized that the deconvolution
acts as a sort of implicit SGS modelling for the filtered NS equations. In this respect, the actual
numerical procedure can be cast in the context of approximate deconvolution modelling procedures
for LES.

REFERENCES
1. Sagaut P. Large Eddy Simulation for Incompressible Flows. An Introduction, (2nd edn). Springer: Berlin, 2002.
2. Berselli L, Iliescu T, Layton W. Mathematics of Large Eddy Simulation of Turbulent Flows. Springer: Berlin, 2005.
3. Lesieur M, Métais O, Comte P. Large-Eddy Simulations of Turbulence. Cambridge Press: Cambridge (UK), 2005.
4. Denaro FM. What does finite volume-based implicit filtering really resolve in large-eddy simulations? Journal
Computational Physics 2011; 230(10):3849–3883.
5. LeVeque RJ. Finite Volume Methods for Hyperbolic Problems. Cambridge Press: Cambridge (UK), 2002.
6. Zandonade PS, Langford JA, Moser RD. Finite-volume optimal large-eddy simulation of isotropic turbulence.
Physics of Fluids 2004; 16:7.
7. Aprovitola A, Denaro FM. On the application of congruent upwind discretizations for large eddy simulations. Journal
Computational Physics 2004; 194(1):329–343.
8. Iannelli P, Denaro FM, De Stefano G. A deconvolution-based fourth order finite volume method for incompressible
flows On non-uniform grids. International Journal for Numerical Methods in Fluids 2003; 43(4):431–462.
9. Mittal R, Moin P. Suitability of upwind-biased finite difference schemes for large eddy simulation of turbulent flows.
AIAA Journal 1997; 35(9):1415–1417.
10. Park N, Yoo JY, Choi H. Discretization errors in large eddy simulation: on the suitability of centered and
upwind-biased compact difference schemes. Journal Computational Physics 2004; 198:580–616.
11. Chorin AJ. Numerical solution of the Navier–Stokes equations. Mathematics of Computation 1968; 22:745–762.
12. Almgren AS, Bell JB, Szymczak WG. A numerical method for the incompressible Navier–Stokes equations based
on an approximate projection. SIAM Journal of Scientific Computing 1996; 17:358–369.
13. Denaro FM. On the application of the Helmholtz–Hodge decomposition in projection methods for the numerical
solution of the incompressible Navier–Stokes equations with general boundary conditions. International Journal for
Numerical Methods in Fluids 2003; 43:43–69.
14. Gresho PM. Incompressible fluid dynamics: some fundamental formulation issues. Annual Review of Fluid
Mechanics 1991; 23:413–453 (Volume publication date January 1991). DOI: 10.1146/annurev.fl.23.010191.002213.
15. Denaro FM. A 3D second-order accurate projection-based finite volume code on non-staggered, non-uniform struc-
tured grids with continuity preserving properties: application to buoyancy-driven flows. International Journal for
Numerical Methods in Fluids 2006; 52(4):393–432.
16. Denaro FM. Time-accurate intermediate boundary conditions for large eddy simulations based on projection
methods. International Journal for Numerical Methods in Fluids 2005; 48:869–908.
17. Geurts BJ. Inverse modeling for large-eddy simulation. Physics of Fluids 1997; 13: 3585–3587.
18. Stolz S, Adams NA. An approximate deconvolution procedure for large-eddy simulation. Physics of Fluids 1999;
11(8):1699–1701.
19. Adams NA, Stolz S. A subgrid-scale deconvolution approach for shock capturing. Journal Computational Physics
2002; 178: 391–426.
20. Iannelli P, Denaro FM. Analysis of the local truncation error in the pressure-free projection method for the Navier–
Stokes equations: a new accurate expression of the boundary conditions. International Journal for Numerical
Methods in Fluids 2003; 42:399–437.
21. Brown DL, Cortez R, Minion ML. Accurate projection methods for the incompressible Navier–Stokes equations. J.
Comp. Phys 2001; 168:464–499.
22. De Stefano G, Denaro FM, Riccardi G. Analysis of 3-D backward facing step incompressible flows via a local
average-based numerical procedure. International Journal for Numerical Methods in Fluids 1998; 25:1073–1091.
23. Felten FN, Lund TS. Kinetic energy conservation issues associated with the collocated mesh scheme for incompress-
ible flow. Journal Computational Physics 2006; 215:465–484.
24. Pereira JMC, Kobayashi MH, Pereira JCF. A fourth order accurate finite volume compact method for the
incompressible Navier–Stokes solutions. Journal Computational Physics 2001; 166:217–243.
25. Moser R, Kim J, Mansour NN. Direct numerical simulation of turbulent channel flow up to Re D 590. Physics of
Fluids 1999; 11(5): 943–945.
26. Denaro FM, Abbà A, Germano M, Icardi M, Marchisio D, Rolfo S, Lampitella P, Colombo E, Inzoli F, Aprovitola
A, Marra FS, Ioveno M, Tordella D. A comparative test for assessing the performances of large-eddy simulation
codes. AIMETA Conference, Bologna, 2011.
27. Aprovitola A, Denaro PFM, Di Serafino D, Filippone S. Scalable algebraic multilevel preconditioners with appli-
cation to CFD. In Lecture Notes in Computational Science and Engineering, Part 1, Vol. 74. Springer: Berlin,
2010; 15–27, DOI: 10.1007/978-3-642-14438-7. ISBN/ISSN: 1439-7358 HYPERLINK http://dx.doi.org/10.1007/
978-3-642-14438-7t_blank.

Copyright © 2012 John Wiley & Sons, Ltd. Int. J. Numer. Meth. Fluids (2012)
DOI: 10.1002/fld

View publication stats