UDF Mechanics

A short course in procedures for

programming and implementing FLUENT
UDFs in serial and parallel

William Wangard, Ph.D.

1
 Overview
• Introduction to User-Defined Functions
• Compiling and using your UDF
• UDF Programming Basics
• Extending your UDF to FLUENT Parallel
• Scheme Programming
• Limitations

2
 Introduction to UDFs
• Programmed with ANSI C (usually)
• Are compiled or interpreted
• Allow you to customize
– Boundary conditions
– Solution controls
– Source terms
– Data analysis
– Customized I/O
– Post processing
Moving mesh via UDF
– And much more…
3
 Sample UDF
UDF source code for the previous animation…
Header file Æ #include "udf.h"
(in all UDFs)
/* UDF to rotate inner square boundary */

DEFINE_ UDF function Æ DEFINE_CG_MOTION(rotate, dt, vel, omega, time, dtime)

{
(DEFINE_s have
/* reset velocities */
prescribed
functionality) NV_S (vel, =, 0.0); /* Macro to set vel = 0 */
NV_S (omega, =, 0.0);

/* Set inner square rotating at 1.0 radians/sec */

omega[2] = 1.0; /* 2 is the z-component */
}

4
UDFs: Compiled vs. Interpreted
• Many UDFs require compilation
• Compiled UDFs are recommended
– Faster, especially with UDFs that are called for each
cell in the domain
• Limitations to Compiled UDFs
– Version-specific
• Interpreted UDFs have limited capability and run
slower.
– Acceptable in some applications

5
Compiling UDFs in FLUENT 6.1
• Define→User-Defined→Functions→Compiled…
• Click Add
• Pick source (.c) and header (.h) files
– Make sure that there are no ^Ms in the file (Win/Unix translation)
• Click Build to automatically compile the UDF…
– Don’t forget to check for compilation errors!
• Click Load to link the UDF to the FLUENT
process
6
 UDF Directory Structure (1)
• The UDF is built in a special directory
– Default directory is ./libudf
– Sometimes between compiles, you need to work with
more than 1 library between bug fixes.
• The UDF will be compiled for the version of
FLUENT currently running: e.g. 2ddp, etc.
– Other versions will be updated if they exist in the
directory structure when you click Build.
– Thus, to build a parallel version, start up Fluent
parallel, and repeat the build process.
7
 What’s your UDF Programming Level?
• Beginner
– Basic understanding of ANSI C
• Knows what a pointer is and what to do with it
– FLUENT UDF Macros
• Expert
– Broader knowledge of ANSI C
• Knows what a function pointer is and how to use it
• Knows that malloc(…) must be accompanied by a free(…)
– Scheme programming for FLUENT GUI
• Guru
– Has memorized Kernighan & Ritchie
– In-depth knowledge of particular models --- funded development
8
 Hooking the UDF (1)
• UDF functions are idle until they are “hooked”
• Hook is made through appropriate GUI panel.
• In pick list, choose “name” of DEFINE_ function whose
type is appropriate for the connection:
– For Boundary Conditions: DEFINE_PROFILE
– For Initialization: DEFINE_INIT
– For Execute on Demand: DEFINE_ON_DEMAND
– For Execute at END: DEFINE_EXECUTE_AT_END
– Execute At End is NEW in FLUENT 6.1!
• Performed when convergence is attained.

9
 Hooking the UDF (2)
• DEFINE_INIT(my_init, d)
Define→User-Defined→Functions Hooks…
– Pick Initialization Function
– In pick list, choose “my_init” from the list

10
Grid Terminology & Data Types
• cell_t
– Control volume
• face_t
– Control volume face
• Thread
– Group of cell_ts or face_ts
• Domain
– Collection of all threads
• Node
– Corners of control volumes
11
 Threads (1)
• Thread is a structure representing collection
of face_ts or cell_ts.
• We work with pointers to the thread.
Thread* t
• Identify type of thread using predicates:
– Defined in threads.h
– CELL_THREAD_P(t) = TRUE if t is a cell
thread
12
 Threads (2)
– FLUID_THREAD_P(t) == TRUE if t is a fluid
continuum zone.
– BOUNDARY_FACE_THREAD_P(t) == TRUE if t is
a face thread and if it is a external boundary.
• Thread Identifiers
– ID of thread matches value in the FLUENT GUI
– THREAD_ID(t) is the ID of thread
– THREAD_TYPE(t) is the type of thread
• == THREAD_F_WALL for a wall
• == THREAD_F_VINLET for a velocity inlet

13
 The Domain Pointer
• Domain pointer allows access to entire problem
data structure
• Passed to some functions, otherwise use
Domain* d = Get_Domain(1);
• Multiphase CFD analyses use arrays of Domain
pointers… each phase has a domain.

14
 UDF Macros
• Purpose of Macros is to allow users’ UDFs to
remain unchanged between FLUENT revisions
• Three classes of Macros
1. DEFINE_XXXX Macros
• Expand to C function definitions that return specific types of
data depending upon nature of function
2. Data Storage Macros
• Access data within FLUENT data structures
3. Utility Macros
• Perform routing tasks such as vector operations, looping, etc.

15
 DEFINE_ Macros
• Expand to functions
• General purpose, e.g.
– DEFINE_INIT(my_init, domain)
• Called just after data is initialized
• void my_init(Domain* domain)

• Model Specific/Solution Control, e.g.

– DEFINE_DT(my_dt, domain)
• Control the size of the time step during the solution
• real my_dt(Domain* domain)

16
 Threads in DEFINE_ functions
• Type depends on nature of DEFINE_
function:
– In DEFINE_PROFILE, Thread* is a face
thread since boundary condition applied to
faces
– In DEFINE_SOURCE, Thread* is a cell thread
since source terms apply to cells
– Thread is passed through the “hooking” process
17
 Data Storage Macros (1)
• Cell Data Macros
– Returns quantities stored at cell centers.
– C_R(c, t) returns the density at cell_t c in (cell)
thread t.
– Other macros for T, U, V, W, enthalpy, etc.
• Face Data Macros
– Returns quantities stored at cell faces
– F_T(f, t) returns the temperature at face_t f of
(face) thread t.
18
 Data Storage Macros (2)
• Data for variable is available only if equation is
being solved at least one iteration.
– Attempting to access C_UDSI(c,t,i)will give a
FATAL error if user-defined scalars are not being
solved
• Check using storage allocation macro:
– (NNULLP(T_STORAGE_R_NV(t, SV_UDS_I(i))))
– If TRUE (pointer is NOT NULL), UDS(i) is defined!

• Face quantities are not stored on all faces…

– Ex: Density is not stored at wall boundary faces.
19
 Utility Macros (1)
• Vector operations
– About 100 available macros
– Defined in flow.h
– NV_S(a, op, b)
• for(j=0; j<ND_ND; j++) { a[j] op b; }
• op is {= += *= /= -=}
– NV_VS(a, op, b, op2, c)
• for(j=0; j<ND_ND; j++) { a[j] op b[j] op2 c; }
• op2 is {+ - / *}

20
 Utility Macros (2)
Purpose Looping Macro
Loop over cell threads → thread_loop_c(t,d){…}

Loop over face threads → thread_loop_f(t,d){…}

Loop over cells → begin_c_loop(c,t) {…}

end_c_loop(c,t)

Loop over faces → begin_f_loop(f,t) {…}

end_f_loop(f,t)

21
 C0 and C1
• All faces are connected to one or two cells
– Boundaries have only c0
– Internals have c0 and c1
• Access to c0, c1 and respective threads in a face loop:
– c0 = F_C0(f,tf) where tf is a face thread
– tc0 = F_C0_THREAD(f,tf) … will never be NULL
– tc1 = F_C1_THREAD(f,tf) … may be NULL
– c1 = F_C1(f,tf) … if above is not null!

22
 Shadow Threads
• Occur at interface between two dissimilar cell
zones.
– Used often in conjugate heat transfer simulations
– Fluid zone next to solid zone
– Used to simulate heat transfer through thin barrier
“between” cell zones
– The “shadow thread” is accessed by the following:
ts = THREAD_SHADOW(t)

23
 Derivative Terms
• Sometimes derivative terms are needed
– C_T_G(c,t) returns gradient of temperature
(a vector) at a cell center
– C_UDSI_G(c,t) returns gradient of UDS
• Particular equation must be activated
• TUI command solve/set/expert
– Keep temporary memory from being freed

24
 Computing Gradient of UDS
• Typically used in a define ADJUST function
• UDS is assigned value
• Following code snippet is called to compute the derivatives
of the UDSs… before iteration is actually performed
/* CODE TO TAKE DERIVATIVE of all UDS */
int ns;
for (ns = 0; ns < N_UDS(); ++ns) {
MD_Alloc_Storage_Vars(d, SV_UDSI_RG(ns),SV_UDSI_G(ns),SV_NULL);
Scalar_Reconstruction(d, SV_UDS_I(ns), -1, SV_UDSI_RG(ns), NULL);
Scalar_Derivatives(d, SV_UDS_I(ns), -1, SV_UDSI_G(ns),
SV_UDSI_RG(ns), NULL);
}

• C_UDSI_G data is now populated. Can copy components

to UDM if desired.
25
 Parallelization of UDFs
• One UDF that works in serial and parallel
• When is parallelization necessary?
– Transfer of data between processes
• Vectors of data used in your UDF
– Global operations
• Computing quantities over groups of cells or threads
– volume or surface integrals
• Determining global minima or maxima
– File I/O
– Messages
26
 Serial Fluent
• 2 Processes
– Cortex (FLUENT GUI)
– Compute node
• Get messages from cortex
• Do computations
• Messaging

27
 Parallel FLUENT Architecture
• Cortex (FLUENT GUI)
• Host
– Communicate between cortex
and compute node 0
– No data
• Compute nodes
– Node 0
• Get commands from host
• Do computations
– Other nodes
• Get commands from node 0
• Do computations

28
Parallel Partitioning of Cells
• Interior Cells
– Completely inside partition
boundary
• Exterior Cells
– Abut the partition boundary
and penetrate inside
adjacent partition one layer
deep

29
Parallel Partitioning of Faces
• Interior faces
– Connects cell to cell
• Boundary faces
– Connected to one cell (c0)
• Partition boundary face
– Shared by two partitions
• External face
– Rarely used

30
 Parallelization
• In order to parallelize your UDF you’ll need
to be familiar with
– Compiler directives
– Predicates
– Data transfer macros
– Parallel looping macros

31
 Compiler Directives (1)
• Compile variables defined in config.h
• Directive variables for FLUENT Parallel
– RP_HOST compile on host
– RP_NODE compile on node
– PARALLEL compile in parallel
• Negated forms also used
– !RP_HOST compile on node and serial
– !RP_NODE compile on host and serial
– !PARALLEL compile in serial

32
 Compiler Directives (2)
• Example….
#if RP_HOST
Message(“I am the host node.\n”);
#endif

• Example….
#if !RP_NODE
Message(“I am either the host node or the serial
compute node!\n”);
#endif

33
 Predicates in Parallel
• Defined in para.h
• Evaluate to TRUE based on the value of compute
node.
– (I_AM_NODE_ZERO_P)
• Same as… (myid == node_zero)
– Typically used for data transfer between individual
compute nodes.

34
 Data Transfer (1)
• Macros defined in para.h
• Host to Compute nodes
host_to_node_type_num(arg1,arg2,…)
– Does nothing in serial
– Communicates num variables between host and all
compute nodes (through node zero)
• type = {int, real, boolean}
• num = {1, 2, …, 7}

35
 Data Transfer (2)
• Compute Node 0 to host
node_to_host_type_num(arg1,arg2,…)
• Passing Vectors
host_to_node_type(v,len)
– Communicate type array of length len
– If type is string, don’t forget to include
terminating null character!

36
 Basic Steps to Parallelize a UDF
1. Define variables that exist on all nodes 5. Perform Global Reductions
real x[10]; int id; #if RP_NODE
FILE* fp; /* Accumulate x over all nodes */
PRF_GRSUM(x,10,work);
2. Get variables from GUI [optional]. #endif
#if !RP_NODE
x[i] = 0.0
6. Send data back to host [optional]
id = RP_Get_Integer(“id_zone”);
node_to_host_real(x,10)
fp = fopen(”output_file”, ”r”);
#endif
7. Print output
3. Pass variables to compute nodes /* To GUI */
host_to_node_real(x,10); Message0(“x0 = %f\n”, x[0]);
host_to_node_int_1(id);
/* To FILE */
4. Do computations on compute nodes (SERIAL Code) #if !RP_NODE
#if !RP_HOST fprintf(fp, “x[0] = %f\n”, x[0]);
thread_loop_c(t,d) {…} fclose(fp);
#endif
#endif

37
 Sample Parallelized UDF (1)
/* UDF to get volume average temp and density */ /* Pass id of thread to nodes */
host_to_node_int_1(id);
#include "udf.h"
#define N 2
/* Main computation section… */
#if !RP_HOST
DEFINE_ON_DEMAND(average_t_and_rho)
{ d = Get_Domain(1);
Domain* d; /* domain pointer */ t = Lookup_Thread(d, id);
Thread* t; /* to be a cell thread */
cell_t c; /* cell */ /* Initialize data on nodes */
voltot = 0.;
#if RP_NODE for(i=0; i<N; i++) x[i] = 0.0;
real work[N]; /* needed in global reduction */
#endif /* loop over all internal cells… avoid overlap */
begin_c_loop_int(c,t)
real x[N]; /* x[0] = temp, x[1] = dens */
{
real vol; /* cell volume */
vol = C_VOLUME(c,t);
real voltot; /* total volume */
int i, id; /* index, id of cell thread */ x[0] += C_T(c,t) * vol;
x[1] += C_R(c,t) * vol;
#if !RP_NODE voltot += vol;
id = 2; }
/* Alternatively... This only works on host node. end_c_loop_int(c,t); /* Don’t forget this! */
id = RP_Get_Integer("my_zone");*/ #endif
#endif
/*… continued on next page … */

38
 Sample Parallelized UDF (2)
/* Global Reductions… accumulate summations from
each compute node… totals will be copied to
each compute node */
• Original Serial code in red
#if RP_NODE
/* Accumulate the vector */ • Comments
PRF_GRSUM(x,N,work);
– Original is basically same
/* Accumulate the scalar */
voltot = PRF_GRSUM1(voltot); as in serial…except
#endif
• begin_c_loop(c,t) is now
/* Send data to host */ begin_c_loop_int(c,t)
node_to_host_real(x,N); /* The vector */
node_to_host_real_1(voltot); /* The scalar */ – Data transfer to compute
/* Message to GUI */ nodes and global reductions
#if !RP_NODE
Message("Mean T = %10.3e\n", x[0]/voltot); are typically the main parts
Message("Mean Rho = %10.3e\n", x[1]/voltot);
#endif that need to be added
/* also – Message passing usually
Message0(…); without the compiler directive*/
} /* Done */ done after global reduction.
39
 Parallel looping (1)
• Cells on a partition contain internal and external
cells
– Internal: Cells that are exclusive to that partition
– External: Those that overlap into another partition
• In a summation process, we loop over the internal
cells (and exclude the overlap)
begin_c_loop_int(c,t)
{…}
end_c_loop_int(c,t)

40
 Parallel Looping (2)
• For face loops, a face may lay on the boundary of
two partitions. In a summation loop, we don’t
want to count it twice! Use the following:
begin_f_loop(f,t)
{
if (PRINCIPAL_FACE_P(f,t))
{ … }
}
end_f_loop(f,t)

41
 Scheme Variables
• Used to modify FLUENT GUI
• Defined in var.h
• Access from FLUENT (serial or host node)
var = RP_Get_Type(variable);
• Type: {Real, Integer, Boolean, String (for char*)
Ref_Int_List (for lists), etc.}
• Must define variable in GUI…

42
 Defining a Scheme Variable
• Cortex uses object-oriented Scheme (not documented)
• Check if variable has been defined
(define (make-new-rpvar name default type)
(if (not (rp-var-object name))
(rp-var-define name default type #f)))

• Define some “rp vars” (rp stands for rampant)

(make-new-rpvar 'var_real '1.2 'real)
(make-new-rpvar 'var_int '3 'integer)
(make-new-rpvar 'var_int_list '(3 4) 'integer-list)
(make-new-rpvar 'var_string ‘”Hello” 'string)

43
 Scheme Variables
• Copy previous lines to the file
“my_vars.scm”
• Load the file all at once to FLUENT using
(load ”my_vars.scm”)

which is a scheme command.

44
 Accessing Scheme Variables
real x;
int i, n, len, *ilist;
char* s;

/* Get the variables in serial or HOST node only */

#if !RP_NODE
x = RP_Get_Real(”var_real”);
i = RP_Get_Integer(”var_int”);
len = RP_Get_List_Length(”var_int_list"); /* Get length of list */
ilist = malloc( len * sizeof (int) ); /* allocate int list */
for(n=0; n<len; n++){
ilist[n] = RP_Get_List_Ref_Int(”var_int_list”,n); /* from var.h */
}
s = RP_Get_String(”var_string”);
#endif

/* DO STUFF HERE */

#if !RP_NODE
free(ilist); /* Don’t forget to free the memory allocated */
#endif

45
 Modifying a Rpvar in a UDF
• Change rpvar in UDF with
RP_Set_Real(”scheme_var”, x);

• Value in GUI is not automatically updated!

• To update the value in cortex
(%rp-var-value ’scheme_var)

46
 What Have We Learned Today?
• How to compile UDFs in • Parallelizing your UDF
FLUENT 6.1 – Compiler directives
• How to hook UDFs – Data transfer
– Messaging
• Basics of UDF
programming: • Scheme variables
– DEFINE_ functions – Defining rpvars in Cortex
(GUI)
– Data accessing macros
– Accessing rpvar values
– Looping macros
from UDF
– Vector operations
– Modifying rpvars in UDF
– Derivative operations and passing back to Cortex
47