EPOCH Notes
EPOCH Notes
After the two days you should be able to setup and run EPOCH on a
problem of real importance to your research.
You should also be in a position to use and understand the manual.
You should learn about PIC codes in general.
You should understand more about the pitfalls of trying to do LPI
studies with PIC.
Advice on how to run EPOCH and setup software on your home
computers.
Give advise to the EPOCH team on new features for the code.
Resources:
All machines, and exercises, are linux based.
EPOCH is a Fortan90 program which uses MPI for parallelization.
You will always need both F90 and MPI to compile and run the code
even on one processor.
MPI on a Windows computer is not easy. Use linux or a Mac.
Workstation Usage
You can use the workstations for simple 1D tests and looking at the code.
You will now have a directory called ‘epoch’. Inside this directory will be
three EPOCH sub-directories epoch1d, epoch2d and epoch3d and some
MatLab and VisiT files. Change directory into the epoch1d directory and
start working through the ‘Getting Started with EPOCH’ guide.
Note: If you don’t have svn on your home computer you can always
download a tar file of epoch when you return to your lab. This you get
from the ‘Releases’ tab on the EPOCH CCPForge webpage. However I
recommend you get, and learn, svn and join the 21st century.
Getting Started with EPOCH
Compiling the code
The first thing you must do is to compile the code. This is done using the
UNIX “make” command. This command reads a file called “Makefile” and
uses the instructions in this file to generate all the steps required for
compiling the code. Most of this is done automatically and the only part
which typically needs changing are the instructions for which compiler to
use and what compiler flags it accepts. The Makefiles supplied as part of
the EPOCH source code contain sections for most commonly used
compilers so it is usually unnecessary to actually edit these files. Usually
you can compile just by passing the name of the compiler on the
command line.
The most commonly used compiler on clusters these days is the Intel
FORTRAN compiler. This is the default one used in the Makefile so for
such machines you need only type “make” without editing the file.
You should rarely need to edit the Makefile more than this. Occasionally,
you may need to change fundamental behavior of the code by changing
the list of flags in the “DEFINES” entry. This is documented in the User
manual.
This is the simplest possible input deck. The file is divided into blocks
which are surrounded by “begin:blocktype” and “end:blocktype” lines.
There are currently ten different blocktypes. The most basic input deck
requires only two.
The first block is the “control” block. This is used for specifying the
domain size and resolution and the length of time to run the simulation.
There are also some global simulation parameters that can be specified in
this block which will be introduced later.
Within the block, each parameter is specified as a “name = value” pair.
The parameters are as follows. “nx” specifies the number of grid points in
the x-direction (since this is a 1D code, the grid is only defined in the x-
direction). “x_min” and “x_max” give the minimum and maximum grid
locations measured in meters. Since most plasma simulations are
measured in microns, there is a “micron” multiplication factor for
convenience. There are also multiplication factors for “milli” through to
“atto”. Finally, the simulation time is specified using “t_end” measured in
seconds.
There are also commented lines in the deck. Any text following the “#”
character is ignored. The character may appear anywhere on a line, so in
the following example:
t_end = 50 #* femto
The value of “t_end” will be set to 50 seconds, since “#* femto” is
ignored.
The other required block is the “boundaries” block. This contains one
entry for each boundary, specifying what boundary condition to apply. For
the 1D code there are two boundaries: “bc_x_min” and “bc_x_max”. The
deck currently has both of these set to use open boundary conditions.
This will run epoch1d in parallel using 4 processors. It will use the
directory named “Data” for all its output and will read the file
“Data/input.deck” to obtain the simulation setup.
The change to the “boundaries” block instructs the code to add a laser
source to the left-hand boundary.
To load the data and assign the result to a structure named “data”, just
issue the following command:
data = getstruct(7,/varname)
Here, “/varname” is any of the variables listed by the previous command.
This will just read the “varname” variable into the data structure.
However, it is usually easiest just to omit the “/varname” flag. If it is
omitted then the entire contents of the file is read.
For the current example the result of this command is the following:
GDL> help,data,/struct
** Structure <2a0ddd8>, 6 tags, length=5096, data length=5080,
refs=1:
FILENAME STRING 'Data/0007.sdf'
TIMESTEP LONG 185
TIME DOUBLE 2.3386179e-14
EY STRUCT -> <Anonymous> Array[1]
GRID STRUCT -> <Anonymous> Array[1]
X DOUBLE Array[200]
The first few entries are fairly self-explanatory. The sixth item is a 1D
array containing the cell-centred grid positions. The fourth item is a
structure containing a 1D array of Ey at these positions. This structure
can be queried in the same way as “data” :
GDL> help,data.ey,/struct
** Structure <2a051a8>, 2 tags, length=1696, data length=1696,
refs=2:
METADATA STRUCT -> <Anonymous> Array[1]
DATA DOUBLE Array[200]
The raw data is contained in the “data” entry. The fifth entry, “GRID” is a
structure which contains :
GDL> help,data.grid,/struct
** Structure <2a0d7a8>, 5 tags, length=1768, data length=1756,
refs=2:
METADATA STRUCT -> <Anonymous> Array[1]
X DOUBLE Array[201]
LABELS STRING Array[1]
UNITS STRING Array[1]
NPTS LONG Array[1]
This is the node-centred grid along with its metadata. The cell-centred
array shown previously is derived from this.
The above plot can be generated by issuing the following command:
plot,data.x,data.ey.data
There are more examples on using idl/gdl in the manual.
More details on using VisIt are in the Manual. We recommend that you
learn VisIt – it’s free and powerful.
To load the data from an SDF file, type the following at the MatLab
prompt:
data=GetDataSDF(‘Data/0007.sdf’);
The “data” variable will now contain a data structure similar to that
obtained with the IDL reader. You can explore the contents of the
structure using MatLab’s built-in variable editor. To plot Ey, you can
browse to “data.Electric_Field.Ey”. The structure member
“data.Electric_Field.Ey.data” contains the 1D array with Ey values. Right-
clicking on it gives a range of options, including “plot”.
Alternatively, from the command prompt you can type
>> x=data.Electric_Field.Ey.grid.x;
>> xc=(x(1:end-1) + x(2:end))/2;
>> plot(xc,data.Electric_Field.Ey.data);
The first two lines set up a cell-centred grid using the node-centred grid
data. In the future, this work will be automatically done by the reader.
A 2D Laser
Next, we will take a look at the 2-dimensional version of the code.
Change to the epoch2d directory: “cd ~/Epoch/epoch2d”
Type “make –j4” to compile the code.
Copy the next example input deck into the Data directory:
“cp ~/EXAMPLES/02-2d_laser.deck Data/input.deck”
Run with “echo Data | mpirun –np 4 ./bin/epoch2d”
This deck is very similar to the 1D version that we have just looked at. It
contains the necessary modifications for adding a new dimension and
some additions to the laser block for driving a laser at an angle.
The “control” block now contains “ny” which specifies the number of grid
points in the y-direction. Notice that we are using the value “nx” to set
“ny”. As soon as “nx” has been assigned it becomes available as a
constant for use as part of a value.
We must also provide the minimum and maximum grid positions in the y-
direction using “y_min”, “y_max”. Like “nx”, the values “x_min” and
“x_max” are available for use once they have been assigned.
The “laser” block is similar to that given in the 1D version except that
there is now a “profile” parameter. In a similar manner to “t_profile” this
is a function ranging between 0 and 1 which is multiplied by the wave
amplitude to give a modified laser profile. The only difference is that this
is a function of space rather than time. When applied to a laser attached
to “x_min” or “x_max” it is a function of Y, defined at all points along the
boundary. When the laser is attached to “y_min” or “y_max”, it is a
function of X.
Finally, the output block has been modified so that it outputs all
electromagnetic field components.
The control block has one new parameter. “npart” gives the total number
of PIC particles to use in the simulation.
The input deck contains a new block type, “species”, which is used for
populating the domain with particles. Every species block must contain a
“name” parameter. This is used to identify the particle species in other
sections of the input deck and is also used for naming variables in the
output dumps. The next parameter is “charge” which gives the charge on
each particle in terms of elementary charge units. “mass” is specified in
units of electron mass. “frac” is the fraction of the total number of PIC
particles (npart) to assign to this species. Both of the blocks in this deck
use “frac = 0.5”, so there will be 1600 particles of each species. The next
parameter, “temp”, sets the average temperature of the particle species
in Kelvin. Alternatively, you can use “temp_ev” to specify the temperature
in electronvolts. Particles are assigned an initial momentum
corresponding to a Maxwell-Boltzmann distribution for this temperature.
It is defined across the entire problem domain, so in 1D it is a function of
X, in 2D a function of X and Y, and in 3D a function of X, Y and Z.
“density” sets the number density across the problem domain. The code
is set to use per-particle weights in the default Makefile. With this option,
the pseudoparticles are distributed evenly across the domain. Then the
weight of each pseudoparticle is adjusted so that it matches the number
density specified in the “density” parameter. The alternative option is to
disable per-particle weighting. In this case, the weight of each
pseudoparticle is the same and the particles are placed on the grid so that
they match the number density at the start of the simulation.
Finally, we have a “drift_x” parameter. This is also defined across the
entire problem domain and is used to give the particles an average
momentum drift in the x-direction. There are similar “drift_y” and “drift_z”
parameters.
This deck has been designed to simulate a two-stream instability, so it
has to groups of particles which are identical in every respect except that
one set is drifting in the opposite direction to the other.
The contents of the output block can be much more complicated than the
examples shown so far. Here, we will cover the options in a little more
depth.
There are a few output variables which are grid-based quantities derived
by summing over properties for all the particles contained within each cell
on the mesh. These are “ekbar”, “mass_density”, “charge_density”,
“number_density” and “temperature”. To find more details about these
variables, consult the output block section of the User manual.
Other Laser-Plasma Example Decks
Now that you have a basic understanding of how the input decks work,
you should be able to work through the remaining example decks by
referring to the User manual for a description of any new parameters
used. Several experienced users of the code will be available throughout
the duration of the workshop, so if you want help with anything please
don’t hesitate to ask.
The decks are:
In the VisIt control window, click the “Open” button which launches a file
browser window. The first entry is called “Host” and contains a drop-down
list of all configure remote machines.
When viewing data across a slow network connection, there is one more
useful thing to know. VisIt has two methods of drawing plots generated
on a remote machine. The first method is to construct the polygons used
in drawing the plot on the remote machine and send them across the
network. The local machine then turns these into a plot image. This
makes manipulating the figure very fast (zooming, rotating, etc), since all
the polygons that generate the image are on the local machine. However,
if there are a lot of polygons then they can be slow to transfer across the
network. They can also use up a lot of memory. For these cases, the
alternative is to render the image on the remote machine and just
transfer the image across the network. The downside of this approach is
that whenever you manipulate the plot, it must be re-drawn on the
remote machine and then transferred across the network again.
The options controlling this behaviour are to be found under
“Options->Rendering” in the “Advanced” tab. The feature is called
“scalable rendering”.
Collisions in EPOCH
EPOCH now contains a collision routine based on the technique outlined in
Sentoku & Kemp [J Comput Phys, 229, 4591 (2010)]
Collisions are enabled using the output block named "collisions" which
accepts the following three parameters.
For example:
begin:collisions
use_collisions = T
coulomb_log = auto
collide = all
collide = spec1 spec2 off
collide = spec2 spec3 0.1
end:collisions
With this block, collisions are turned on and the Coulomb logarithm is
automatically calculated. All values of the frequency array are set to one
except (spec1,spec2) is set to minus one (and also (spec2,spec1)) and
(spec2,spec3) is set to 0.1
Ionisation in EPOCH
EPOCH includes field ionization which can be activated by defining
“field_ionisation = T” in the control block along with ionisation energies
and an electron for the ionising species in one of the species blocks. This
is done via the species block in the "ionisation_energies" and
"electron_species" parameter respectively. "ionisation_energies" should be
given as a list in joules, and "electron_species" should be the name of the
species to be used as the electron species. For example, ionising carbon
species might appear in the input deck as:
begin:species
charge=0.0
mass=1837.2
name=carbon
ionisation_energies=(11.26*ev,24.38*ev,47.89*ev,64.49*ev,392.1*ev,
490.0*ev)
electron_species=electron
rho=den_gas
end:species
begin:species
charge=-1.0
mass=1.0
name=electron
rho=0.0
end:species
begin:species
charge=0.0
mass=1837.2
name=carbon
ionisation_energies=(11.26*ev,24.38*ev,47.89*ev,64.49*ev,392.1*ev,
490.0*ev)
electron_species=(electron,electron,electron,fourth,electron,electron)
rho=den_gas
end:species
begin:species
charge=-1.0
mass=1.0
name=electron
rho=0.0
end:species
begin:species
charge=-1.0
mass=1.0
name=fourth
rho=0.0
end:species
Once the code is built with QED support, actually turning on QED for a
specific simulation requires the addition of a new block into the input deck.
This block is simply called “qed” and starts with the usual “begin:qed” and
“end:qed” markers of the other blocks. The parameters which can go into
the block are
QED also requires that the code now know which species are electrons,
positrons and photons. Rather than try to do this automatically the user
has to specify the type of a species. This is done by using a single
“identify” tag in a species block. To specify an electron the block in the
deck would look like
begin:species
name=electron
frac=0.5
rho=7.7e29
identify:electron
end:species
Once the identity of a species is set then the code automatically assigns
mass and charge states for the species. At present, the user cannot
override these. Possible identities are
electron : A normal electron species. All species of electrons in
the simulation must be identified in this way or they will not
generate photons.
positron : A normal positron species. All species of positron in
the simulation must be identified in this way or they will not
generate photons.
photon : A normal photon species. One species of this type is
needed for photon production to work. If multiple species are
present then generated photons will appear in the first species of
this type.
bw_electron : The electron species for pair production. If a
species of this type exists then electrons from the pair
production module will be created in this species. If no species of
this type is specified then pair electrons will be generated in the
first electron species.
bw_positron : The positron species for pair production. If a
species of this type exists then positrons from the pair
production module will be created in this species. If no species of
this type is specified then pair positrons will be generated in the
first positron species.
Updating EPOCH
To update to the latest version of EPOCH simple cd into your Epoch
directory and enter ‘svn update’. This will work fine provided you haven’t
edited any of the Fortran source code. If you have edited the source code
then you need to learn svn.