USING STANDARD TOOLS TO PACKAGE AND DISTRIBUTE SCIENTIFIC SOFTWARE C AND FORTRAN

LIBRARIES: A DEMONSTRATION WITH THE GENERAL PURPOSE TIMING LIBRARY (GPTL)

​ ​James Rosinsk​i 3,
Edward Hartnett 1,2,
1 CIRES, University of Colorado, Boulder, CO 80309, USA
2 NOAA/ESRL/GSD, Boulder, CO 80305, USA
3 ​NCAR/JCSDA

ABSTRACT 1.2 Autotools

Important scientific C and Fortran software packages Standard Linux tools autoconf, automake, and (for
are frequently distributed through use of home-grown libraries) libtool are used to configure and build
approaches rather than the standard GNU tools packages in a standard way. Collectively these tools are
autoconf, automake, and libtool. This results in wasted referred to as “autotools”.
effort for the distributors of the software, as they
struggle to achieve portability. Home grown approaches Anyone who has built open-source software on Linux
also tend to waste the time of all software installers, machines has encountered these tools. To build, the
which have to struggle to correctly build the software. In user first (optionally) specifies some standard compiler
this poster the conversion of a scientific software flags, and then runs the ​configure​ script. The ​configure
package, the General Purpose Timing Library (GPTL) script queries the target machine for information needed
from a home-rolled build system to a standard build is to correctly build the software. It then constructs
demonstrated. This results in nearly an order of Makefiles on the build system. The user runs ​make​ to
magnitude decrease in build system complexity, build the code, ​make check​ to build and run tests, and
measured by lines of code, and an increase in build make instal​l to install the code.
system features for the user, including automatic shared
library builds, standard configure options, and standard To accomplish this, the developer creates an autotools
make options, as well as increased portability and ease build system, which includes one configure.ac file for the
of use. As demonstrated by major scientific software project, which is the basis of the configure script, and
packages like netCDF and HDF5, these tools can be one Makefile.am file for each directory in the build. The
used very effectively to package software portably, Makefile.am files are used to construct the final
including software that is built and run on High makefiles.
Performance Computing (HPC) systems.
1.3 GPTL
1. INTRODUCTION
The General Purpose Timing
1.1 Motivation Library(​https://jmrosinski.github.io/GPTL​) provides
detailed timing information for an application by
Complex software packages can require considerable instituting calls to​ start ​and​ stop​ timers. These calls can
effort to build and install. Scientific software, such as be manually inserted by the user, or automatically
numerical models, make frequent use of math, I/O, generated through the use of flags available on most
compression, and other libraries, which must be compilers (e.g. -finstrument-functions on GNU and Intel,
installed, and their location communicated to the -Minstrument=functions on PGI) . The library is
software build. Dependencies between software thread-safe, and provides per-thread timing information.
packages can also become complex; features may be It also can provide optional summary information across
used which require a specific version of a library, or MPI tasks. There are other optional capabilities, such as
specific information about the library’s capabilities. integration with the PAPI performance counter library,
and automatic generation of MPI stats if C-preprocessor
The GPTL legacy build system must determine the flag ENABLE_PMPI is defined.
availability (or not) of several other libraries. It also has
some code that only allies to x86 microprocessors. The 2. LEGACY BUILD SYSTEM
build system was functional, but required manual
intervention from the user. 2.1 Makefiles

In order to reduce the maintenance burden, increase The GPTL libraries were built with ​make​. A selection of
portability, and support shared library builds, a new different Makefiles were included for different systems
autotools-based build system has been developed. and compilers. To build the code, the user first copies

Corresponding author address:​ Edward J. Hartnett,

[email protected]
the makefile template that most matches the build used as inputs to the autotool utilities ​autoconf ​and
system, edits it, then types ​make. automake.​ These programs create the​ configure​ script
and Makefile.in files.
2.2 Combined C and Fortran Libraries
When installing the package, the​ configure​ script is run,
Like many science codes, the original GPTL library which uses the ​Makefile.in​ files, and information
contains both C and Fortran libraries. Although these gathered from the target machine, to create ​Makefile
seem intimately connected to the science developer, files. These files are then used to build the package
they are best distributed and maintained as separate when the user types ​make.
libraries, repositories, and distributions.
To assist the developer in managing the build system,
2.3 Setting Compilers and Flags for the User the ​autoreconf​ command will perform a complete build
of the build system, running all necessary scripts and
GPTL, like many home-rolled build systems, attempts to tools in the correct order.
set compilers and flags in the makefiles. That is, with
certain compilers, the build attempts to use certain flags, 3.2 Configuration
and those flags are pre-programmed into the build
system. The configuare.ac file is where all configuration of the
build system is managed. This file is in turn converted
Increased portability is attempted by providing a variety by ​autoconf ​to the ​configure​ script that is run by the end
of makefile templates, each with different compilers and user.
compiler flags. Although this works for a small number
of cases, it is not a scalable solution. The configure.ac script is in m4, a macro language that
is part of the POSIX standard. It is a simple language,
It is not scalable because it is impractical for all compiler similar to the C preprocessor, but with more features.
and compiler options to be supported in the makefiles. A The configure.ac script also includes Bourne shell script
new compiler or a new flag requires editing of the commands.
makefiles, and ultimately a new release to support the
new compiler or flag. The​ configure ​script must learn about the target build
system, and must also provide options to the user. The
Setting flags for the user is also not a long-lasting --help option is automatically provided to help end-users
solution. Flags that are optimal with the state of the decide what options are available.
system today, may be counterproductive in a few years
when the version of the compiler or the capabilities of 3.2.1 Handling Configure Options
hardware have changed. Over time, the flags that are
built into the build system may become less relevant Many configuration options are provided by ​autoconf,
and less correct. with no developer effort required. For example,
--enable-shared vs. --disable-shared, or the --prefix
The autotools approach allows the user full control over option, which allows the user to specify the installation
compilers and flags. Only the user can select the directory.
optimal compiler and flags for their target system. Thus
autotools-based builds can easily take advantage of Other options are specific to the GPTL libraries, like
new features or special performance options on the --enable-pmpi, which enables automatic profiling of
target system, even if these features are unknown to the application calls to MPI functions. This option is
GPTL developers. When a new compiler is introduced supported by the following code in ​configure.ac​:
or new flags added to existing compilers, the autotools # Does the user want to turn on PMPI?
build system does not have to change to support the AC_MSG_CHECKING([whether PMPI is to be
new compiler or new features. enabled])
AC_ARG_ENABLE([pmpi],
The legacy build system also included a configure.ac, [AS_HELP_STRING([--enable-pmpi],
which built a single program: the suggestions script. [build with PMPI capability])])
This script helped with the manual creation of Makefiles test "x$enable_pmpi" = xyes || enable_pmpi=no
by the GPTL developer. AM_CONDITIONAL(ENABLE_PMPI, [test
x$enable_pmpi = xyes])
3 C LIBRARY if test $enable_pmpi = yes; then
AC_DEFINE([ENABLE_PMPI], [1], [enable
pmpi])
3.1 Building the Build System fi
AC_MSG_RESULT($enable_pmpi)
The build system consists of the configure.ac file and all
the Makefile.am files, but these files cannot be directly
run to build the GPTL libraries. Instead, these files are
 print_rusage.c pr_summary.c util.c
This code sets automake conditional ENABLE_PMPI,
which is used in the Makefile.am files to add a test if 3.3.1 Target ​make all
PMPI is enabled (from test/Makefile.am):
The ​all​ target builds the library, but no tests are built.
if ENABLE_PMPI
check_PROGRAMS += pmpi
TESTS += run_par_pmpi_test.sh
3.3.2 Target ​make check
endif
The ​check​ target first runs the ​all​ target, then builds and
The configure.ac code also sets a C preprocessor runs the tests. Tests are simply programs that use the
macro if ENABLE_PMPI is defined . This macro is library, and return 0 for success or any other value for
output to a special header file, config.h, when the user failure.
runs the configure script. The config.h file contains
many macros that describe the target build system. If all tests build and pass, then ​make check​ returns 0,
There are many macros that are provided by autoconf otherwise it fails.
for all projects. Using the AC_DEFINE in configure.ac,
we can add macros in support of the GPTL code, such 3.3.3 Target ​make install
as ENABLE_PMPI.
The ​install​ target first runs the ​all​ target, then installs the
The config.h file must be included as the first header library in /usr/local, or elsewhere, if the user used the
included in all library code and test code. This allows --prefix configure option. An ​uninstall​ target is also
developers to control how code is built on systems with provided automatically.
different capabilities.
Note that the ​install​ target does not run the tests.
3.2.2 Finding Libraries
3.3.4 Targets ​make dist/distcheck
Autoconf is good at finding libraries or header files.
Here’s how we can check for a library (from The ​dist​ target builds a tarball with all files needed to
configure.ac): build the GPTL C library. Users of the tarball do not
have to have any autotools software installed, nor do
# Check for pthread library. they run autoreconf on their system (the autotools files,
AC_CHECK_LIB([pthread], [pthread_mutex_init]) configure.ac and the Makefile.am files, are provided in
if test the tarball for completeness, but the tarball also
"x$ac_cv_lib_pthread_pthread_mutex_init" =
contains the created configure script, and the
xyes; then
AC_DEFINE([PTHREADS], [1],
Makefile.in files that are converted to Makefiles when
[pthreads library is present]) the configure script is run.)
fi
The ​distcheck​ target first builds a distribution tarball, and
3.3 Makefiles then unpacks it and does a ​make check​, and a ​make
distclean​, then ensures that everything is deleted. The
Makefiles are generated when the user runs the distcheck​ target, as its name implies, allows you to
configure script. The configure script runs many tests on check that your distribution tarball is complete and
the system, and can set automake conditionals to correct.
control how the makefiles are constructed. The
automake conditionals are used in the Makefile.am files. 3.3.5 Target ​make clean/distclean

Each directory in the project gets a Makefile.am file. The ​clean​ target works as expected. The distclean
This file specifies what is built in that directory. target does an extra level of cleaning, returning the build
to a state before the configure script was run.
In most cases, Makefile.am files are simple. Here is the
Makefile.am that builds the GPTL C library (including 4 FORTRAN LIBRARY
shared library):
4.1 Configuration
libgptl_la_CPPFLAGS = -I$(top_srcdir)/include
As with the configuration of the C library, necessary
# This is our output. The GPTL library. libraries and headers are found when the user runs the
lib_LTLIBRARIES = libgptl.la
configure script. One such library is the GPTL C library,
# These are the source files.
which must be built and installed before the
libgptl_la_SOURCES = f_wrappers.c \ GPTL-Fortran library is built. This is done in
getoverhead.c gptl.c gptl_papi.c \ configure.ac:
hashstats.c memstats.c memusage.c pmpi.c \ # Find the GPTL C library.
AC_CHECK_LIB([gptl], [GPTLinitialize], [], environment, while still maintaining separate C and
[AC_MSG_ERROR([Can't find or Fortran library repositories.
link to the GPTL C library.])])
We start by cloning the combined distribution project,
As is good practice in configure scripts, error out if and using the following commands to add submodules:
required supporting libraries or tools cannot be found.
git submodule add
4.2 Makefiles [email protected]:jmrosinski/GPTL.git
git submodule add
The automake file to build the Fortran library is in the src [email protected]:NOAA-GSD/GPTL-fortran.git
subdirectory. It is slightly more complex than the one
used to build the C library (src/Makefile.am): This creates directories GPTL and GPTL-fortran, which
contain the contents of the C and Fortran library
libgptlf_la_FCFLAGS = -I$(top_srcdir)/include
if HAVE_PAPI
repositories.
libgptlf_la_FCFLAGS += -DHAVE_PAPI
endif Now the C and the Fortran libraries my be edited and
if HAVE_MPI tested as a unit. A separate git commit is required for
libgptlf_la_FCFLAGS += -DHAVE_MPI each of the three repositories which have been
endif changed. But the developer can make changes in either
# This is our output. The GPTL-fortran library, and then use the combined build to ensure that
# library. both libraries still work, and still work together.
lib_LTLIBRARIES = libgptlf.la
libgptlf_la_SOURCES = gptlf.F90 \ 5.2 Configuration
printmpistatussize.F90 \
process_namelist.F90 The configure.ac file for the combined distribution
# Install these in the include directory. project is small. After initializing autoconf, automake,
include_HEADERS = gptl.mod and libtool, the script launches the C and Fortran library
configuration scripts.
Note the .mod file, which will be installed in the include
directory. An additional option, --enable-package-build, has been
added to the Fortran library builds. Setting this option
The Fortran build does not use the config.h file, but notifies the Fortran library build that this is a combined
fortran 90 does support preprocessor ifdefs, and those library build, so it can find the C library.
are supported by setting macros with the -D option in
FCFLAGS. # Build the GPTL C library.
AC_CONFIG_SUBDIRS([GPTL])
4.2.1 Standard Targets
# Add this arg for the fortran build, to tell
As with the C library, standard targets are supported, # it to use the C library we just built.
ac_configure_args="$ac_configure_args \
including ​all​, ​check,​ ​clean​, ​install,​ ​dist,​ ​distcheck,​ and --enable-package-build"
uninstall​.
# Build the GPTL Fortran library.
5 COMBINED DISTRIBUTION CONTAINING C AND AC_CONFIG_SUBDIRS([GPTL-fortran])
FORTRAN LIBRARIES
AC_OUTPUT
The original GPTL distribution included both C and
Fortran libraries. We have broken these into separate 5.3 Top-Level Makefile.am
projects.
The Makefile.am file for the combined release launches
However, we would still like to have a combined the build in each of the two subdirectories:
distribution which includes both C and Fortran libraries.
SUBDIRS = GPTL GPTL-fortran
There is an easy way to accomplish this using autoconf.

We start by creating a new, third repo, which will hold This will cause first the C, and then the Fortran GPTL
the configuration files for the combined distribution. libraries to be built.

5.1 Use of git Submodules 5.4 Fortran Configuration Changes

The new combined repository also offers an The Fortran test directory needs to link to the C library.
improvement in programmer workflow. Using git In stand-alone Fortran library builds, the location of the
submodules we can achieve a combined development C library is specified in CPPFLAGS and LDFLAGS, and
the configure script adds the -lgptl to the link lines.
 flags in the hands of the end user. As a result, it will be
But with a combined build, the C library is not going to able to build with compilers and compiler options that
be available in that way, and is instead to be found in are not known to the GPTL developers. Even future
the GPTL/src directory. compilers and compiler options can be supported.

To accommodate this, the following was added to the 6.2 Additional Features
configure.ac:
The autotools packages provide some build and
AC_ARG_ENABLE([package-build], configure features that were not supported in the
[AS_HELP_STRING([--enable-package-build], previous build system:
[Set internally for package builds, \
should not be used by user.])]) ● Shared library builds.
test "x$enable_package_build" = xyes || \
enable_package_build=no
● Standard make targets dist, distcheck,
AM_CONDITIONAL([BUILD_PACKAGE], [test \
"x$enable_package_build" = xyes]) uninstall.

# Find the GPTL C library, unless this is a ● Standardised help setting specific configure
# combined C/Fortran library build. options with ​configure --help​.
if test $enable_package_build = no; then
AC_CHECK_LIB([gptl], [GPTLinitialize], [],\ 6.3 Reduction in Complexity
[AC_MSG_ERROR([Can't find or link to \
the GPTL C library.])]) Significant reduction in complexity has been achieved.
fi The total number of files required to support the build
has dropped from 30 to 14. The number of lines of code
This adds the --enable-package-build option to the (contents of make and configure files) has been reduced
Fortran build. When used, this option tells the configure by almost an order of magnitude.
and Makefiles that the C library is located as part of the
combined package, rather than already being installed.
build system total files lines of code

5.5 Fortran Test Makefile.am Changes

Within the Fortran library Makefile.am, the follow change legacy 30 4807
allows the build to find the C library, in combined library
builds:
autotools 14 593
# For combined C/Fortran builds, find the C
(combined
# library.
if BUILD_PACKAGE
C/Fortran
LDADD = ${top_builddir}/../GPTL/src/libgptl.la build)
endif
Reduction in complexity reduces maintenance costs,
5.6 Installing the Combined Package enables changes to be made quickly, and reduces the
chance of serious errors in the build system.
The combined package is built exactly like the individual
libraries. The options given to configure will be passed 6.4 Friendly to Linux Package Management Systems
to both C and Fortran builds. (Unknown configuration
options are ignored.) Using autotools, and separating the C and Fortran
libraries, allows the GPTL C and Fortran libraries to
work within the standard Linux package management
6 RESULTS systems. This will allow end users to install the library
using yum/apt package management tools, which is the
6.1 Greater Portability easiest and simplest way to install software on a Linux
system.
The previous build system for GPTL supported a variety
of build platforms, but the approach of specifying 6.5 Developer and User Workflow
compilers and flags in the makefile is fundamentally
limited to those that have been previously encountered 6.5.1 User Workflow
by the development team.
The end-user of the GPTL libraries does not need to
The new build system is capable of building on many know anything about the autotools: autoconf, automake
more systems, and leaves the selection of compiler and
and libtool. The end user does not even have to have
these tools installed on their system.

The GPTL libraries are installed from the distribution

tarball. This contains the configure script, and the
Makefile.in files that it uses to construct the Makefiles on
the build system.

To install the libraries, the user unpacks the tarball, then

runs ​configure​ and ​make install​.

6.5.2 Developer Workflow

Developers of GPTL will start from the git repository,

instead of a distribution tarball. The repository does not
(and should not) contain the intermediate files
generated by autoconf/automake. Instead, the
repository contains the source files that autoconf and
automake need to generate the build system.

After cloning the repository, developers must run

autoreconf -i​ to build the build system. This command
must also be run whenever the configure.ac file is
changed, to ensure that a new configure script is
generated.

When running autoreconf -i, many intermediate files will

be generated.These intermediate files must not be
committed to the repository. They will change slowly
over time, as the autotools packages release new
versions. Using autoreconf -i on the clean repository
clone will always result in a correct and consistent set of
intermediate files. Committing any of these files to the
repository may cause future subtle bugs if the autotool
that generates or needs that intermediate file is
upgraded. In such a case, the file that has been
committed to the repository may be incorrect for the new
version of the tool.

Developers must never edit the intermediate files.

Developers interact with the build system through the
configure.ac file (one per project) and the Makefile.am
files (one per subdirectory). These are the only build
system files that are ever edited by the developer.

7 ACKNOWLEDGMENTS

Jim Rosinski of NCAR is the author of the GPTL. His

assistance in the transition to the autotools build system
was invaluable.

8 REFERENCES

GPTL GitHub Site, ​https://github.com/jmrosinski/GPTL

GPTL-Fortran GitHub Site,

https://github.com/NOAA-GSD/GPTL-fortran

GPTL-all GitHub Site,

https://github.com/NOAA-GSD/GPTL-all