SAGA User Group Association Automation of Workflows with SAGA AGIT 2013

[vw@localhost ~]$ saga_cmd
Error: no arguments for saga call
Error: library

63 loaded module libraries (569 modules):
­ imagery_tools
­ ta_morphometry
...

type ­h or ­­help for further information

Automation of Workflows with

SAGA
Command line scripting in a nutshell

Volker Wichmann
alpS GmbH – Centre for Climate Change Adaptation

Laserdata GmbH
SAGA User Group Association Automation of Workflows with SAGA AGIT 2013

SAGA | Introduction – why scripting ?

Why scripting?Why


Allows to automate workflows

Why automatisation?Why


Automation is the use of ... information technologies to optimize productivity in the
production of goods and delivery of services (http://en.wikipedia.org/wiki/Automation)

several similar calculations that need Web Processing Service (WPS)

to be done for a large number of tiles, - a web interface requests a certain
e.g. import fromASCII, calculation processing result, e.g. a hillshade
of ... computation

V. Wichmann 2
SAGA User Group Association Automation of Workflows with SAGA AGIT 2013

SAGA | Introduction – SAGA CMD and Environmental Variables


Command line application: saga_cmd (.exe)


Environmental variable PATH:
– either change to the directory of the saga_cmd (.exe) binary
– or add the path to saga_cmd (.exe) to the PATH environmental variable
• print contents of the PATH variable
echo %PATH% [windows]
echo $PATH [linux]
• set PATH variable
set PATH=%PATH%;C:\my_path_to_saga_cmd [windows]
export PATH=$PATH:/my_path_to_saga_cmd [linux]


Environmental variable SAGA_MLB
– SAGA_MLB stores the path to the (optional) SAGA module libraries to be
used with saga_cmd (.exe)
• Remark: since SAGA 2.1.0, all native SAGA module libraries are found
automatically, making it obsolete to set SAGA_MLB in case you do not use any
own libraries stored in an additional path
set SAGA_MLB=C:\my_path_to_saga_mlbs [windows]
export SAGA_MLB=/my_path_to_saga_mlbs [linux]

V. Wichmann 3
SAGA User Group Association Automation of Workflows with SAGA AGIT 2013

SAGA | CMD usage


a simple call to saga_cmd results in:

[vw@localhost ~]$ saga_cmd
Error: no arguments for saga call
Error: library

63 loaded module libraries (569 modules):
­ shapes_points
­ io_grid_grib2
­ imagery_tools
­ ta_morphometry
...

type ­h or ­­help for further information

V. Wichmann 4
SAGA User Group Association Automation of Workflows with SAGA AGIT 2013

SAGA | CMD usage


a call to saga_cmd with --help results in:
[vw@localhost ~]$ saga_cmd –help
...
_____________________________________________
Usage:
saga_cmd [­h, ­­help]
saga_cmd [­v, ­­version]
saga_cmd [­b, ­­batch]
saga_cmd [­d, ­­docs]
saga_cmd [­f, ­­flags][=qrsilpx][­c, ­­cores][=#] <LIBRARY> <MODULE> <OPTIONS>
saga_cmd [­f, ­­flags][=qrsilpx][­c, ­­cores][=#] <SCRIPT>
[­h], [­­help]   : help on usage
[­v], [­­version]: print version information
[­b], [­­batch]  : create a batch file example
[­d], [­­docs]   : create module documentation in current working directory
[­c], [­­cores]  : number of physical processors to use for computation
[­f], [­­flags]  : various flags for general usage [qrsilpx]
  q              : no progress report
  r              : no messages report
  s              : silent mode (no progress and no messages report)
  i              : allow user interaction
  l              : load translation dictionary
  p              : load projections dictionary
  x              : use XML markups for synopses and messages
<LIBRARY>        : name of the library
<MODULE>         : either name or index of the module
<OPTIONS>        : module specific options
<SCRIPT>         : saga cmd script file with one or more module calls
_____________________________________________
Example:
  saga_cmd ­f=s ta_lighting 0 ­ELEVATION=c:\dem.sgrd ­SHADE=c:\shade.sgrd

V. Wichmann 5
SAGA User Group Association Automation of Workflows with SAGA AGIT 2013

SAGA | CMD usage


a call to saga_cmd with a <LIBRARY> results in:

[vw@localhost ~]$ saga_cmd ta_lighting

...

Error: module

modules:
 0 ­ Analytical Hillshading
 2 ­ Potential Incoming Solar Radiation
 3 ­ Sky View Factor
 4 ­ Topographic Correction
 5 ­ Topographic Openness
 6 ­ Visibility (points)

type ­h or ­­help for further information

V. Wichmann 6
SAGA User Group Association Automation of Workflows with SAGA AGIT 2013

SAGA | CMD usage


a call to saga_cmd with a <LIBRARY> and <MODULE> results in:
[vw@localhost ~]$ saga_cmd ta_lighting "Analytical Hillshading"

...
library path: /usr/local/lib/saga/libta_lighting.so
library name: Terrain Analysis ­ Lighting, Visibility
module name : Analytical Hillshading
author      : O.Conrad, V.Wichmann (c) 2003­2013
_____________________________________________

Usage: saga_cmd ­ELEVATION <str> [­SHADE <str>] [­METHOD <str>] [­AZIMUTH <str>]     
[­DECLINATION <str>] [­EXAGGERATION <str>] [­NDIRS <num>] [­RADIUS <str>]

  ­ELEVATION:<str>    Elevation   ­DECLINATION:<str>  Declination [Degree]

Grid (input) Floating point
  ­SHADE:<str>        Analytical Hillshading Default: 45.000000
Grid (output)   ­EXAGGERATION:<str> Exaggeration
  ­METHOD:<str>       Shading Method Floating point
Choice Default: 4.000000
Available Choices:   ­NDIRS:<num>        Number of Directions
[0] Standard Integer
[1] Standard (max. 90Degree) Minimum: 2
[2] Combined Shading Default: 8
[3] Ray Tracing   ­RADIUS:<str>       Search Radius
[4] Ambient Occlusion Floating point
Default: 0 Minimum: 0.001000
  ­AZIMUTH:<str>      Azimuth [Degree] Default: 100.000000
Floating point
Default: 315.000000
V. Wichmann 7
SAGA User Group Association Automation of Workflows with SAGA AGIT 2013

SAGA | CMD usage


a call to saga_cmd with a <LIBRARY> <MODULE> and <OPTIONS> results in:
[vw@localhost ~]$ saga_cmd ta_lighting "Analytical Hillshading" 
­ELEVATION=Mt_St_Helens_SRTM.sgrd ­SHADE="Mt_St_Helens_shade.sgrd" ­METHOD=0
...
_____________________________________________
library path: /usr/local/lib/saga/libta_lighting.so
library name: Terrain Analysis ­ Lighting, Visibility
module name : Analytical Hillshading
author      : O.Conrad, V.Wichmann (c) 2003­2013
_____________________________________________

Load grid: Mt_St_Helens_SRTM.sgrd...

100%okay

Parameters
Grid system: 30; 312x 458y; 557970x 5108130y
Elevation: Mt_St_Helens_SRTM
Analytical Hillshading: Analytical Hillshading
Shading Method: Standard
Azimuth [Degree]: 315.000000
Declination [Degree]: 45.000000
Exaggeration: 4.000000
Number of Directions: 8
Search Radius: 100.000000

101%Save grid: Mt_St_Helens_shade.sgrd...
100%okay
V. Wichmann 8
SAGA User Group Association Automation of Workflows with SAGA AGIT 2013

SAGA | CMD usage – Parameter Types

SAGA Parameter Types


Dataobjects:
– (Optional) Input
– (Optional) Output


Dataobject Types:
– Grid filename.sgrd
– Shapes filename.shp
– Tables filename.txt or filename.dbf
– Pointclouds filename.spc
– File(path) filepath/filename.ext
– Dataobject Lists filename.ext;filename.ext;filename.ext

– Table Field (shapes, tables, pointclouds):

• Fieldnumber (first field = 0)
• Fieldname “the field name”

– Table Fields (shapes, tables, pointclouds):

• Fieldnumbers 0,3,7,8
• Fieldnames id,landuse,slope,aspect V. Wichmann 9
SAGA User Group Association Automation of Workflows with SAGA AGIT 2013

SAGA | CMD usage – Parameter Types

SAGA Parameter Types


Boolean if ­IDENTIFIER is passed: Parameter = TRUE

if ­IDENTIFIER is NOT passed: Parameter = FALSE


Value integer or floating point value (decimal separator is .)

Degree unlike in the GUI: decimal degree (floating point)

Range two parameters: ­IDENTIFIER_MIN and ­IDENTIFIER_MAX

Choice the number of the choice to use

String character string, quoted in case it contains spaces

Grid System
­USER_XMIN Left
­USER_XMAX Right
­USER_YMIN Bottom
­USER_YMAX Top
­USER_SIZE Cellsize
V. Wichmann 10
SAGA User Group Association Automation of Workflows with SAGA AGIT 2013

SAGA | CMD Scripting Environment

Scripting Environment


Windows:
– Microsoft command interpreter (cmd.exe)
– batch files (*.bat): text files containing a series of commands to be executed
– http://en.wikibooks.org/wiki/Windows_Batch_Scripting


GNU/Linux, Mac OSX:
– Bash command processor (Unix shell)
– Bash files (*.sh): text files containing a series of commands to be executed
– http://tldp.org/LDP/abs/html/


Drawbacks:
– The parsing of a command line into a sequence of commands is complex and
varies from command interpreter to command interpreter, including
• Variable substitution
• Quoting
• Syntax
• Redirection
– Different scripts are needed on different operating systems
V. Wichmann 11
SAGA User Group Association Automation of Workflows with SAGA AGIT 2013

SAGA | CMD Scripting Environment

Scripting Environment


Python:
– Widely used general-purpose, high-level programming language
– Often used as scripting language
– http://docs.python.org/2/tutorial/


Advantages:
– Platform independent --> use the same script on different operating systems
– It's a complete programming language --> very powerful
• Statements and flow control
• Expressions, e.g. string manipulation
• Functions
• Libraries, providing tools suited to many tasks
• Redirection (logging)
• ...
– ... and there is also a SAGA Python API available, providing even more
flexibility

V. Wichmann 12
SAGA User Group Association Automation of Workflows with SAGA AGIT 2013

SAGA | CMD Scripting Examples

Scripting Examples


Task:
– Convert ESRI ASCII files (DEMs) stored in a folder to SAGA grids
– Calculate an analytical hillshade for each tile


Next slides show scripts for Windows, GNU/Linux and Python

V. Wichmann 13
SAGA User Group Association Automation of Workflows with SAGA AGIT 2013

SAGA | CMD Scripting Example Windows – batch_example.bat

V. Wichmann 14
SAGA User Group Association Automation of Workflows with SAGA AGIT 2013

SAGA | CMD Scripting Example GNU/Linux – bash_example.sh

V. Wichmann 15
SAGA User Group Association Automation of Workflows with SAGA AGIT 2013

SAGA | CMD Scripting Example Python – python_example.py

V. Wichmann 16
SAGA User Group Association Automation of Workflows with SAGA AGIT 2013

SAGA | CMD Scripting Example Python – python_example.py

V. Wichmann 17
SAGA User Group Association Automation of Workflows with SAGA AGIT 2013

SAGA | CMD Scripting Example Python – python_example.py

V. Wichmann 18
SAGA User Group Association Automation of Workflows with SAGA AGIT 2013

SAGA | CMD Scripting - Outlook

Outlook


SAGA Python API:
– Direct access to the SAGA API from the script
– No need to save datasets between module calls
– Examples on how to use the SAGA Python API are available in the SAGA
sources, see folder saga-gis/src/scripting/python/


... and in case you really like to stick to batch scripting: create an example script file
from the GUI as template ;-)

V. Wichmann 19