Unofficial ISCE Guide
Unofficial ISCE Guide
Contents
1 Binary files in ISCE 3
1.1 Creating a simple image object . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.2 Using casters: Advanced image object . . . . . . . . . . . . . . . . . . . . . . . . 4
1.3 Loading a file with an ISCE XML . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.4 Short hand . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1
Abstract
This document introduces various aspects of the ISCE software for users who are
familiar with basic Python constructs. This document tries to explain the structure and
usage of ISCE modules (same as Python classes), as stand-alone functional units. The
document is meant to be a guide to help users build their own workflows using ISCE
modules. Basic functional knowledge of the following concepts in Python are assumed -
lists, dictionaries, iterators, classes, inheritance, property, constructors and pickle.
ISCE is a mix of Python / C / C++ / Fortran routines. All intensive computing is
carried out in the C/C++/Fortran and Python provides an easy user interface to string
together these routines. In developing workflows, users should never have to deal with
the C/C++/Fortran directly. The basic assumption in ISCE is that if the information
is wired correctly at the Python level then the C/C++/Fortran routines should work
seamlessly. The Python level ISCE objects/ classes that are accessible to the users follow
standard Python conventions and should be easy to understand for regular python users.
This document is most useful for users who are not afraid of digging into Python source
code.
The document introduces basic ISCE concepts first and then moves on to describe some
example workflows. Remember that ISCE has an Application Framework (with extensive
features) that should be used to design production ready workflows. This document is a
guide to developing prototype workflows for testing or for personal use. These workflows
should be migrated to the Application Framework, once they mature and are intended to
be used in a production environment.
2
Learn Python
The authors of this document assume that the users are operating with basic functional knowl-
edge of Python. ISCE processing modules are stand alone components and are typically in-
cluded in the distribution as separate Python files (one per module). In our examples, we will
not bother ourselves with the implementation of the various processing algorithms. This doc-
ument focuses on interacting with the various Python classes in ISCE and assumes that users
are able and willing to read the Python source code for the various modules. ISCE modules
are reasonably well documented with doc strings and the source code is the best place to dig
for answers.
1. Filename
2. Width
3. Datatype
4. Interleaving Scheme
5. Number of Bands
6. Access mode
obj=isceobj.createImage()
obj.setFilename(’input.rmg’)
obj.setWidth(1000) #Width is required
#See components/iscesys/ImageApi/DataAccessor/bindings
3
obj.setDataType(’FLOAT’)
obj.setInterleavedScheme(’BIL’) #BIL / BIP / BSQ
obj.setBands(2) # 2 Bands
obj.setAccessMode(’read’) #read / Write
obj.createImage() #Now object is completely wired
#See components/iscesys/ImageApi/DataAccessor/bindings
obj.setDataType(’SHORT’)
obj.setScheme(’BIL’) #BIL / BIP / BSQ
obj.setBands(1) # 2 Bands
obj.setAccessMode(’read’) #read / Write
obj.setCaster(’read’,’FLOAT’) #Mode needed in caster definition
obj.createImage() #Now object behaves like float image
4
1.3 Loading a file with an ISCE XML
If you already have an image with a corresponding ISCE XML file describing it, you can use
obj = Image()
obj.load(’input.xml’)
obj.setAccessMode(’read’)
#obj.setCaster(’read’, ’FLOAT’) #if needed
obj.createImage()
obj.initImage(filename,mode,width,dataType)
#obj.setCaster(mode,’FLOAT’) #if needed
obj.createImage()
There are a number of helper image functions that are available. These functions use a
predetermined scheme and number of bands. They only need the filename, widths and casters
to create Image objects.
1. isceobj.createSlcImage() #CFLOAT, 1 band
2. isceobj.createDemImage() #SHORT, 1 band
3. isceobj.createIntImage() #CFLOAT, 1 band
4. isceobj.createAmpImage() #FLOAT, 2 band, BIP
5. isceobj.createRawImage() #BYTE, 1 band
5
2.1 Constructor
The __init__.py method of the Python class is called its constructor. All constructors of ISCE
components take in the keyword “name” as an optional input. When creating an instance of
a particular ISCE class, the value of this “name” field ties it to an external XML file that can
optionally be used to control its values. Here is an example:
import isce
from mroipac.Ampcor.Ampcor import Ampcor
In this case, “myexample.xml” will be scanned, whenever available, for setting up default
parameters when this instance of Ampcor class is first created. Note that loading of information
from XML files is done only during construction of the instance/ object. If the user does not
wish the instance to be controlled by users, do not provide the keyword “name” as an input
to the constructor. In general, the constructor is supposed to initialize the private variables of
the ISCE Component that are not meant to be accessed directly by the users.
2.2 Parameters
Parameters represent the input information needed by an ISCE Component / Class. One will
often see a long list of parameters defined for each Component. Here is an example:
This code snippet defines a parameter that will be part of an ISCE Component and will be
available as the member self.windowSizeWidth of the class after construction. The default
value for this member is an integer of value 64. The “mandatory” keyword is set to False, to
indicate that the user need not always specify this value. A default value is typically provided
/ determined by the workflow itself.
The public_name of a parameter specifies the tag of the property key to control its value
using an XML file. If we create an instance obj with the public name “myexample”, then an
XML file with the name “myexample.xml” and following structure will modify the default value
of obj.windowSizeWidth of the instance during construction.
6
##File 1
obj = Ampcor(name=’myexample’)
###myexample.xml
<myexample>
<component name="myexample">
<property name="WINDOW_SIZE_WIDTH">
<value>32</value>
</property>
</component>
</myexample>
An ISCE component typically consists of more than one configurable parameter. Each ISCE
component is also provided with an unique family name and logging_name. These are for
use by advanced users and are beyond the scope of this document. The list of all configurable
parameters of a Component are always listed in the self.parameter_list member of the
Class. Shown below is the list of parameters for Ampcor
class Ampcor(Component):
family = ’ampcor’
logging_name = ’isce.mroipac.ampcor’
parameter_list = (WINDOW_SIZE_WIDTH,
WINDOW_SIZE_HEIGHT,
SEARCH_WINDOW_SIZE_WIDTH,
SEARCH_WINDOW_SIZE_HEIGHT,
ZOOM_WINDOW_SIZE,
OVERSAMPLING_FACTOR,
ACROSS_GROSS_OFFSET,
DOWN_GROSS_OFFSET,
ACROSS_LOOKS,
DOWN_LOOKS,
NUMBER_WINDOWS_ACROSS,
NUMBER_WINDOWS_DOWN,
SKIP_SAMPLE_ACROSS,
SKIP_SAMPLE_DOWN,
DOWN_SPACING_PRF1,
DOWN_SPACING_PRF2,
ACROSS_SPACING1,
ACROSS_SPACING2,
FIRST_SAMPLE_ACROSS,
LAST_SAMPLE_ACROSS,
FIRST_SAMPLE_DOWN,
7
LAST_SAMPLE_DOWN,
IMAGE_DATATYPE1,
IMAGE_DATATYPE2,
SNR_THRESHOLD,
COV_THRESHOLD,
BAND1,
BAND2,
MARGIN,
DEBUG_FLAG,
DISPLAY_FLAG)
The key thing to remember when dealing with configurability is that the user defined XML
file is only used for instance initializtion. If the any of the parameters are redefined in the
workflow, the user-defined values are lost.
2.4 Ports
Ports provide a mechanism to configure an ISCE component with relevant information in a
quick and efficient way. For example, if a particular component needs the PRF, range sampling
rate and velocity; it might be easier or relevant to pass the metadata of a SAR raw / SLC
object with all this information through a port in one call. Instead of making three separate
function calls to add the three parameters individually, adding a port can help us localize the
setting up of these parameters to a single function and makes the code more readable.
Note that the ports are not the only way of setting up ISCE components. They only provide
a simple mechanism to efficiently do so in most cases. We will illustrate this with an example.
Ports are always defined using the createPorts method of the ISCE class. We will use
components/stdproc/rectify/geocode/Geocode.py to describe port definitions. There are
multiple equivalent ways of defining ports.
8
def createPorts:
#Method 1
slcPort = Port(name=’masterSlc’, method=self.addMasterSlc)
self._inputPorts.add(slcPort)
#Method 2
#self.inputPorts[’masterSlc’] = self.addMasterSlc
Both code snippets above define an input port named masterSlc, that automatically invokes
a class method self.addMasterSlc. In this case, the function addMasterSlc is defined as
follows:
def addMasterSlc(self):
formslc = self._inputPorts.getPort(name=’masterslc’).getObject()
if(formslc):
try:
self.rangeFirstSample = formslc.startingRange
except AttributeError as strerr:
self.logger.error(strerr)
raise AttributeError
Note that if the input port masterSlc is not wired up, no changes are made to the instance
/ object. Alternately, users can directly set the rangeFirstSample without using the ports
mechanism with a call as follows:
obj.rangeFirstSample = 850000.
Ports is a mechanism to use localize the wiring up of relevant information in different sections
of the code, and helps in the logical organization of the ISCE components.
In ISCE every port is accompanied by an addPortName method. The method takes one
python object as input. The exact manner in which these ports are used will become clear in
the next subsection. There are two equivalent ways of wiring up input ports:
9
2.5 Main Processing Method
In ISCE, component/ class names use camel case, e.g, DenseOffsets, Ampcor etc. The main
processing method of each of these components is the lowercase string corresponding to the
name of the Class. For example, the main method for class Ampcor is called —ampcor— and
that of class DenseOffsets is called denseoffsets. The class instances themselves are also
callable. Making a function call with the class instance results in automatic wiring of input
ports and execution of the main processing method. Typical use of an ISCE component follows
this pattern:
10
includes utilities that help users convert standard Python dictionaries into meaningful XML
files.
import isce
from isceobj.XmlUtil import FastXML as xml
if __name__ == ’__main__’:
’’’
Example demonstrating automated generation of insarApp.xml for
COSMO SkyMed raw data.
’’’
####Slave info
slave = {}
slave[’hdf5’] = ’slave.h5’
slave[’output’] = ’slave.raw’
#####Set sub-component
insar[’master’] = master
insar[’slave’] = slave
####Set properties
insar[’doppler method’] = ’useDEFAULT’
insar[’sensor name’] = ’COSMO_SKYMED’
insar[’range looks’] = 4
insar[’azimuth looks’] = 4
#####Catalog example
insar[’dem’] = xml.Catalog(’dem.xml’)
This produces an output file named ”insarApp.xml” that looks like this
11
<insarApp>
<component name="insar">
<component name="master">
<property name="hdf5">
<value>master.h5</value>
</property>
<property name="output">
<value>master.raw</value>
</property>
</component>
<component name="slave">
<property name="hdf5">
<value>slave.h5</value>
</property>
<property name="output">
<value>slave.raw</value>
</property>
</component>
<property name="doppler method">
<value>useDEFAULT</value>
</property>
<property name="sensor name">
<value>COSMO_SKYMED</value>
</property>
<property name="range looks">
<value>4</value>
</property>
<property name="azimuth looks">
<value>4</value>
</property>
<component name="dem">
<catalog>dem.xml</catalog>
</component>
</component>
</insarApp>
FastXML can be easily used to quickly generated input files for ISCE applications like
make raw.py, insarApp.py and isceApp.py.
12
#!/usr/bin/env python
import isce
import logging
import isceobj
import mroipac
import argparse
from mroipac.ampcor.Ampcor import Ampcor
import numpy as np
def cmdLineParser():
parser = argparse.ArgumentParser(description=’Simple ampcor driver’)
parser.add_argument(’-m’, dest=’master’, type=str,
help=’Master image with ISCE XML file’, required=True)
parser.add_argument(’-b1’, dest=’band1’, type=int,
help=’Band number of master image’, default=0)
parser.add_argument(’-s’, dest=’slave’, type=str,
help=’Slave image with ISCE XML file’, required=True)
parser.add_argument(’-b2’, dest=’band2’, type=int,
help=’Band number of slave image’, default=0)
parser.add_argument(’-o’, dest=’outfile’, default= ’offsets.txt’,
type=str, help=’Output ASCII file’)
return parser.parse_args()
13
####Stage 1: Initialize
objAmpcor = Ampcor(name=’my_ampcor’)
objAmpcor.configure()
if slaveImg.getDataType().upper().startswith(’C’):
objAmpcor.setImageDataType2(’complex’)
else:
objAmpcor.setImageDataType2(’real’)
if objAmpcor.downGrossOffset is None:
objAmpcor.downGrossOffset = coarseDown
14
######Stage 5: Get required data out of the processing run
offField = objAmpcor.getOffsetField()
logging.info(’Number of returned offsets : %d’%(len(offField._offsets)))
15
One can control the ampcor parameters using a simple example file in the same directory
as shown below:
<!--my_ampcor.xml-->
<my_ampcor>
<component name="my_ampcor">
<property name="ACROSS_GROSS_OFFSET">60</property>
<property name="DOWN_GROSS_OFFSET">-164</property>
</component>
</my_ampcor>
###These import lines are important. If the class definitions from these are
###unavailable, pickle will not be able to unpack binary data in pickle files
###to the correct objects
import isce
import isceobj
def load_pickle(fname=’PICKLE/formslc’):
import cPickle #Optimized version of pickle
if __name__ == ’__main__’:
’’’
Dummy testing.
’’’
insar = load_pickle()
16
vel, hgt = insar.vh()
planet = insar.masterFrame._instrument._platform._planet
print ’Planet :’, planet
Once the object is loaded, users can play around with the values stored in the InsarProc
object.
import isce
import isceobj
17
#!/usr/bin/env python
insarObj = cPickle.load(open(’PICKLE/{0}’.format(step),’rb’))
return insarObj
def createDummyModel():
’’’Creates a model image.’’’
wid = 401
lgt = 401
startLat = 20.0
deltaLat = -0.025
startLon = -156.0
deltaLon = 0.025
data.tofile(’model.enu’)
18
objModel.setFilename(’model.enu’)
objModel.setWidth(wid)
objModel.scheme = ’BIP’
objModel.setAccessMode(’read’)
objModel.imageType=’bip’
objModel.dataType=’FLOAT’
objModel.bands = 3
dictProp = {’REFERENCE’:’WGS84’,’Coordinate1’: \
{’size’:wid,’startingValue’:startLon,’delta’:deltaLon}, \
’Coordinate2’:{’size’:lgt,’startingValue’:startLat, \
’delta’:deltaLat},’FILE_NAME’:’model.enu’}
objModel.init(dictProp)
objModel.renderHdr()
return parser.parse_args()
if (modelImg.bands !=3 ):
19
raise Exception(’Model input file should be a 3 band image.’)
modelImg.setAccessMode(’read’)
modelImg.createImage()
20
for img in (objLon, objLos):
if (img.width != objLat.width) or (img.length != objLat.length):
raise Exception(’Lat, Lon and LOS files are not of the same size.’)
print(’Actual processing’)
####The actual processing
#Stage 1: Construction
converter = ENU2LOS()
converter.configure()
converter.enu2los(modelImage = modelImg,
latImage = objLat,
lonImage = objLon,
losImage = objLos,
outImage = objOut)
21
objLon.finalizeImage()
objLos.finalizeImage()
objOut.finalizeImage()
objOut.renderHdr() ###Create output XML file
22
7 Advanced Example 3: WGS84 orbits vs SCH orbits
In this example, we demonstrate the use of Orbit class in ISCE to deal with different types of
orbits. The example compares two different orbit interpolation schemes
1. WGS84 raw state vectors (to) WGS84 line-by-line vectors using Hermite polynomials (to)
SCH line-by-line vectors
2. WGS84 raw state vectors (to) SCH raw state vectors (to) SCH line-by-line vectors using
linear interpolation
23
#!/usr/bin/env python
import numpy as np
import isce
import isceobj
import stdproc
import copy
from iscesys.StdOEL.StdOELPy import create_writer
from isceobj.Orbit.Orbit import Orbit
if __name__ == ’__main__’:
##### Load insarProc object
print(’Loading original and interpolated WGS84 state vectors’)
iObj = load_pickle(step=’mocompath’)
24
####Delete the insarProc object from "mocomppath"
del iObj
####Note:
####insarApp converts WGS84 orbits to SCH orbits
####during the orbit2sch step
print(’*********************’)
orbSch = stdproc.createOrbit2sch(averageHeight=pegHavg)
orbSch.setStdWriter(stdWriter)
orbSch(planet=planet, orbit=origOrbit, peg=peg)
print(’*********************’)
schOrigOrbit = copy.copy(orbSch.orbit)
del orbSch
print(’Original WGS84 vectors to SCH’)
print(’Number of state vectors: %d’%len(schOrigOrbit._stateVectors))
print(’Time interval: %s %s’%(str(schOrigOrbit._minTime),
str(schOrigOrbit._maxTime)))
print(str(schOrigOrbit._stateVectors[0]))
25
####Line-by-line interpolation of SCH orbits
####Using SCH orbits as inputs
pulseOrbit = Orbit()
pulseOrbit.configure()
26
print(np.mean(np.abs(vdiff), axis=0))
print(’ ’)
print(’RMS in meters/sec’)
print(np.sqrt(np.mean(vdiff*vdiff, axis=0)))
27
8 Advanced Example 4: Working with GDAL, Landsat
and ISCE
In this example, we demonstrate the interaction between numpy, gdal and ISCE libraries. We
extract the Panchromatic band from a Landsat-8 archive obtained from Earth Explorer and
create a corresponding ISCE XML file. This file can then readily be ingested with any ISCE
module. An example would be dense pixel offset estimation on optical imagery.
28
#!/usr/bin/env python
import numpy as np
import argparse
from osgeo import gdal
import isce
import isceobj
import os
def cmdLineParse():
’’’
Parse command line.
’’’
parser = argparse.ArgumentParser(description=’Convert GeoTiff to ISCE file’)
parser.add_argument(’-i’,’--input’, dest=’infile’, type=str,
required=True, help=’Input GeoTiff file. If tar file
is also included, this will be output file extracted
from the TAR archive.’)
parser.add_argument(’-o’,’--output’, dest=’outfile’, type=str,
required=True, help=’Output GeoTiff file’)
parser.add_argument(’-t’,’--tar’, dest=’tarfile’, type=str,
default=None, help=’Optional input tar archive. If provided,
Band 8 is extracted to file name provided with input option.’)
return parser.parse_args()
data[’minx’] = gt[0]
data[’miny’] = gt[3] + data[’width’] * gt[4] + data[’length’]*gt[5]
data[’maxx’] = gt[0] + data[’width’] * gt[1] + data[’length’]*gt[2]
data[’maxy’] = gt[3]
data[’deltax’] = gt[1]
data[’deltay’] = gt[5]
data[’reference’] = ds.GetProjectionRef()
29
band = ds.GetRasterBand(1)
inArr = band.ReadAsArray(0,0, data[’width’], data[’length’])
inArr.astype(np.float32).tofile(outfile)
return data
fid = tarfile.open(intarfile)
fileList = fid.getmembers()
if src is None:
raise Exception(’Band 8 TIF file not found in tar archive’)
print(’Extracting: %s’%(src.name))
##Copy content
shutil.copyfileobj(srcid, destid)
fid.close()
destid.close()
if __name__ == ’__main__’:
####Parse cmd line
inps = cmdLineParse()
30
if inps.tarfile is not None:
extractBand8(inps.tarfile, inps.infile)
# print(meta)
####Create an ISCE XML header for the landsat image
img = isceobj.createDemImage()
img.setFilename(inps.outfile)
img.setDataType(’FLOAT’)
dictProp = {
’REFERENCE’ : meta[’reference’],
’Coordinate1’: {
’size’: meta[’width’],
’startingValue’ : meta[’minx’],
’delta’: meta[’deltax’]
},
’Coordinate2’: {
’size’ : meta[’length’],
’startingValue’ : meta[’maxy’],
’delta’: meta[’deltay’]
},
’FILE_NAME’ : inps.outfile
}
img.init(dictProp)
img.renderHdr()
31