Store, Manipulate and Analyze Raster Data in POSTGIS RASTER
Store, Manipulate and Analyze Raster Data in POSTGIS RASTER
Boreal Avian
Pierre Racine Modelling Project
Research Assistant
Centre for Forest Research
Steve Cumming
Centre for Forest Research
Wood and Forest Science Department
University Laval, Quebec, Canada
FOSS4G Denver
September 2011
Introducing PostGIS Raster
• Support for rasters in the PostGIS spatial database
- RASTER is a new native base type like the PostGIS GEOMETRY type
- Implemented very much like and is as easy to use as the PostGIS
GEOMETRY type
One table row = one raster (or tile)
One table = one coverage
- Integrated as much as possible with the GEOMETRY type
SQL API easy to learn for users already familiar with PostGIS
Full raster/vector analysis capacity taking nodata value into account
Operators & functions works seamlessly when possible
- First release with PostGIS 2.0 (soon)
• Development Team
- Current: Bborie Park, Jorge Arevalo, Pierre Racine,
David Zwarg, Regina & Leo Obe
- Past: Sandro Santilli, Mateusz Loskot
Chapter 13 on
• Founding PostGIS Raster
missing missing
tile tile
Table 1
empty space
Table 2
• Two modes
1. ONE_RASTER_PER_ROW
2. ONE_RASTER_PER_TABLE
What You Can Do Now?
Get raster statistics…
• ST_SummaryStats(raster)
- Return a set of (min, max, sum, mean, stddev, count (of
withdata pixels)) records
- 10 seconds for one SRTM tile of 3600 x 3600 pixels, 70MB
• ST_Quantile(raster, quantiles[ ])
- Return a set of values for an array of quantile
• ST_ValueCount(raster, values[ ])
- Return the frequency for an array of value
What You Can Do Now?
Display rasters…
• QGIS plugin by Maurício de Paulo
([email protected])
• gvSIG plugin by Nacho Brodin ([email protected])
• MapServer through GDAL
- Normally any software using GDAL to read raster and allowing
passing database connection parameters to GDAL
• ST_Reclass() a raster
• ST_MapAlgebra(raster, band,
-4 2 0 6 null null
expression, null
-1 -4 2 9 6
nodatavalueexpr, null null
-2 0 1 8
pixeltype)
- Expressions are evaluated by the PostgreSQL parser
- You can use any complex SQL expression
- e.g. 'CASE WHEN rast < 0 THEN rast+10 ELSE NULL END'
- You can provide a nodatavalueexpr to handle source nodata
values
What You Can Do Now?
Convert rasters to any GDAL format in SQL…
1000m
0m
What You Can Do Now?
Intersects rasters with polygons…
• Compute the mean temperature for each polygons of a table
SELECT bufID, (gv).geom buffer, (gv).val temp
FROM (SELECT bufID, ST_Intersection(geom, rast) gv
FROM buffers, temperature
WHERE ST_Intersects(geom, rast)
buffers temperature result
geom pointid raster geom pointID temp
id=24
polygon 24 raster polygon 24 11.2
polygon
polygon
46
31
∩ raster
raster
= polygon
polygon
53
24
13.4
15.7
temp=11.2
• With the raster API, PostGIS is now a very complete SQL GIS
- All data are implicitly tiled and spatially indexed
- No need to write complex C,C++, Python or JAVA code to
manipulate complex geographical datasets.
Desktop or Web
- Use SQL: The most used, most easy and most
Applicaton
minimalist though complete language to work (query building
with data in general. Easily extensible (PL/pgSQL) & display)
- Keep the processes close to the data where the
data should be: in a database! attribute,
vector
• Lightweight multi-users specialized SQL or raster
desktop and web GIS applications table
- All the (geo)processing is done in the database
- Applications become simple SQL query builders Spatial Database
and data (results) viewers (geoprocessing)
What You Can Do Now?
Implement a WPS server raster/vector geoprocessor…
Desktop or Web
WPS Client
WPS WPS
query response
WPS Server
attribute,
vector
SQL
or raster
table
PostGIS
(geoprocessing)
Performance?
• Import of 1 GB SRTM DEM files
- tiled to 48373 100x100 pixels tiles: 3 minutes
- tiled to 525213 30x30 pixels tiles: 6 minutes
• ST_Intersection() of 814 buffers with the 30x30 SRTM
- 4 minutes
• ST_Intersection() of 100 000 lines with a 300 MB landsat
coverage
- 8 minutes
• Recently selected by the main Canadian governmental
provider of geospatial data (GeoBase)
- online on-the-fly and internal elevation product generation
• PostGIS raster is still a baby, many optimizations are
still possible
Summary
• PostGIS Raster is multiband, tiled, multiresolution
- Each band supports one nodata value, one pixel type.
- One row = one raster, one table = one coverage.
- Supports many tile arrangement.
- Very much like a vector coverage.
- Import is done the same way as usual with PostGIS:
raster2pgsql
• There are plenty of functions to…
- manipulate,
- edit,
- do raster and raster/vector analysis,
- get raster statistics,
- create new rasters,
- write web and desktop applications.
Summary
• Roadmap…
- Two raster version of ST_Intersection()
- Neighbor version of ST_MapAlgebra()
- Two rasters version of ST_MapAlgebra()
- Aggregate rasters with ST_Union()
- Statistic functions as aggregates
- ST_Interpolate() from irregular grid of point (lidar)
- ST_AsDensity() to produce density maps
Chapter 13 on PostGIS
Raster April 2011
Some extra slides…
Comparison with Oracle GeoRaster
• ST_MakeEmptyRaster()
• ST_AddBand()
- Empty band or copy a band from another raster
• All georeference setters
- ST_SetScale (), ST_SetSkew(), ST_SetUpperLeft(), ST_SetGeoReference()
• ST_SetBandNodataValue
• ST_SetValue()
• Coordinates transformation helpers
- ST_World2RasterCoordX(), ST_World2RasterCoordY(),
ST_Raster2WorldCoordX(), ST_Raster2WorldCoordY()
• ST_Intersection() & ST_intersects()
- To interact with vector data
• Many more…
What You Can Do Now?
Develop new raster processing functions…
• It should allows
- loading raster in the database using gdal_translate
- loading many raster at the same time
- any application writing to GDAL to write to PostGIS raster
- tiling a raster to any tile size
- to create overviews
What You Can Do Soon?
Complex MapAlgebra analyses…
• Already available: One raster version of ST_MapAlgebra()
• Soon: Faster user-defined function version
- Function taking a pixel value and some
parameters and returning a computed value
CREATE FUNCTION polynomial(x float,
VARIADIC args TEXT[])
RETURNS FLOAT AS $$
DECLARE
m FLOAT;
b FLOAT;
BEGIN
m := args[1]::FLOAT;
b := args[2]::FLOAT;
return m * x + b;
END; $$ LANGUAGE 'plpgsql';
• ST_Interpolate(pts geometry)
- Should be an aggregate returning one raster
(or a set of tiles)
- Implementing many different interpolation
algorythms
Nearest neighbor, linear, polynomial
- Very useful to convert lidar data to raster
• ST_AsDensity(geometry)
- Count the number of features touching each
pixel and then smooth the surface using a
moving window (neighbor map algebra)
What You Can Do (maybe not too) Soon?
Create a clean raster coverage… from a messy one…
3. ST_MakeEmptyRasterCoverage()
- Create a vector grid or an empty raster coverage based on a set
of parameters