Using GDAL, PostGIS, and Parallel Computing Three Free Tools That Make Our Life Better

1
Using GDAL, PostGIS, and Parallel
ComputingThree Free Tools That Make Our Life
Better

• Zack Stauber and John Nipper
• Bohannan-Huston, Inc.

2
Why Use Open Source?

• Free, usually
• Better community support
• More standards compliant (e.g. OGC)
• Runs in parallel
• More transparency into whats going on under the
  hood
• Can program or tinker with it more easily

3
GDAL/OGR
4
GDAL

• Set of C programming tools for reading,
  writing, and manipulating images (including
  reprojection and datum shifts).
• Set of pre-built utilities for Windows and UNIX
  to use out-of-the-box. These are individual
  executables run from the command line
• http//www.remotesensing.org/gdal/

5
GDAL is Not

• Image processing tool no color,
  brightness/contrast, histograms, filters.
• Check OSSIM link on GDAL homepage for different
  image processing tools.
• Graphically oriented. Must learn to love the
  command line and it helps to be able to write a
  simple batch file.

6
GDAL Supported Formats
7
GDAL Tools

• FWTools
• http//www.gdal.org/dl/fwtools/
• OpenEV_FW
• Loads any images GDAL can read and keeps them
  georeferenced.
• OGR is a vector library bundled with GDAL.
  OpenEV_FW can also load shape files, MicroStation
  version 7 design files, Arc/Info coverages, some
  personal geodatabses, SDTS files, and PostGIS
  (yay!). No DXF.

8
GDAL Command Line Tools

• gdalinfo.exe
• gdal_translate.exe
• gdal_warp.exe
• gdalwarp.exe of GTiff s_srs EPSG26912 t_srs
  EPSG26913 myquad.tif myquadprj.tif
• gdal_tindex.exe
• gdaltindex.exe mytiles.shp .tif
• ogr2ogr.exe
• gdal_merge.py
• gdal_merge.py o 35108a4.tif of GTiff
  35108a41.tif 35108a42.tif 35108a43.tif
  35108a44.tif
• ogr2ogr.exe

9
GDAL/OGR SDK

• Open Source
• Spatial Imagery
• Vector Data
• Many Formats Supported by Drivers
• C Programming API
• Python

10
Calling GDAL with Python

• shp_driver ogr.GetDriverByName('ESRI
  Shapefile')
• shp_ds shp_driver.CreateDataSource(dirname)
• shp_layer shp_ds.CreateLayer(layerName,
  geom_type ogr.wkbPoint)
• fldName ogr.FieldDefn('Name', ogr.OFTString)
• fldName.SetWidth(15)
• shp_layer.CreateField(fldName)

11
PostGIS
12
PostGIS

• PostgreSQL
• http//www.postgresql.org/
• PostGIS
• http//postgis.refractions.net/
• OpenGIS Simple Features Spec for SQL
• GiST Index for Spatial Data
• Projections

13
Why use PostGIS?

• One big bag-O-data
• Get it for free
• Store, retrieve, and manage large datasets
• Multi-user
• Lots of reference material
• Easier than other spatial databases
• Program it using SDK and GDAL
• Indexed (easy)
• Custom Data Types (many included)
• Open Source and Standards Complient

14
PostGIS Queries

• Standard SQL
• select from streets where name Central
• Spatial Queries
• select from streets where intersects(GeometryFr
  omText(POLYGON((731.7 785.1 , 731.7 876.1 ,
  748.6 876.1 , 748.6 785.1 , 731.7 785.1))',-1),
  streets.geom)
• Indexed Queries

15
PostGIS Drawbacks

• Backups
• Viewing/Editing client support
• Database Administration

16
Parallel Computinga.k.a. Cluster Computing,
a.k.a. High Throughput Computing
17
Parallel Computing

• Condor University of Wisconsin
• http//www.cs.wisc.edu/condor/
• Free, not open source (anyone can use it, not
  alter it)
• Easy to install, hard to customize
• Runs standalone programs or batch files from
  command line
• Workers can be old, slow, unused machines out on
  network

18
Why would I use it?

• Build a large mass of base data in appropriate
  tiling/coordinate scheme
• Classify many satellite images at once
• Create shaded relief images from every available
  DTED of a new area (e.g. Mexico)

19
Submitting to Cluster
20
Condor Manages Submitted Jobs

• Knows which machines are ready and idle
• Holds jobs until a worker is found for them
• Handles security and priorities among users and
  jobs
• Continues to run jobs that fail (worker crashes,
  reboots, or worse is co-opted by machines
  rightful owner, tc.)

21
Condor Issues

• Worker must either have licensed software
  installed on it, or it must be sent to the worker
  with each job
• GDAL can be sent
• Public utilities can be sent (BLM tools, USGS
  tools, tc.)
• ESRI freebies can be sent
• GRASS can be installed on machines for free
• Some small utilities that come packaged with big
  expensive GIS packages can be run standalone
  without a license.
• Condor is ideal for many small jobs
• Image translation, reprojection, processing
  (GDAL)
• Rendering 3D frames (Lightwave 3D, POV-ray)
• Condor cannot be used for interactive tasks or
  ones that are done all at once
• Mosaicking orthophotos
• Editing shape files

22
Demo
23
Questions?
Zack Stauber zstauber_at_bhinc.com John Nipper
jnipper_at_bhinc.com Bohannan-Huston, Inc.