GISC 6389 GIS MASTER PROJECT A Web Servicebased Electrical GIS MapServer via Open Source GIS Softwar

1
GISC 6389 GIS MASTER PROJECTA Web Service-based
Electrical GIS MapServer via Open Source GIS
Software

• Tanakorn Jangkakul
• Advisor Dr. Fang Qiu
• Dr. Ronald Briggs
• Fall 2007

2
Overview

• Introduction
• Problem Statement
• Project Objective
• Literature Review
• Data Source
• Design
• Methodology
• Result
• Conclusions
• References
• Application Demo

3
Introduction

• Currently, Internet GIS has emerged to overcome
  several limitations of popular desktop GIS
  software by providing the following advantages
• There is no need to install and maintain GIS
  software on every desktop
• Users can access to geospatial data and spatial
  analysis tools on any desktop with internet
  access at any time
• Users do not need to purchase proprietary GIS
  application software, because of the availability
  of many open-source GIS software, which provides
  a variety of different display and spatial
  analysis functions as well as a open development
  environment

4
Introduction

• In this fashion, electric companies also
  capitalized these advantages to provide Internet
  GIS services
• Outage Management
• Reduce outage duration
• Reduce operation cost and increase revenue
• Increase crew accessibility
• Enhance customer satisfaction

5
Problem Statement

• ArcGIS geometric network can determine the flow
  of electricity that assists utility companies to
  determine outage devices via a upstream tracing
  method
• Is there any approach to export the relationship
  in geometric network so that it can be displayed
  on Internet GIS and utility companies can predict
  location of outage devices using the exported
  data ?

6
Project Objective

• Develop a methodology for the efficient
  conversion of electric utility data from a
  geometric network model in desktop ArcGIS to the
  PostGIS open source spatial database.
• Develop an algorithm to allow utility company to
  predict outage devices using PostGIS based SQL to
  query spatial database
• Display the geometric network and predicted
  device location stored in PostGIS online using
  open source MapServer

7
Literature Review

• Menno-Jan Kraak. 2004, The role of the map in a
  Web-GIS environment. Journal of Geographical
  Systems. Publisher Springer Berlin/Heidelberg
  Volume 6 Number 2 June, 2004. Pages 83-93
  (accessed September 15, 2007)
• Maps have a important rule to play in Web-GIS
  environment in both interactive and dynamic
  appearance. They also assist the users in solving
  geospatial analysis problems, and unlimited
  number of users can access the maps anywhere at
  all times. Further, well-designed Web map can
  help users to attract attention.
• However, the performance of working in Web-GIS
  environment depends on several technological
  factors such as internet connection, traffic
  intensity, data efficiency, and the capability of
  client and server machines.

8
Literature Review

• Ranga RaJu Vatsavai, Shashi Shekhar, Thomas E.
  Burk, and Stephen Lime. 2006. UMN-MapServer A
  High-Performance, Interoperable, and Open Source
  Web Mapping and Geo-Spatial Analysis System.
  Geographic, Information Science Book Series
  Lecture Notes in Computer Science. Publisher
  Springer Berlin / Heidelberg volume 4197/2006
  Pages 400-417 (accessed August 10, 2007)
• Generally, Web GISes are predominantly
  designed under a thin-client/fat-server
  paradigm which causes several disadvantages. For
  example, as the users increases, the load of
  server also increases, therefore, the system
  performance would be decreased. To alleviate this
  drawback, a load balancing client/server approach
  would be presented by gradually shifted toward
  client-side GIS through client technology such as
  java applet, JavaScript, ActiveX controls and
  extendible Web Clients.
• Also, MapServer has interoperability to read
  various OGC standard data types such as GML,
  shapefile, WMS, and more

9
Literature Review

• Bo Huang and Hui Lin. 2004. Design of a Query
  Language for Accessing Spatial Analysis in the
  Web Environment. Journal GeoInformatica Publisher
  Springer Netherlands volume 3, Number 2 / June,
  1999 Pages 165-183 (accessed August 10, 2007)
• Commonly, Web GISes have been used for spatial
  data access, spatial visualization, and simple
  retrieval mapping. This lacks of analytic
  capabilities for GIS applications. The spatial
  query language can act as a guideline for query
  process that is essential for Web GIS. Also,
  there are 2 main implementation approaches to
  query spatial feature in tables common SQL and
  SQL with geometry types.
• Currently, there have been a number of
  different approaches implementing Web GIS such
  as
• Plug-in which programs are downloaded into the
  browser environment
• Functions invoked from the browser are executed
  on the server via CGI
• Hybrid model where certain functions are executed
  on the server delivering a image file and an
  Application Programming Interface (API) delivers
  specific data to a client for local processing

10
Data Source

• Data acquires from Idaho Falls Electric Power
  Company
• Electric data
• Transformer, Switches, Breakers, Capacitors,
  Fuses, Meters,
• Acquires on June 1, 2007 exported from ArcSDE to
  personal Geodatabase
• Coordinate system Idaho Falls Ground
• Projection system Transverse Mercator
• Datum NAD 83
• Unit Foot

11
Data Source

• Data acquires from Idaho Falls Electric Power
  Company
• Base map feature classes
• Highways, Streets, Parcels, Railroad, Building
• Acquires on June 1, 2007 exported from ArcSDE to
  personal Geodatabase
• Coordinate system Idaho Falls Ground
• Projection system Transverse Mercator
• Datum NAD 83
• Unit Foot

12
Software

• Apache 2.2.4
• PHP
• JavaScript
• HTML/DHTML
• CGI

• ArcGIS Desktop 9.2
• Visual Studio 2005
• MapServer 4.10
• PostgreSQL/ PostGIS

13
Design

• Construct geometric network in ArcGIS
• Generate XML configuration file
• Create database in postgreSQL
• Export the connectivity of geometric network from
  ArcGIS to postGIS
• Produce map file for MapServer
• Design html pages based on JavaScript and PHP
• Program outage device prediction

a diagram of a mapping website interacts with
programs running on the servers (Tyler 2005)
14
How Power Electric Distribution Works

• After 3-phase power enters a substation
• The substation uses a large transformer to step
  transmission voltage down to distribution
  voltages
• Busbars are used to split the distribution power
  off in multiple direction
• Breakers and switches are used to separate
  distribution lines (primary line, or circuit)
• At each house, there is a transformer to reduce
  voltage from primary line to secondary line and
  run into a meter of each house

Source http//science.howstuffworks.com/power.htm
15
Methodology Construct Geometric Network

• Geometric Network is a relationship model of
  connected edges (lines) and junctions (points) to
  represent behavior of network infrastructure in
  the real world.
• Geometric network consists of
• Simple edge/junction represents as a single
  feature
• Complex edge represents as a single feature that
  allows edges to connect without junctions
• Network junction, or orphan junction used to
  maintain network integrity

Source http//webhelp.esri.com/arcgisdesktop/9.2/
index.cfm?TopicNameAn_Overview_of_Network_Analyst

16
Methodology - Logical Network

• Created correspondingly when a geometric network
  is created
• Represent a model connectivity relationships
  between features used for tracing and flow
  calculations

Edge Table
Junction Table
12
Connectivity Table
2
11
13
circuit
10
1
transformer
Secondary line
Logical Network
Network junction
Source http//webhelp.esri.com/arcgisdesktop/9.2/
index.cfm?TopicNameAn_Overview_of_Network_Analyst

17
Methodology Connectivity Rule

• When utility network models become complicated,
  the network connectivity rules constrain the type
  of network to be connected to the others.

• For example
• Transformer can be connected from circuit to
  circuit or circuit to secondary line
• Circuit can be connected to crossarm via
  transformer

18
Methodology Export Geometric Network to PostGIS
using ArcObject

• ArcObject is a collection of platform-independent
  components (COMs) that provide service to support
  ArcGIS applications
• COM is a protocol that connects one software
  component with another
• .Net Framework interacts with COM via COM wrapper
  classes to allow COM communicating with .Net
  components

Source http//edndoc.esri.com/arcobjects/9.2/Comp
onentHelp/esriGeoDatabase/GeoDatabase_overview.htm
19
Methodology Geometric Network Exporting
Algorithm
20
Methodology Generate XML Configuration file

• To express configuration of each feature class
  and fields to be exported into PostGIS
• Mandatory fields are current element
  (enetworkid), parent element (parentid), and
  geometry (geodata) of each feature
• Optional fields can be selected and rename from
  existing fields of each feature class
• Also, a connection description and EPSG (European
  Petroleum Survey Group) numbers

EPSG codes are numeric codes associated with
coordinate system definitions Source
http//mapserver.gis.umn.edu/docs/glossary/epsg
21
Methodology Modifying Projected Coordinate
System

• To standardize from a custom projected coordinate
  system to UTM NAD 83 with the right EPSG number
• IGeometry geometry
• string geo ""
• ISpatialReferenceFactory srFactory new
  SpatialReferenceEnvironmentClass()
• IProjectedCoordinateSystem pcs
  srFactory.CreateProjectedCoordinateSystem((epsgNum
  ber))
• ISpatialReference sr pcs

C.net code example
EPSG codes are numeric codes associated with
coordinate system definitions Source
http//mapserver.gis.umn.edu/docs/glossary/epsg
22
Methodology Reading Point and Line Coordinates
C.net code example

• if (currentElem.ElementType esriElementType.esr
  iETEdge)
• (IGeometry)geometry _network.get_GeometryForEdg
  eEID(currentElem.EID)
• if (geometry.GeometryType esriGeometryType.es
  riGeometryPolyline)
• IPolyline polyline (IPolyline)geometry
• IPointCollection polylinePoint
  (IPointCollection)polyline
• for (int i 0 i lt polylinePoint.PointCount
  i)
• IPoint pntTemp polylinePoint.get_Point(i)
  
• pntTemp.Project(sr)
• geo geo Convert.ToString(pntTemp.X) "
  " Convert.ToString(pntTemp.Y)
• int j i1
• if (j ! polylinePoint.PointCount) geo
  geo ", "
• else if (currentElem.ElementType
  esriElementType.esriETJunction)
• (IGeometry)geometry _network.get_GeometryForJu
  nctionEID(currentElem.EID)
• if (geometry.GeometryType
  esriGeometryType.esriGeometryPoint)
• IPoint point (IPoint)geometry
• point.Project(sr)

23
Methodology PostgreSQL library

• PostgreSQL.NET library
• The library provides a complete .NET functions to
  allow .NET program to access a PostgreSQL
  database

C.net code example
using System using Npgsql class Program
static void Main(string args)
NpgsqlConnection conn new NpgsqlConnection("S
erver127.0.0.1Port5432User IdxxxPasswo
rdxxxxDatabasexxxxx") conn.Open()
conn.Close()
24
Methodology Inserting a Record to PostGIS
Database
C.net code example
using System using Npgsql class Program
static void Main(string args)
NpgsqlConnection conn new NpgsqlConnection("S
erver127.0.0.1Port5432User IdxxxxxPass
wordxxxxxDatabasexxxxxxxxxx")
conn.Open() Console.WriteLine("DB
connected") string SQL "INSERT INTO
SubstationXFMR values('1','9','STR.9','-1','7','Su
bstationXFMR','STR','1',GeometryFromText
('POINT(411089.808356836 4813503.22635251)',26912)
)" NpgsqlCommand command new
NpgsqlCommand(SQL, conn) try
rowsaffected command.ExecuteNonQuery()
finally Console.WriteLine("It was
added 0 lines in table ", rowsaffected)
conn.Close()
25
Methodology - Spatial Database

• A relational DB allowing to store a large sets of
  objects (BLOB) including location, shape, and
  dimension
• Historically, DB were not spatial enabled and
  could not directly store information about
  location and shape. Therefore, the spatial
  component was store separately from the main
  attribute database and using a reference present
  in both systems. This caused inconsistent and
  desynchronized.
• Therefore, bringing the spatial information
  together with attribute information enables the
  DB to provide its standard transaction support,
  integrity check, access control, indexing and
  other services.

26
Methodology - PostGIS Spatial Database

• A Spatial DB extension for the PostgreSQL (RDBMS)
  same as Oracle Spatial, DB2 Spatial, and SQL
  server Spatial
• Providing spatial DB functionality spatial
  objects, spatial indexing, standard input/output
  representations, spatial functions and spatial
  operators
• Provide all of the functions and objects defined
  in the openGIS Simple Features of SQL
  specification, standard open source spatial
  database
• Support advance spatial processing and querying
  entirely by using the SQL command-line

27
Methodology Creating PostGIS Spatial Database

• Creating A PostGIS Database
• Creating object definitions
• Creating extra tables that support the PostGIS
  system
• geometry_column for keeping track of the geometry
  columns and constraints
• spatial_ref_sys for keeping track of the
  different spatial referencing systems that
  geometries are stored in
• Inserting spatial object functions
• Inserting coordinate system codes

28
Methodology Creating PostGIS Spatial Database

• Creating A PostGIS Table using SQL command
• Using psql internal command interpreter, to add
  table to database
• Inserting Data into PostGIS
• PostGIS uses the Well-Known Text (WKT) format to
  describing geometries such as point, linestring,
  multilinestring, polygon, multipolygon.
• Creating A Spatial index
• A spatial index used to speed up bounding-box
  based queries
• Update the databases cache of statistics and
  reorganize structures of some indexes to increase
  the speed of queries

29
Methodology - MapServer

• Map-rendering engine that works in a web
  environment as a CGI script or as a standalone
  application via an API from several programming
  language such as C, Perl, Python
• Started since 1994 and supported by NASAs ForNet
  and TerraSIP Projects
• Features
• Open Source Server-based application
• Support OGC Web Service Specifications
• Support raster data file such as TIFF/GeoTIFF
• Support vector data file such as shapefiles,
  mapinfo, coverages, Tiger
• Support spatial database connections such as
  postGIS, ArcSDE
• Extremely Configurable
• Work with any HTTP server that Support the CGI
  protocol
• Apache, IIS, Tomcat, etc.

30
Methodology How MapServer Works

• Template based
• Receive a web request from a client
• Read a configuration file (mapfile) from web
  server

• Draw and save the map in image files
• Send the data to the web server
• Forward to the browser

basic operation of MapServer (Tyler 2005)
31
Methodology CGI

• A protocol coping with the information exchange
  between the client and the server.
• Thus, when a client submits a web request, hidden
  variables are passed via CGI to MapServer in the
  Web server
• ltform name"mapserv" methodGET
  action"/Scripts/Maps/mapserv.exe"gt
• ltinput type"hidden" name"map"
  value"/xxx/map_export.map"gt
• ltinput type"hidden" name"program"
  value"/xxx/xxx/mapserv.exe"gt
• ltinput type"hidden" namemode" valuemap"gt
• ltinput type"hidden" name"layer"
  valuecircuit"gt
• ltinput type"submit" value"Submit"gt
• After MapServer finished processing,
• it sends results (image variables) back to
  the client

http//localhost/Scripts/Maps/mapserv.exe?program
Scripts/Maps/mapserv.exemap/xxx/map_export.mapm
odemaplayercircuit
32
Methodology Produce Mapfile for MapServer

• Define the content of each features to be drawn
• Points MapServer to where data are located
• Define how features are to be drawn

LAYER NAME 'circuit' TYPE line CONNECTIONTYPE
postgis CONNECTION "userxxxx'
dbnamexxxxx' host'LOCALHOST' port5432
passwordxxxx'" DATA 'geodata from circuit'
STATUS ON TRANSPARENCY 50 CLASS
NAME 'circuit' SYMBOL 'bigline' COLOR 0 255
0 green SIZE 2 END end of Class END
end Layer circuit
33
Methodology Predicting Outage Devices

• A customer reports an outage
• Locate upstream to the nearest secondary line and
  a transformer with a same phase
• Locate all other secondary lines to this
  transformer
• Update the database and flag possibly affected
  secondary lines
• Display the updated database with highlights

34
Methodology Predicting Outage Devices

• If more reported customers in a different
  transformer network
• Locate the primary line and predict the upstream
  devices such as fuse, switch or capacitor.

35
Results PostGISCreating Spatial Database via a
Batch file

• _at_echo off
• echo.
• echo This batch file will create PostGIS
  database
• echo.
• echo 1. create PostGIS Database
• echo 2. create language and spatial reference
  in PostGIS Database
• echo 3. create geometry colume in PostGIS
  Database
• echo.
• cd C\Program Files\PostgreSQL\8.2\bin
• echo 1. create PostGIS Database
• createdb.exe -U root idahofalls
• echo.
• echo 2. create language and spatial reference
  in PostGIS Database
• createlang.exe -U root plpgsql idahofalls
• echo CreateTable spatial_ref_sys (coordinate
  system codes)
• psql.exe -U root -d idahofalls -f lwpostgis.sql
• echo.
• echo 3. create geometry colume in PostGIS
  Database
• echo This file also contains the CreateTable SQL
  for the metadata table geometry_columns

36
Results PostGIS Creating Spatial Database via
a Batch file

• echo.
• remcd \"Program Files"\PostgreSQL\8.2\bin\
• echo 4. create tables in PostGIS Database via
  psql command
• echo.
• psql.exe -U root -d idahofalls -f
  \NetworkExport\SQL\substationxfmr.sql
• psql.exe -U root -d idahofalls -f
  \NetworkExport\SQL\capacitor.sql
• psql.exe -U root -d idahofalls -f
  \NetworkExport\SQL\busbar.sql
• psql.exe -U root -d idahofalls -f
  \NetworkExport\SQL\circuit.sql
• psql.exe -U root -d idahofalls -f
  \NetworkExport\SQL\transformer.sql
• psql.exe -U root -d idahofalls -f
  \NetworkExport\SQL\switches.sql
• psql.exe -U root -d idahofalls -f
  \NetworkExport\SQL\fuse.sql
• psql.exe -U root -d idahofalls -f
  \NetworkExport\SQL\breaker.sql
• psql.exe -U root -d idahofalls -f
  \NetworkExport\SQL\secondary.sql
• psql.exe -U root -d idahofalls -f
  \NetworkExport\SQL\meter.sql
• pause

37
Results Creating a XML Configuration File
ltSourceFeaturesgt ltSourceFeature
FeatureClassName"SubstationXfmr" SQL"ObjectID
in (9)" /gt lt/SourceFeaturesgt ltFeaturesgt ltFeatureC
onfig FeatureClassName"SubstationXfmr"
Abbrev"STR"gt ltFieldsgt ltNetworkIDFieldgt ltOut
putFieldNamegtENETWORKIDlt/OutputFieldNamegt lt/Netw
orkIDFieldgt ltParentFieldgt ltInputFieldNamegtENE
TWORKIDlt/InputFieldNamegt ltOutputFieldNamegtPAREN
TIDlt/OutputFieldNamegt lt/ParentFieldgt lt/Fieldsgt
lt/FeatureConfiggt lt/Featuresgt lt/ExportConfiggt

• Substation Transformer
• Capacitor
• Breaker
• Transformer
• Fuse
• Switch
• Meter

• Busbar
• Circuit (primary line)
• Secondary (secondary line)

38
Results ArcGISExporting Geometry Network to
PostGIS
39
Results ArcGIS Exporting Geometry Network to
PostGIS
40
Results ArcGISExporting Result
41
Results Mapfile

• A mapfile code created and displayed by MapServer

http//localhost/Scripts/Maps/mapserv.exe?map/dat
aVoice/ Config/Maps/idahofalls/idahofalls6-63.map
modemap layersall
42
Results IMS Website created by customizing
MapServer

• A IMS website has been created by using HTML,
  JavaScript, MapServer CGI variables and PHP
  script

43
Results Read and Display a Mapfile
44
Results Predicted Outage Devices
45
Demo
46
Conclusions

• Open source spatial database can be created by
  imported from proprietary format such as ArcGIS
  personal database
• Open source software (MapServer) can be used and
  displayed spatial feature and analysis results
  online
• Open source software can provide open development
  environments to implement use specific analysis
  functions
• Open source can achieve almost everything that
  expensive commercial software can do but does not
  cost anything except an interns work

47
Future Works

• Utilizing more SQL-spatial function
• Improving web interface
• Determining how to present flow directions via
  MapServer
• Optimizing exporting application to enhance the
  performance

48
References

• Tyler Mitchell. 2005. Web Mapping Illustrated.
  OReilly Publisher
• Bil Kropla. 2005. Beginning MapServer Open
  Source GIS Development. Apress Publisher
• Robert Burke. 2003. Getting to Know ArcObjects.
  ESRI Press Publisher
• Kang-Tsung Chang. 2004. Programming ArcObjects
  with VBA, A Task-Oriented Approach. CRC Press
  Publisher
• Karli Watson, Christian Nagel, Jacob Hammer
  Pederson, Jon D. Reid, Morgan Skinner, and Eric
  White. 2006. Beginning Visual C 2005. Wrox
  Publisher.
• Korry Douglas, Susan Douglas. 2003. PostgreSQL A
  comprehensive guide to building, programming, and
  asminsitratering PostgreSQL databases. Sams
  Publisher
• Menno-Jan Kraak. 2004, The role of the map in a
  Web-GIS environment. Journal of Geographical
  Systems. Publisher Springer Berlin/Heidelberg
  Volume 6 Number 2 June, 2004. Pages 83-93
• Ranga RaJu Vatsavai, Shashi Shekhar, Thomas E.
  Burk, and Stephen Lime. 2006. UMN-MapServer A
  High-Performance, Interoperable, and Open Source
  Web Mapping and Geo-Spatial Analysis System.
  Geographic, Information Science Book Series
  Lecture Notes in Computer Science. Publisher
  Springer Berlin / Heidelberg volume 4197/2006
  Pages 400-417
• Bo Huang and Hui Lin. 2004. Design of a Query
  Language for Accessing Spatial Analysis in the
  Web Environment. Journal GeoInformatica Publisher
  Springer Netherlands volume 3, Number 2 / June,
  1999 Pages 165-183
• MapServer. http//mapserver.gis.umn.edu
• PostGIS. http//postgis.refractions.net
• PostgreSQL. http//www.postgresql.org
• GDAL Geospatial Data Abstraction Library.
  http//www.gdal.org/
• OGR Simple Feature Library. http//www.gdal.org/
  ogr/
• ArcGIS Desktop Help 9.2. http//webhelp.esri.com/a
  rcgisdesktop/9.2/index.cfm?TopicNamewelcome

49
Acknowledgements

• Special Thank you for their helpful suggestions
• Dr. Fang Qiu
• Dr. Ronald Briggs
• DataVoice International Inc.
• Roy Rooker
• Albert Law
• Stuffs
• David McCourt

50
ArcObject COM Implemented

• ISpatialReferenceFactory Interface
• ISpatialReference Interface
• IProjectedCoordinateSystem Interface
• IGeometry Interface
• IPoint Interface
• IPointCollection Interface
• -------------------------------------------
• IGxDialog Interface
• IGxObjectFilter Interface
• IEnumGxObject Interface
• ITrackCancel Interface
• IProgressDialogFactory Interface
• IProgressDialog2 Interface
• -------------------------------------------
• esriElementType Constant
• esriFeatureType Constant
• -------------------------------------------

• IRow Interface
• IFeatureClassContainer Interface
• IFeatureDataset Interface
• IDataset Interface
• IFeatureWorkspace Interface
• IFeatureSelection Interface
• IFeatureClass Interface
• IFeatureCursor Interface
• IFeature Interface
• ILayerFile Interface
• ILayer Interface
• ISelectionSet Interface
• IQueryFilter Interface
• -------------------------------------------
• IGeometricNetwork Interface
• IUtilityNetwork Interface
• INetElementDescriptionEdit Interface
• INetworkFeature Interface
• IEdgeFeature Interface