GIS Data: Processing

ArcToolbox, Geoprocessing and

ModelBuilder

Spring 2008

1
GISC 6382 Applied GIS UT-Dallas Ron Briggs
ArcToolbox, Geoprocessing, ModelBuilder : Overview

ArcToolbox provides access to geoprocessing

• ArcToolbox is a dockable window in ArcCatalog
or ArcMap
• Comprises a set of Tools (750+)
• Tools support all data types (coverages,
shapefiles, gdb, raster)
• Multi- step workflow available via Modelbuilder
and/or scripting
• Command line interface also available for
traditionalists and/or power users
2
GISC 6382 Applied GIS UT-Dallas Ron Briggs
 What is geoprocessing?
• the modification and analysis of spatial (geographic) data.

CAD GDB

Projections Conversion

Database

Spatial Data
analysis management

3
GISC 6382 Applied GIS UT-Dallas Ron Briggs
What is Geoprocessing used for?

–Data processing/ data automation: Creation,

Preparation, Conversion, Management
–Data analysis/spatial modeling: Suitability,
Sensitivity, Capacity, Risk

4
GISC 6382 Applied GIS UT-Dallas Ron Briggs
 Accessing geoprocessing tools
• Geoprocessing is carried out using tools stored in ArcToolbox
– A dockable window available in ArcMap, ArcCatalog, ArcGlobe, or ArcScene
• Tools run processes

Opens ArcToolbox

5
GISC 6382 Applied GIS UT-Dallas Ron Briggs
 Inside of ArcToolbox
• Toolbox: Container
for toolsets and tools
Toolbox
– Note: cannot have a
toolbox within a toolbox Toolset
• Toolset: Logical
container of tools
and other toolsets Tools
(i.e. folder)
• Tool: Single
geoprocessing
operation (includes
dialogs, models,
and scripts)
6
GISC 6382 Applied GIS UT-Dallas Ron Briggs
 Supported data types
• The tools work with many data types
Coverage Geodatabase

Shapefile Tables

Layer file
CAD

Raster Behavior

Geoprocessing

7
GISC 6382 Applied GIS UT-Dallas Ron Briggs
 The tools work with many data types
…Caution
• Often there are different tools for different data types. Often
there is one:
– For shapefiles and geodatabse features classes
– Another for coverages
– Another for raster datasets
• And they have the same name!
– For example, there is a separate Clip tool for each of these input
data types
– Similarly, there is a separate Define tool for each of these input
data types
• Occasionally, there is a capability only available for
coverages
– e.g. conversion of SDTS (Spatial Data Transfer Format) or VPF (a
vector format used by military)
– Convert first to coverage, then convert coverage to shapefile
8
GISC 6382 Applied GIS UT-Dallas Ron Briggs
 Tools with the same name
• Some tools have the same name
– Must distinguish between these tools in scripts
– Be sure to get the correct tool for the data type you are processing

Clips a
coverage

Clips a
feature
class

Clips a
raster
9
GISC 6382 Applied GIS UT-Dallas Ron Briggs
 A special note on the
Coverages Tools toolbox
• Only available if you have ArcInfo
Workstation installed
• The toolset titles (Analysis,
Conversion, Data Management)
duplicate many of the toolboxes in
ArcToolbox as a whole
– You need to use these tools to process
coverages
– In some cases, there are capabilities
here not available for other data types
• e.g. conversion of SDTS (Spatial Data
Transfer Format) or VPF (a vector format
used by military)
• Convert first to coverage, then convert
coverage to shapefile

10
GISC 6382 Applied GIS UT-Dallas Ron Briggs
 System (Esri provided) toolboxes
• All tools are organized
into toolboxes
• System toolboxes
are provided by ESRI.
Number depends on
–License level (ArcVIEW,
ArcEditor, ArcInfo)
–Extensions licensed
• and you can create
your own toolboxes
(and tools)GISC 6382 Applied GIS UT-Dallas Ron Briggs
11
Tool Counts—nearly 750 now (9.2)
• Counts vary depending on license/extensions
– ArcView: 102 tools
– ArcEditor: 104 tools
– ArcInfo: 251 tools
– Spatial Analyst extension: 158 tools
– 3D Analyst extension: 45 tools
– Geostatistical Analyst extension: 1 tool
• A total of 455 tools came with 9.0
– Feature class tools – 120
– Spatial Analyst tools – 185
– Geocoding tools – 5
– 3D Analyst tools– 20 Counts as of 9.0.
– Linear Referencing–10 9.1 has an additional 171 tools
9.2 has additional 121 tools
– Coverage tools – 57
12
GISC 6382 Applied GIS UT-Dallas Ron Briggs
 The geoprocessing framework:
four ways to run tools 2

Command line

1 3
Dialog Models

4
Scripts

4 13
ArcObjects
GISC 6382 Applied GIS UT-Dallas Ron Briggs
 Selecting the way to run tools:
Choosing between the four ways
Selection depends on a user’s needs and abilities
1. As a dialog inside of any ArcGIS application
• Simple, one at a time processing (GIS for dummies?)
• Good for ad hoc processing
2. As a command from a command line
• Very efficient for command line afficianados
• Supports intelligent auto-completion of commands
3. As a process in a model (Modelbuilder)
• An “executable flow chart” for GIS processing
• Powerful multi-step processing, yet simple to create and use
4. As a function call in a script or program
• Similar to AML in ArcInfo 7, but avoids learning proprietary
language
• provides decision making and looping to support batch processing
In this course we will use #1 and #3
14
GISC 6382 Applied GIS UT-Dallas Ron Briggs
 Parameters
• all tools have one or more parameters
– pieces of information that the tool needs in order
to run
• some parameters are required,
– such as the name of the input data set
• some parameters are optional
• most parameters have a default set, which
– you should always review, and
– change if it’s inappropriate for your application

15
GISC 6382 Applied GIS UT-Dallas Ron Briggs
 Environment settings
• “Customizable defaults”
• Common parameters that are applied to all
tools within a geoprocessing session
• Settings work (“persist”) in all
environments (dialogs, command line,
models, scripts,)
General settings
Current workspace
Output coordinate system

Geodatabase settings Raster settings

XY Domain Cell size
M and Z Domains Mask
Geodatabase raster
Coverage environment
Statistics
Comparison between prj files
Compression
Precision for new coverages 16
GISC 6382 Applied GIS UT-Dallas Ron Briggs
 Four levels of environment settings
• User Profile
– The settings will persist throughout
all applications loaded by the user
• Application Level
– The settings will apply to all tools
executed from that application
• For a specific model or script
– Settings apply to all tools within this
specific model or script.
• For an individual tool
– Settings apply to a specific tool in a
model or script

Lower order (e.g. individual tool) settings

override higher order (e.g. user profile) settings
17
GISC 6382 Applied GIS UT-Dallas Ron Briggs
 Using Modelbuilder

ArcGIS 9.2 has some major

enhancements to Modelbuilder:
Whats_New_in_Arctoolbox.pdf

18
GISC 6382 Applied GIS UT-Dallas Ron Briggs
 Why Use Modelbuilder?
• document the processing steps (workflow) that
you carried out
– You will forget
– You may need to do it again
• re-run the analysis to experiment with different
parameters to see how results are affected
• rerun the analysis if you find a mistake
• carry out repetitive, multi-step tasks
• provide other users (e.g. operations staff) with
an exact workflow that they can replicate

19
GISC 6382 Applied GIS UT-Dallas Ron Briggs
 The Elements of ModelBuilder
• Model diagram window
– Input variable
– Tool
– Derived data variable
• Toolbar
– Add Data or Tools
– Layout
– Zoom and Pan
– Add connection
– Run

20
GISC 6382 Applied GIS UT-Dallas Ron Briggs
 Building a model: constructing
• Create a new model
• Drag tool into model
• Drag data into
model or onto a tool
• Link data and tool

21
GISC 6382 Applied GIS UT-Dallas Ron Briggs
 Building a model: running and documenting
• Save and rename
the model
• Run model
• Document the
model

22
GISC 6382 Applied GIS UT-Dallas Ron Briggs
 Setting model parameters

Or double click on the tool’s process

box to open its parameter dialog
23
GISC 6382 Applied GIS UT-Dallas Ron Briggs
 Element States
• Not ready to run
(parameters not set)

• Ready to run

• Has been run

(note the grey
shadow)

24
GISC 6382 Applied GIS UT-Dallas Ron Briggs
An example,
which we will
now build

25
GISC 6382 Applied GIS UT-Dallas Ron Briggs
 Alternatives to Modelbuilder:
Options for Developing Additional Capabilities
• Modelbuilder
– Multi-step, sequential processing of ArcTools, using a visual
development environment which comes as part of ArcGIS
• Python, Jscript and VBScript
– standardized and relatively simple scripting languages for repetitive
processing, including loops and decision trees, using ArcTools
– Python scripts can be generated from Modelbuilder
• Visual Basic for Applications
– Permits writing of VB macros for sophisticated customization and
development within standard ArcGIS (ArcMap/ArcCatalog) interface
(and thus requires license for these)
– may incorporate ArcObjects, the COM compliant software objects
out of which ArcGIS is constructed.
• ArcGIS Engine
– Set of embeddable GIS components (ArcObjects software objects)
for use in building custom applications, independent of ArcGIS
interface
– Runs under Windows, Unix and Linux, with support for Java, C++,
COM and .NET 26
GISC 6382 Applied GIS UT-Dallas Ron Briggs
 Appendix

ESRI GIS Software

27
GISC 6382 Applied GIS UT-Dallas Ron Briggs
 Environmental Systems Research Institute, Inc.
(ESRI) Redlands, CA
• Privately held by Jack & Laura Dangermond
• One of the 50 largest software companies in the world
• Leader in GIS with at least 1/3rd of market
• 1 million users (2002) and 2,600 employees
• Originator of commercial GIS in 1981 with release of ArcInfo
• Released their first GUI (graphics user interface) product, ArcView, in
1991 using proprietary Avenue programming language (for MS
Windows, UNIX, Macintosh)
• These two product lines (ArcView 3 and ArcInfo 7) combined together
in ArcGIS v.8 released in 2000
– complete rewrite based on Microsoft COM/Active X software objects
• Only run on MS Windows
– completely different interface from earlier ArcView and ArcInfo
– old, Avenue based, ArcView still available as ArcView 3.3,
• But Old Avenue scripts will not run in ArcGIS 8
– old command line ArcInfo 7 still available as ArcGIS Workstation
• Old AML scripts will run in ArcGIS 9
• ArcGIS 9 released in 2004
• Current 2008 release is ArcGIS 9.2 28
GISC 6382 Applied GIS UT-Dallas Ron Briggs
 ESRI Product Line-up: ArcGIS client products
ArcReader (“adobe acrobat” for maps) & ArcExplorer (spatial data viewer)
– Free viewers for geographic data.
ArcGIS 9.x Desktop: two primary modules (MS NT/2000/XP only)
1. ArcMap: for data display, map production, spatial analysis, data editing
2. ArcCatalog: for data management and preview
ArcToolbox, for specialized data conversions and analyses, available as a window in both
Available capabilities within these modules are “tiered”
• ArcView: viewing, map production, spatial analysis, basic editing
• ArcEditor: ArcView, plus specialized editing
• ArcInfo: ArcView & ArcEditor plus special analyses and conversions
Extensions: for special apps.: Spatial Analyst, 3D Analyst, Geostatistics, Business Analyst,
Network Analyst.
ArcObjects: build specialized capabilities within ArcMap or ArcCatalog using VB for Applications
ArcGIS Workstation (for UNIX and MS NT/2000/XP)
– the old command line ArcInfo 7.1
ArcGIS Engine (MS NT/2000/XP)
– Set of embeddable GIS components (ArcObjects software objects) for use in building custom
applications
– Runs under Windows, Unix and Linux, with support for Java, C++, COM and .NET
– Replaces MapObjects which were based upon a previous generation of GIS objects
Notes:
ArcGIS 8 released 2000 to integrate two previous standalone products: ArcView and ArcInfo
ArcGIS 9 released 2004 providing the full capability that should have been in ArcGIS 8!!!
--full support for all data types (coverages, shapefiles, geodatabases)
--full support for all previous geoprocessing analyses
--Modelbuilder for scripting and repetitive processing
--ArcEngine for building custom applications
ArcView 3.3 (the predecessor to ArcGIS 8.x) the only GUI option for UNIX.
 ESRI Product Line-up: ArcGIS server products (Spring 2007)

ArcGIS Server
– Permits the creation of server-based specialized GIS applications
– Provides full range of GIS capabilities to a user without a desktop GIS
ArcIMS (Internet Map Server)
– Software to develop Internet server-based mapping and basic analysis
– Provides maps and simple query to the user without a desktop GIS
SDE (Spatial Database Engine)
– middleware to support spatial data storage in standard DBMS
– Supports all major industry databases:
• Oracle, SQL-Server, IBM DB2, Ingres
Note: SDE and ArcIMS now distributed as part of ArcGIS Server package as of 9.2

ArcGIS Services
– Server based applications built and operated by ESRI or its partners
and made available on the Internet for subscription
– Normally charged on a “per transaction” basis, but can be flat fee
– presumably built using ArcGIS Server

1/13/2006 GISC 6382 Applied GIS UT-Dallas Ron Briggs

 Other ESRI Products:
• ArcPAD
– Mapping on PDAs (“handhelds”) with Windows CE operating system
• ArcLogistics Route
– Specialized business application for delivery routing
• ArcFM
– water and telecom: industry specific facilities management
• ArcGIS Extensions
– Spatial Analyst: raster data analysis
– 3D Analyst: 3-dimensional data display
– Geostatistics: surface analysis
– Business Analyst: marketing and site selection
– Survey Analyst: update of ArcInfo COGO (coordinate geometry) module
– Network Analyst: network routing algorithims; shortest path, etc.
– Maplex: automated, high quality labeling for maps (now part of ArcGIS 9)
– Publisher: creates .MXP maps for reading with ArcReader
Extensions work irrespective of ArcView/ArcEditor/ArcInfo tier
• BusinessMap:
– $99 standalone business mapping (originally Richardson-based MapLynx)
31
GISC 6382 Applied GIS UT-Dallas Ron Briggs
 ArcGIS System
Consistent interface
c:\ ArcGIS Workstation Clients Increasing capability

ArcInfo ArcEditor ArcView $ ArcExplorer Browser

ArcMap ArcMap ArcMap
ArcCatalog ArcCatalog ArcCatalog
ArcToolbox ArcToolbox ArcToolbox

ArcEngine/ ArcServer Services

ArcObjects ArcIMS Services
Application
Development & ArcSDE Services
Customization

ArcPad
Internet
Databases
Files Multi-user Geodatabases
(Personal Geodatabase,
Shapefiles, Coverages, (in Oracle, SQL Server, Source: ESRI with mods.
Handheld/Wireless Grids, tins, etc) IBM DBII, etc)
Some additional detail…

33
GISC 6382 Applied GIS UT-Dallas Ron Briggs
 Discontinued Products
• ArcCAD
– CAD product from ESRI
• PC ArcInfo
– 1st effort at PC based GIS
– DOS based, command-line driven
– Data not compatible with ArcInfo UNIX
– Replaced by ArcInfo 8 and ArcView 3.2
• DAK (Data Automation Kit)
– Subset of PC ArcInfo for data preparation for ArcView
3.2
• Atlas/GIS
– once a leader in PC-based mapping
– Bought by ESRI in 1996 & discontinued in 2001
34
GISC 6382 Applied GIS UT-Dallas Ron Briggs
 ArcGIS Version 8/9
With Version 8 & 9, now have two flavors:
Desktop:
– Largest Microsoft COM/ActiveX application to date
– Full GUI interface
– Customization via Visual Basic for Applications
– New data base concepts: Geodatabase
– Runs on XP/2000/NT only
– no UNIX version available
Workstation:
– classic, command-line ArcInfo with AMLs (Arc Macro Language) for
customization
– same as version 7 and earlier, with minor enhancements
– the only option for UNIX, but also available on MS XP
– With release of ArcGIS 9, little reason to use unless under UNIX
• Now, all capabilities have been moved to ArcGIS

35
GISC 6382 Applied GIS UT-Dallas Ron Briggs
 ArcGIS Desktop Primary Characteristics
• GUI-based tools
– ArcCatalog, ArcMap, ArcToolbox
• Geodatabase spatial data model supported in
standard database environments
– MS Access (.msb) for personal applications
– File-based geodatabase with 9.2
– Any industry db via SDE for multi-user applications
• Modeling of real world as intelligent objects
– Houses, poles, not points, lines, polygons
• COM/ActiveX components (ArcEngine) for
embedding geography in other applications
ArcInfo7: simple data complex applications
ArcInfo8/9: intelligent data simpler applications
36
GISC 6382 Applied GIS UT-Dallas Ron Briggs
 ArcGIS 9 Desktop Modules
ArcCatalog (schema editor, with VISIO generation)
• The base application for ArcInfo Desktop
• Windows Explorer-like interface
• for organizing access to data and metadata
• For launching other Desktop apps: MAP and TOOLBOX
ArcMap (object editor)
• GUI for map creation and spatial data editing
• ArcPlot/ArcEdit (from ArcInfo v. 7) & ArcView 3.2
View/Layout combined
• Map projections on the fly (not via conversion as in AV)
ArcToolbox (geoprocessor)
• An interface to geoprocessing tools
• In ArcGIS 8 it was a separate module
– In ArcGIS Release 9 it’s an integrated window in ArcCatalog or
ArcMap 37
GISC 6382 Applied GIS UT-Dallas Ron Briggs
 ArcGIS Desktop Capability Tiers:
Each tier has the same interfaces (ArcCatalog, ArcMap,
and ArcToolbox), but an increasing set of capabilities
are available within them (and $ price rises
accordingly!)
ArcView:
– viewing, map production, spatial analysis, basic editing
ArcEditor:
– ArcView, plus topologic editing of geodatabases and
editing of coverages
ArcInfo:
– ArcView and ArcEditor, plus more geoprocessing
analysis, conversions, and full support for coverages.
– Old, command line ArcInfo including AML support
38
GISC 6382 Applied GIS UT-Dallas Ron Briggs
 ArcGIS 9.0 versus ArcGIS 8.3
• Main differences from 8.x are in ArcToolbox
– ArcToolbox built into ArcCatalog and ArcMap rather
than a separate module
– All ArcToolbox tools support all data types
(geodatabase, shapefiles, coverages)
• 8.3 primarily support coverages
– ModelBuilder diagrammatic modeling tool
• Invaluable for tracking and replicating geoprocessing steps
– New scripting capability for repetitive actions
• Python, JScript and VBScript--simpler to use than VB for
Applications, the only alternative in 8.x
• Old aml (arc macro language from ArcInfo 7) also supported

What ArcGIS 8 should have been when it was first released!

Incorporates just about everything from ArcInfo 7/ArcView 3. 39
GISC 6382 Applied GIS UT-Dallas Ron Briggs
 ArcGIS 9.1 & 9.2 versus ArcGIS 9.0
For Spring 2007 we will use ArcGIS 9.2
Ormsby text contains copy of 9.0
Gorr and Kurland contains copy of 9.1

Main differences in 9.1 from 9.0

• General performance improvement and user enhancements (e.g. pause
drawing)
• Additional geoprocessing tools (63 in ArcView, 86 in ArcEditor, 22 in ArcInfo)
• Improved labeling of features on maps (Big change here was in 9.0)
• Support for linear referencing
• Network Analyst extension for transportation networks (shortest path, etc..)

Main differences in 9.2 from 9.1

• Geodatabase and map documents (.mxd files) incompatible!
• File based geodatabase
– Faster, larger capacity (x 1,000) than MS Access-based personal geodatabase
– Higher precision of coordinate storage
• Identity/info. and measurement tools rewritten and greatly enhanced
• Graphing re-written
– Supports brushing (linking) between graph & map (as in Anselin’s GeoDA) software)
• Supports cartographic representation
– Can re-position features on map without changing coordinate values in geodatabase
 Computing
Evolution
And it will all keep changing!

Internet Pervasive
Computing
Desktop

Workstation • Small Hardware

(Nano)
Mini • Wireless Internet
Source: ESRI, Inc. • Interoperable
Mainframe • Embedded
41
GISC 6382 Applied GIS UT-Dallas Ron Briggs