ArcGIS 9 and the Geodatabase

Why this is important is that as data set become larger and accessible by multiple users, without the proper practices, performance is reduced, the data loses standards and interoperability may be lost.

In ArcGIS 9 there were several overall goals for improvement.These were outlined by James Neild and Gudmundur Hafberg (ESRI), in their presentation at the ESRI Users' Conference.Those improvements are outlined here along with ESRI's ten steps in geodatabase design.

Goals

• Make ArcSDE significantly faster - (faster queries, raster enhancements)
• Reduce the server load
• Reduce the network traffic
• Reduce the number of SQL statements (an example was that in one application, they were able to reduce the number of SQL queries from around 4,000 to 279).
• Improve the application server and direct connection performance

• ArcGIS 9 supports four databases, DB2, Informix, Oracle 8i to 10G and SQL Server.O/S support is for Window, Unix, Linux and Solaris.
• They now support raster formats L277 and JPEG 2000.
• Pyramids now allow for partial creation and will build pyramids only for the area loaded/unloaded. *(See definition at end of this document).
• Log files - database log file in previous versions became very large and difficult to manage.This area has been improved so that SDE now owns a pool of log files.
• Support for OGC's "well known binary format" (WKB), WKB geometries, 2D simple features and 3D.
• Better database compression and related rules for compression
• ArcSDE now has an API trace tool for developers - traces API calls
• XML support as a data type
• A "check schema" tool to insure consistency in Metadata (Windows only)

Specific Database Improvements

DB2

• Operating system group support
• Multi-versioning view
• Added additional O/S support for Linux Enterprise Server, Solaris & AIX

• Multi-versioning view
• Improved versioning model
• Faster
• Support to version 9.4 UC3

• Performance in server speed (versions less than 10)
• Oracle 10G support (does not take advantage of 10G spatial functions, and they treat 10G the same as version 9.0)
• Added a "Check Feature command" to validate spatial data (validates the Geometry)

• Single spatial database model
• Windows Group support
• Increased performance
• Increased security
• Metadata is now in a single SQL Server database (a single SDE database holds all the metadata, for all objects regardless of actual database location)
• Users can connect to one database and access all others if they have permission
• Simplified administration

• XML support for Geodatabases
• For import/export
• To allow schema and data interchanges
• Can use XML to export changes in the geodatabase
• Can use XML to export geodatabase features
• Interoperability Connector
• Safe Software's FME tools for connection to disparate data without conversion
• Geodatabase raster support
• Supports raster catalogs
• Raster datasets
• Raster as an attribute column
• Raster in the enterprise and personal geodatabases

Concepts in Geodatabase Design
Database design does not usually share the glamor of 3D imagery or watching a feature layer drape over a topographical representation.However, with large datasets the database design is similar to the concept of having a good foundation when erecting a new building.

This function requires significant thought and understanding of the data structures you will be using.The ten steps below are taken from a new (August 2004) ESRI publication Designing Geodatabases.This book was written by David Arctur and Michael Zeiler (ESRI), and their ten steps for design are presented here to give you an appreciation of not only the design of geodatabases, but also how the above improvement in ArcGIS 9 Geodatabases can affect the design and implementation of geodatabases.

See also...www.support.esri.com/datamodels also http://arcscripts.esri.com/

[ Color Key: Light Blue = Conceptual Design, Orange = Logical Design, Red = Physical Design]

1. Identify the information products that will be produced with your GIS -- Inventory map products, analytical modest, database reports, Web access, data flows and enterprise requirements
2. Identify the key thematic layers based on your information requirements -- Specify the map use, data source, spatial representations, map scale and accuracy, and symbology and annotation.
3. Specify the scale ranges and spatial representations for each thematic layer -- GIS data is compiled for specific scale use; feature representation often changes between point, lines and polygons at larger scales.Rasters are sampled to include multi-resolution pyramids.* (See pyramid definition)
4. Group representations into datasets -- Discrete features are modeled with feature datasets, feature classes, relationship classes, rules and domains.Continuous data is modeled with raster datasets.Measurement data is modeled with survey datasets.Surface data is modeled with raster and feature datasets.
5. Define the tabular database structure and behavior for descriptive attributes -- Identify attribute fields, specify valid values and ranges, apply subtypes to control behavior, and model relationships.
6. Define the spatial properties of your datasets --Use networks for connected systems of features and topologies to enforce spatial integrity and shared geometry.
7. Propose a geodatabase design -- Make informed decision on applying structural elements of the geodatabase and prepare a design.Study existing designs for examples.
8. Implement, prototype, review and refine your design -- From the initial design, build a geodatabase and load data. Test and refine your designs.
9. Design work flows for building and maintaining each layer -- Each layer has distinct data sources, accuracy, currency, metadata and access.Define workflows to conform to your agency's business practices.
10. Document your design using appropriate methods -- Use drawings, layer diagrams, schema diagrams and reports to communicate your data model.

Summary
The concepts of geodatabase design is complex and requires a substantial amount of work to tie GIS products, middle ware and relational databases together.Today, because of the interaction of government, free enterprise and GIS vendors to provide multi-platform solutions and commonality in data access, these designs become even more important.Web services, mobile computer and a whole host of innovations also affect data access.The data foundation that is built, perhaps while not as glitzy as 3D, holographic geographic displays and fly throughs, is really the means that allows those things to be useful.

* Definition - Raster Pyramids (Source: From the ESRI Web Site)

When a raster has to be represented in a series of reduced/increased resolutions, a "pyramid" is built for that particular raster.A pyramid is a series of reduced resolution representations of the dataset, mainly used to improve the display performance of rasters when one is not working with the pixel information at full resolution.It contains a number of layers, each re-sampled at a more generalized level.Thus each level of the pyramid is a re-sampled representation of the raster at a coarser spatial resolution.