The changes in question concern the USGS' digital elevation models, or DEMs.DEMs are essential for mapping the shape of the earth's surface and for much environmental analysis.Digital elevations can be used to calculate where water flows, how steep the slopes are, how much sunlight is received at the surface, and many other things.
The USGS has improved its DEMs.The previous versions are now obsolete and increasingly difficult to obtain.This has created a surge in demand for the new DEMs.
Many of the new DEMs are being released on the Web in a new computer format, "SDTS decimal meters." This makes them difficult (for the time being) for many people to process.
In the most controversial move of all, the USGS has contracted with only one commercial vendor, GeoCommunity, to serve the data to the public at http://www.gisdatadepot.com.This vendor has been unable to keep up with the demand for these (and other) datasets.
Directions Magazine conducted download tests over several days at the end of July from sites scattered across the U.S.We experienced consistent DEM download speeds between 2.36 and 2.57 kilobytes per second, regardless of time of day.A typical compressed high-resolution DEM requires fifteen minutes to download at these speeds.Since most analytical work uses several DEMs, these long download times can be significant and costly.
Representatives of the USGS state that the download speed problem was unexpected.Beth Duff, a computer specialist with the USGS in Washington, DC, adds that the USGS is "checking into the [previous download] statistics" on the USGS Web sites in an effort to understand why speeds on the vendor's site appear to be so much slower than speeds previously available on USGS
Responding to Internet discussions of the problems last week, T.C.Herman, Executive Vice President of GeoCommunity, wrote "The GeoCommunity is in the process of addressing the speed problem as we have just contracted for an additional T-1 line....[It's] probably going to be a few weeks before it is installed....A good portion of this [USGS] data isn't available anywhere else on-line."
The USGS elected to outsource the Web delivery of its data "with the goal of making more public domain data available at no charge and involving the community" in the process.It notes that the price of CD delivery of data through GeoCommunity is less than previously charged by the USGS.
One of the concerns users have about this new arrangement is the apparent monopoly position of GeoCommunity.Hylan Beydler, contacted at the Earth Sciences Information Center in Rolla, MO, explained that this is just the first step.The USGS is hoping that this outsourcing program will grow to include downloads at other sites.GeoCommunity was the first to become involved because of its privileged "business partner" status with the USGS.
Duff confirmed this intention and volunteered that the USGS is "definitely" planning on offering the new data on the Geography Network [http://www.geographynetwork.com].This will probably happen later in 2001.Duff adds that there will probably be a charge for data retrieved from the Geography Network, however.
The USGS is not serving the new DEMs itself because of the effort is would take.According to Duff, it was "more efficient to load all new DEMs on the GeoCommunity site." At the time this was done, the GeoCommunity site added another T1 line (high-speed Internet connection).Apparently both the USGS and GeoCommunity expected this to be sufficient bandwidth to support future demand.
Beydler explained that the old DEMs are no longer available from the USGS because they had been subject to two kinds of "positional error." The first is a minor error in recording the location of a DEM's "bounding rectangle." Rounding the coordinates could cause some points within the DEM to lie as much as one meter outside the rectangle.On a dataset with ten-meter resolution this is not a problem, but evidently it could cause some software not to read the data.
The other positional error arose because of "an incorrect UTM origin." A USGS DEM typically covers a small portion of the earth, but the earth's curvature does affect the calculation of coordinates.Slightly different results arise if coordinates are calculated relative to one corner of the DEM instead of another.Using the wrong corner, which may have been done with many DEMs, the elevation data could be off as much as one cell, or ten to thirty meters, depending on the DEM's resolution.
Many DEM users probably would not notice or care about such errors, but some would.The USGS is so concerned about the quality of these data that they undertook to fix the errors in the tens of thousands of DEM data sets.This is the motivation for creating the new DEMs.
At the time the positional errors were being corrected, the USGS noted that the elevations themselves were recorded only to the nearest foot or meter.To increase the precision, a new format-"decimal meters"-was introduced.On digital computers, fractional numbers are represented in a different manner than whole numbers.Thus this apparently minor change necessitated a major overhaul of the underlying data format.
As part of the overhaul, the USGS elected to represent the new DEM data in a relatively new standard format, Spatial Data Transfer Standard, or SDTS.SDTS was designed to contain just about any kind of geographic data.This made it a natural choice for the new DEMs.Unfortunately, most older GIS and mapping software cannot read SDTS data at all, and if it can, it still cannot handle the decimal meter version.
It is evident the USGS made efforts to inform the GIS community of these changes and to provide assistance in migrating to the new format.Dimitri Rotow is the spokesman for Manifold GIS [http://www.manifold.net].Writing on the GISList Internet discussion group, Rotow stated, "USGS has been great about contacting us and letting us know far, far in advance of changes.They told us very clearly what was going on, provided samples, the works.There were absolutely no hassles at all in adjusting things at our end....USGS and SDTS have been super."
In the past, the USGS has written software for viewing DEMs and other kinds of geographic data.When asked about whether there were plans to support the new DEM format with conversion software, the USGS's Beydler explained that the USGS "wants to stay out of the data converter business."
DEMs will soon not be the only publicly available elevation data covering the U.S.The USGS is working on the National Elevation Dataset (NED) [http://edcnts12.cr.usgs.gov/ned/].These data are seamlessly matched, smoothed, and filtered (to remove data artifacts introduced by old processing methods), says Duff.They can be accessed by location, so users will obtain in one dataset exactly the information they need, rather than having to collect it in DEM quarter-quads (rectangles 3.75 minutes on a side) and mosaicing the pieces together.Some NED data, which reach 30 meter resolution, are available now.
For more information on SDTS DEM positional error please see http://edcwww.cr.usgs.gov/doc/edchome/ndcdb/geowarning.html
The USGS GeoData download site is at http://edcwww.cr.usgs.gov/doc/edchome/ndcdb/ndcdb.html
An excellent place to begin searching for GIS data on the Internet is the Center for Advanced Spatial Technologies' "Starting the Hunt" page at http://www.cast.uark.edu/local/hunt/
As a reader service, Directions Magazine provides searchable and chronological access to all major GIS-related Internet discussion list messages.You can review the full discussion about the USGS DEMs by visiting our 'Discussion' page.Look at MapInfo-L and GISList.(GISList is hosted by GeoCommunity.) Most of the messages appeared between 25 and 27 July 2001.Their subject lines included 'DEM Ransom', 'EROS -- SDTS -- DEM', and 'Need help with DEM/SDTS conversion'.
What do you think of this map data controversy? Write and let us know.
Letters to the Editor
- 8/10/01 - Good idea...poor execution
- 8/9/01 - Paying for the data twice
- 8/2/01 - This was an inside deal
- 8/1/01 - Concerned about single data serving format and vendor