February 16, 2007
There's been a lot of coverage of Google's recent announcement via a blog of a KML search capability from Google Earth and Google Search. Michael Jones, Google's Chief Technologist for Google Earth, Maps, Local answered some questions to clarify what it does, how it works and explored some of its implications for searching for geodata.
DM:Are all publicly accessible KML files on the Web indexed by Google? Do their creators have to do something for them to be in the index?
MJ: Every KML & KMZ file on the web that is found by the Google web crawl is noted and indexed. The crawl honors include/exclude guidance from robots.txt files and is educated by site maps to find content that would otherwise be difficult to locate. Every resulting KML & KMZ file found by the crawl is indexed by its name, location, and by the contents of the KML description. Through KML Search, all of these files are now searched by the text string entered in the Google Earth search box.
Creators need only place their KML/KMZ on a publicly accessible web site and their geospatial data will be universally discoverable.
People and program agents can also search directly using Google Web Search. For example, visit www.google.com and try the following search:
filetype:kmz adena
This will show you all seven (do not suppress duplicates) of the KMZ files containing 'adena' in their descriptions. ;-)
DM: Does the search have a geographic part and a text part? How do those work? Based on where you are in GE? Based on text in KML?
MJ: We show the 'best' result subset of all the results. The details are subtle, but the idea is that the list of textual matches is also scored geospatially to produce a conflated score representing a good match. A perfect text match right where you are looking is a perfect score, a great match nearby or a so-so match on screen would be next, followed by great matches far away and poor matches on-screen. Then the best 'N' of these are selected and presented as the results in such a way that the Google Earth client zooms in/over/out to encompass the set of selected results. Users can explore these or follow the provided "more..." link to get more results, which is just like going to page 2, 3, and subsequent pages in Google Web search results.
DM: Might this be a way for all geo data to be found – both for advertising needs and for the sort of geodata search folks might currently do at GOS, etc? I'm thinking a small bit of KML in a page could make it geosearchable in a way "local searches" are not today.
Could this be the answer to the old .geo idea?
MJ: yes, Yes, YES!
You are right on target with the "small bit of KML" comment.
[Pre-KML-Search]
If you want your county's fire plug Shape file to be findable on the WEB OF PAGES, you would have made an HTML reference page and decorated that with text that made searchers notice it when traversing your website, text that made it findable by web search tools like www.google.com, and added a hyperlink on the page referencing the Shape-file collection.
[Post-KML-Search]
Now, you have an additional choice. If you want your county's fire plug Shape file to be findable on the WEB OF PLACES (using an Earth browser such as Google Earth), then you make a KML reference placemark and load it's description with text so that searchers notice it when looking at the placemark (even when part of a collection), find it when using tools like Google Earth Search (aka KML Search), and you'd add a hyperlink in the description of the placemark that references the Shape-file collection.
This simple step of creating a KML placemark (and waiting for the next web crawl) is all you need to let every one of the 200+ million users of Google Earth who flies nearby and types "fire plug" into the search box find your KML and be presented with the hyperlink to the Shape file (and by extension, MapInfo TAB files, Autodesk formats, NITFs, etc., all based on desired audience.)
Note that it is the author's option to also convert the referenced data into KML too. They would do this if their goal is to have those who browse, search, and explore the planet using Google Earth see the results (such as the fire plug locations) right there in Google Earth. This is an option, but is separate from using what you correctly describe as a small bit of KML to make the original data discoverable. This is the application of the world's most popular search technique to the task of finding data on a geospatial, view- based basis – addressing in many ways the goals of GOS and SDI efforts both past and present.
DM: How does standard geo metadata play into such a search? I'm thinking not at all now, but maybe in the future?
MJ: Everything in the KML is indexed. If the metadata are placed into the KML description, then they are searchable. However, this is not a smart search in the sense of "select fire plugs painted more than 6 years ago", so there is much more to be done in this area. You’ll note that Google started out indexing page-describing HTML, and then moved to index other popular document formats such as PDF and Word’s “.DOC”; likewise, we’re indexing place-describing KML and may later understand a larger collection of geospatial formats. If so, we’ll be in a better position to deal structurally with important metadata at that time.
DM: So this is part of Google larger search vision?
MJ: When I present a slide with the web browser on one side and Google Earth and Maps on the other, and say "everything you can do on the web of pages you will be able to do on the web of places (via a browser such as Google Maps or Google Earth)", the launch of KML Search is what has been on my mind as the most significant move in that direction.
The Google Earth and Maps teams work to geolocate all information and help users find that information geospatially. While users need both halves, the finding part is a core Google skill and one that is very useful even when what is found is not hosted at Google, as is famously the case with Google Web Search. The launch of Google KML Search initiates this Google Earth Search capability for all of the world's spatially organizable data.
|
Your Comments Post a comment All comments provided in this section are those of the individual who has created the post. These are not the opinions of Directions Media, its editors, staff or owners unless otherwise noted. Directions Media retains the right to edit or delete any comments posted herein.
|
|
||||||
| Please please please please please let's get ESRI and ADSK and INGR and MAPS and OGC and the rest of the world to plug in an automatic KML metatag option on any file created in their product suites. Maybe, just maybe, the geospatial community might end up with a discovery capability that actually works in the real world. |
||||||
|
||||||
| OGC doesn't make software, just standards and sometimes, reference implementations. I don't think there's a role for that organization here, but I could be wrong. Archie is correct, though. If you want this sort of tool in your software, tell you vendor/developer now! [For those who are concerned: I no longer consult to OGC.] |
||||||
|
||||||
| IMO: I think that it is about time that Google started to play _with_ the rest of the spatial team and not against it. While Google have done a lot of good for the spatial industry by raising its profile and hosting a great set of tools in GoogleEarth and GoogleMaps, they are not _the_ spatial industry. We have seen a couple of instances recently where they appear to be deliberately working against the spatial industry and long term efforts to make systems interoperable and spatial data (sd) easily usable regardless of tool used. Some examples are: this announcement; poor support for spatial data served via WMS etc in GE; pushing for KML at the expense of existing standards (i.e. GML). For over fifteen years various organisations around the world have been working together to develop Spatial Data Infrastructures to put some consistancy in sd management and to help people to describe and find sd that is relevent to their work. Some examples of this work are: Global, GSDI ( http://www.gsdi.org ); US, FGDC; Australia, ASDI; Canada, GGDI; Europe, INSPIRE. OGC and ISO are also significant players in this in developing relevant spatial standards. One aspect of this work relevant to this announcement is a set of integrated search engines/tools that allow a user to discover spatial data that may be relevent to their work. An example is the Australian Spatial Data Directory ( http://asdd.ga.gov.au/ ). A search on this site for sd will search all registered jurisdictions in Australia. Other countries are doing similar work. The search tools will return spatial metadata that will allow a user to determine if the sd is fit for purpose prior to getting the data. This is a key component that is missing from Google's announcement. Rather than Google continuing to try and split the spatial industry with disruptive technology, I'd prefer to see them working with us to develop and fine tune existing work and standards. Google, why not include good functional support for OGC standards such as WMS and also SDI search engines in GE. This will potentially make a wealth of data available to your users without causing spatial data managers to maintain duplicate sets of sd (that may change daily) in an often inferior data format (KML/KMZ). You do not have to go it alone. Bruce |
||||||
|
||||||
| I'd have to disagree with Bruce's perspective. I think it's plain that Google has demonstrated that it "knows search", and probably better than the far from homogenous "spatial team". Google Maps/Earth and its 200 million users have done a great service in advancing the recognition of spatial data issues. The mechanism of plugging geospatial query into the Google infrastructure (both query engine and robust viewing platform) simply by including "small bits of KML" seems like a reasonable cost for a huge benefit. Give credit where it is due, while many of us on the "spatial team" have been talking about and planning for national/regional spatial data infrastructures for some time, Google has leveraged their ability to "fund a vision" and has made fantastic inroads on ~building~ a *global* spatial data infrastructure (that we can plug into, if we want). At this juncture, I think it's clear they are part of the "spatial team". |
||||||
|
||||||
| Bruce, Michael, I guess the middle ground between Google desruptive innovation techniques and existing extent of spatial standards and mark-ups would be more of a Semantic Web soution, when simple geospatial ontology will be published and adopted and RDFizers e.g. made available for publishing the metadata from proprietary sources, so that every owner of spatial metadata can easily publish his content and expect it to be crawled, found and displayed within a Web application context or context of a popular mash-up based on GE/GM e.g. And this moment is not too far away, I believe. |
||||||
|
||||||
| Imo: Bruce Bannerman has hit the nail on the head so to speak. Whilst google's innovation is great it should be harnessed to comply with the geospatial community as a whole. This may slow innovation which might seem counterintuitive, but the GIS community is already plentiful in ad-hoc disparate applications of geospatial technology that until recently (past 7-10 years with ISO, FGDC, CEN, OGC) had no real regard for interoperability except within its own company or organisation for example. I believe GIS is on the verge of a new golden age where its application as a science is realised and not its application as a tool only. In order for this age to be realised the geospatial community must have a coordinated spatial framework to work from. This is where google’s innovation might prove divisive as it has such a huge userbase already, especially in private and commercial user areas. It may offer rewards but this will only be in the short term and as Bruce points out GML is potentially much more powerful. Vasily's compromise could be a way forward however, any solution must be dynamic enough to support the ever growing complexity of geospatial data. Furthermore, it would perhaps be interesting to look at google’s approach from a legal side. Will people be able to use the KML script for their own company services free of charge? For example. |
||||||
|
||||||
| Does anyone have any ideas how to get the FGDC metadata to KML? |
||||||
