Author's Note: This article explores a new
technology, one with potential uses for geospatial professionals. At
this time it's not fully geo-enabled. We feel it's our responsibility
to highlight emerging technologies that will touch our community. We
welcome your feedback.
My name is Adena and I'm not a programmer…
Most of us in the geospatial arena fall somewhere between two extremes; we are neither "programmers" nor "non-programmers." For those who fall closer to the "non-programmer" side, whenever a software vendor says, "With some customization you can add features you need," we squirm. Can we do that customization on our own? Do we need to call in a coder? Will we need to learn something new? Yikes!
Vendors have realized that a great many of their users are not programmers, but experienced users who could be even better (and perhaps more loyal, meaning more software sales) with some ability to customize programs and build their own tools. That vision, thankfully, has driven "more accessible" customization environments and the use of visual tools for building models and small programs.
Not long ago Yahoo! stepped into this space with a Web 2.0-type tools set that allows customization and manipulation of "feeds" without programming. It's called Pipes. Here's how Yahoo! describes Pipes: "Pipes is an interactive feed aggregator and manipulator. Using Pipes, you can create feeds that are more powerful, useful and relevant." If the term "feed" is still a bit fuzzy in your mind, try this: insert the term "Web data" instead of "feed" in the two sentences above. Now, that sounds valuable, doesn't it?
Pipes is in beta and it's not yet fully geo-enabled, though those requests were at the top of the list for enhancements. Sources note: (1) "Pipes will output GeoRSS. They plan to release a map viewer soon." (2) "Yahoo will open up access to the Pipes engine to programmers and add support for the KML data source, which is used to display geographic data in Google’s popular Google Earth mapping application and Google Maps website." These tidbits, among others, suggest that this environment has significant potential. Furthermore, this is another case where "doing some homework" will likely get you ahead either in your life (creating a pipe that does something personal for you!) or on the job (creating a pipe that speeds up, simplifies or automates a task). And, no, you don't have to be a programmer… honest.
Finding, Running and Looking at a Pipe
I'm a big manual reader, but, alas, the "official" tutorial for Pipes is not yet available, so I had to bumble around a bit. The good news: I didn't bumble long at all. (There are some intro articles by non-Yahoos listed on the Pipes blog.)
As with many tech endeavors, before jumping into building my own pipe, I looked at other people’s pipes. In fact, on the main page you can search for pipes that address topics of interest. I chose to search on "GIS." (Figure 1) I got back three pages of pipes!
I chose to explore "Free Software GIS News" (Figure 2) since I know MarkusN (he's the GRASS guy). His pipe, I learn by clicking on it, "fetches the latest Free Software GIS news for you!" Clicking on "Run this pipe" runs the program and does indeed provide the latest Free Software GIS news (Figure 3). What can you do with that information? Well, you could come to the pipe’s website and run this pipe now and again. But you can also, using the Subscribe pulldown, subscribe to a feed with an RSS reader, get the results via e-mail each day, or a few other things. You can also use the output of this pipe (the list of news items) as the input for another pipe.
Now, Marcus is a programmer, but this pipe is not complex at all. How do I know? I clicked on the View/Edit link near “How This Pipe Was Made.” That brings up the pipe editor development environment. (Since Pipes is a Yahoo! tool, to access the editor or create your own pipe you'll need a free Yahoo! user ID. You may have one if you use Yahoo! Mail or Groups.) Marcus' pipe has a bunch of boxes hooked together with octopus-like links. (Figure 4) All pipes look like that. Some have more boxes and connections, some have fewer. This one is simple, but has many inputs. Let's start with something simpler.
Creating a Pipe
MyFirstPipe is much simpler than Marcus' attempt. It has two inputs that get tossed into a "source" (which is basically a place to "get information," like Flickr or in this case Yahoo! Maps) to create the output. The inputs are a location and a text value. The output is a list of results that fall near the place related to the text (Figure 5). It's basically a local search, one you could already do via Yahoo! Maps.
Now, let's go inside the editing environment and see what I did to build the pipe (Figure 6). First, I decided what my goal was: to have users input where and what they were interested in and have the pipe output results. I had options in the input for what prompt was issued. I used "Where?" and "What?" I could include a default, which I did for both items, so if the user didn't input anything, they'd get a search on "kite" around "Boston." The most fun part of building the pipe? Linking up the components! You click on the "dot" of an input form and drag it to the "dot" that it will "feed" in the "source." So I attached the "Location Input" box to the "Location" field in the Yahoo! Local source. Then I took the "Text Input" and linked it to the "Find" box in the source. The result of this operation is fed into the “Pipe Output.”
Before we leave the edit environment and run the pipe, let's take a look at some of the options in each category. Sources are slim just now and include many Google properties like Yahoo! Local and Flickr, but also GoogleBase. You can also use "Fetch" to search any RSS feed you like (for example, one from eBay or our All Points Blog). User inputs are what someone would key in. They come in many flavors (like database elements or cell types in Excel): text, number, date, location, etc. I like the fact that that there's location and I imagine that'll be more useful to geospatial professionals in time. The list of Operators parallels those you might know from a database or Excel. There are tools to sort, filter and merge data, for example. And, you can even translate data to another language!
Running the Pipe and Beyond
Now, let's get down to brass tacks: running the pipe and thinking about what to do with the output (Figure 7). I chose not to key in any text and took the defaults (kite and Boston). The result was six items (Figure 8). Clicking on one, Danehy Park, fires up Yahoo! Maps. It shows the location of interest, noting in the text that a kite festival is held there (Figure 9). The next step is to take the output of a pipe and use it. I could, for example, put my pipe output as RSS on my Web page (Figure 10).
This is a simple example which shows how easy it is to play with pipes. Yahoo! promises more sources and operators in time. Pipe authors can publish pipes for others to use, like Marcus did. You'll want to check out the published ones (search on "map" for some of interest), some of which are very clever in their own right and others which power map mashups elsewhere. The real power for us in geospatial will come as Pipes expands to support more sources and more operators. One of the top requests is further integration with Yahoo! Maps, so that the output is a map. By the way, if you feel like you could do something like what I did in my example, consider yourself a step closer to the "programmer" end of the continuum!
Further Pipes Resources for the Geospatial Community
Playing the Pipes
Mikel Maron
Yahoo!'s Pipes — The universal mash-up engine
Stefan Geens
Pipes: Something working, RSS MetaCarta search output
Christopher Schmidt
Yahoo! Pipes: Some Tips
Brady Forrest
Yahoo Pipes - What It Means for Google Earth/Maps - See Earthify
Frank Taylor
My name is Adena and I'm not a programmer…
Most of us in the geospatial arena fall somewhere between two extremes; we are neither "programmers" nor "non-programmers." For those who fall closer to the "non-programmer" side, whenever a software vendor says, "With some customization you can add features you need," we squirm. Can we do that customization on our own? Do we need to call in a coder? Will we need to learn something new? Yikes!
Vendors have realized that a great many of their users are not programmers, but experienced users who could be even better (and perhaps more loyal, meaning more software sales) with some ability to customize programs and build their own tools. That vision, thankfully, has driven "more accessible" customization environments and the use of visual tools for building models and small programs.
Not long ago Yahoo! stepped into this space with a Web 2.0-type tools set that allows customization and manipulation of "feeds" without programming. It's called Pipes. Here's how Yahoo! describes Pipes: "Pipes is an interactive feed aggregator and manipulator. Using Pipes, you can create feeds that are more powerful, useful and relevant." If the term "feed" is still a bit fuzzy in your mind, try this: insert the term "Web data" instead of "feed" in the two sentences above. Now, that sounds valuable, doesn't it?
Pipes is in beta and it's not yet fully geo-enabled, though those requests were at the top of the list for enhancements. Sources note: (1) "Pipes will output GeoRSS. They plan to release a map viewer soon." (2) "Yahoo will open up access to the Pipes engine to programmers and add support for the KML data source, which is used to display geographic data in Google’s popular Google Earth mapping application and Google Maps website." These tidbits, among others, suggest that this environment has significant potential. Furthermore, this is another case where "doing some homework" will likely get you ahead either in your life (creating a pipe that does something personal for you!) or on the job (creating a pipe that speeds up, simplifies or automates a task). And, no, you don't have to be a programmer… honest.
Finding, Running and Looking at a Pipe
I'm a big manual reader, but, alas, the "official" tutorial for Pipes is not yet available, so I had to bumble around a bit. The good news: I didn't bumble long at all. (There are some intro articles by non-Yahoos listed on the Pipes blog.)
As with many tech endeavors, before jumping into building my own pipe, I looked at other people’s pipes. In fact, on the main page you can search for pipes that address topics of interest. I chose to search on "GIS." (Figure 1) I got back three pages of pipes!
 |
I chose to explore "Free Software GIS News" (Figure 2) since I know MarkusN (he's the GRASS guy). His pipe, I learn by clicking on it, "fetches the latest Free Software GIS news for you!" Clicking on "Run this pipe" runs the program and does indeed provide the latest Free Software GIS news (Figure 3). What can you do with that information? Well, you could come to the pipe’s website and run this pipe now and again. But you can also, using the Subscribe pulldown, subscribe to a feed with an RSS reader, get the results via e-mail each day, or a few other things. You can also use the output of this pipe (the list of news items) as the input for another pipe.
 |
 |
Now, Marcus is a programmer, but this pipe is not complex at all. How do I know? I clicked on the View/Edit link near “How This Pipe Was Made.” That brings up the pipe editor development environment. (Since Pipes is a Yahoo! tool, to access the editor or create your own pipe you'll need a free Yahoo! user ID. You may have one if you use Yahoo! Mail or Groups.) Marcus' pipe has a bunch of boxes hooked together with octopus-like links. (Figure 4) All pipes look like that. Some have more boxes and connections, some have fewer. This one is simple, but has many inputs. Let's start with something simpler.
 |
Creating a Pipe
MyFirstPipe is much simpler than Marcus' attempt. It has two inputs that get tossed into a "source" (which is basically a place to "get information," like Flickr or in this case Yahoo! Maps) to create the output. The inputs are a location and a text value. The output is a list of results that fall near the place related to the text (Figure 5). It's basically a local search, one you could already do via Yahoo! Maps.
 |
Now, let's go inside the editing environment and see what I did to build the pipe (Figure 6). First, I decided what my goal was: to have users input where and what they were interested in and have the pipe output results. I had options in the input for what prompt was issued. I used "Where?" and "What?" I could include a default, which I did for both items, so if the user didn't input anything, they'd get a search on "kite" around "Boston." The most fun part of building the pipe? Linking up the components! You click on the "dot" of an input form and drag it to the "dot" that it will "feed" in the "source." So I attached the "Location Input" box to the "Location" field in the Yahoo! Local source. Then I took the "Text Input" and linked it to the "Find" box in the source. The result of this operation is fed into the “Pipe Output.”
 |
Before we leave the edit environment and run the pipe, let's take a look at some of the options in each category. Sources are slim just now and include many Google properties like Yahoo! Local and Flickr, but also GoogleBase. You can also use "Fetch" to search any RSS feed you like (for example, one from eBay or our All Points Blog). User inputs are what someone would key in. They come in many flavors (like database elements or cell types in Excel): text, number, date, location, etc. I like the fact that that there's location and I imagine that'll be more useful to geospatial professionals in time. The list of Operators parallels those you might know from a database or Excel. There are tools to sort, filter and merge data, for example. And, you can even translate data to another language!
Running the Pipe and Beyond
Now, let's get down to brass tacks: running the pipe and thinking about what to do with the output (Figure 7). I chose not to key in any text and took the defaults (kite and Boston). The result was six items (Figure 8). Clicking on one, Danehy Park, fires up Yahoo! Maps. It shows the location of interest, noting in the text that a kite festival is held there (Figure 9). The next step is to take the output of a pipe and use it. I could, for example, put my pipe output as RSS on my Web page (Figure 10).
 |
 |
 |
 |
This is a simple example which shows how easy it is to play with pipes. Yahoo! promises more sources and operators in time. Pipe authors can publish pipes for others to use, like Marcus did. You'll want to check out the published ones (search on "map" for some of interest), some of which are very clever in their own right and others which power map mashups elsewhere. The real power for us in geospatial will come as Pipes expands to support more sources and more operators. One of the top requests is further integration with Yahoo! Maps, so that the output is a map. By the way, if you feel like you could do something like what I did in my example, consider yourself a step closer to the "programmer" end of the continuum!
Further Pipes Resources for the Geospatial Community
Playing the Pipes
Mikel Maron
Yahoo!'s Pipes — The universal mash-up engine
Stefan Geens
Pipes: Something working, RSS MetaCarta search output
Christopher Schmidt
Yahoo! Pipes: Some Tips
Brady Forrest
Yahoo Pipes - What It Means for Google Earth/Maps - See Earthify
Frank Taylor
From Our Homepage
Saying Farewell to an Amazing Journey
Communicating with Maps
Is There a GIS Career Ladder?
What does it mean to be geospatially smart? Series
Ways Real Estate and Property Developers Utilize Melissa GeoData for Data-Driven Decisions
Unlocking Value From Daily Satellite Imagery and Insights
Maximizing the Value of Your Address Data with Geo Addressing
How Indoor Mapping Enhances the Security of Smart Buildings
Look Ahead: AI, Location Intelligence and Efficiency
Collaboration Takes on Sea Level Rise & Dynamic Technology Environments
Brownies for Brownfields
Has Everything Been Mapped Already?
How Is Data Literacy Changing in an Artificial Intelligence Landscape
Portfolios for GIS Professionals: More Than Just Maps
How to Create a Distance Matrix in QGIS - A Step-by-Step Guide
7 Ideas for Bringing GIS into the K-12 Classroom
The Geography of Movement