Until recently, people had been asking on Vallywag and Digg what Yahoo Pipes might be. Well, now we know. Yahoo Pipes is a powerful way to munge and aggregate feeds using a very slick graphical interface. Editing feeds in as a workflow means you can get some powerful results very quickly. For sources it currently supports Flickr, Google Base, Yahoo Search and Yahoo Local, as well as fetching arbitrary URLs (which you can also munge with a URL builder). All sorts of user inputs are supported, as well as iterative blocks to annotate and replace content, content analysis (which finds significant terms, and has long been a useful Yahoo API in its own right), content filters, BabelFish, string manipulation, date formatting, and more. The list goes on and will hopefully keep growing. Imagine taking the feed for your favourite blog and automatically having a relevant photo from Flickr added to each post. This not only makes such a thing possible, it’s one of the many included examples.
To try it out, I decided to create a simple pipe. I went for the standard feed-scrubbing example; BoingBoing with Xeni’s posts removed (nothing against Xeni, it’s been a long-running hobby of BB readers to make it more worksafe by filtering out her content, originally using Greasemonkey).
Pipes can published and shared with the wider world, so now you too can see my BoingBoing sans Xeni pipe here and of course subscribe directly to the RSS feed.
Other than couple of small usability gripes, I really love the system already, and am looking forward to making some cool things with it.
What gripes, you ask? Well, for one thing to populate a for-each module I have to drop the other module into it directly from the side bar; it doesn’t work if you drag them in from your workspace. A small thing, and it only took me 30 seconds to figure it out, but it’s not obvious. It’s a compliment to the system really, and when something is this close to being perfect a couple of tiny flaws make you notice how good it really is.
Now the internet is not just a series of tubes, it’s held together by a series of pipes.
I read that title multiple times as “Yahoo Pipes is Pretty Awful”. I don’t know why. The UI is indeed rather slick. It’s good to see this happening, just to get the ball rolling in people’s minds.
The closing line’s the zinger! The interface looks subtlely striking (!), somewhat like Web 2.0 cliches combined with early puzzle games (like The Incredible Machine).
Has anyone created any great pipes involving Second Life yet? I’d sure like to know!
I’ve been enjoying Pipes some more this weekend, and while I’m still impressed with the interface, the limitations are starting to annoy me. I want to be able to do more things with the feed than filter, translate, extract significant terms (with no input into which might be the most useful elements to check by the way), etc. I want to be able to get to the tags in my recent del.icio.us bookmarks, and make further searches based on that.
Of course, since Pipes is web based there’s nothing stopping them sneaking more features in there over time, and I hope they will.
The other thing to note is that of course while this is whizzy and pretty and usable, it’s not the only, or the first, mashup maker out there. I’ve been tagging some more using ‘mashupmaker‘ on del.icio.us. The one I should probably mention (partly because it’s an emerging tech prototype from IBM, and so I’ve been talking about it at conferences and with clients for a while now) is QEDWiki.
Now that really is starting to look like the beginnings of the future. You need a full scripting language in there, and the ability to store and retreive to a database. Throw in a connection to amazons mechanical Turk, and you’ll really be living declaratively….
Pipes and QEDWiki are pretty cool since they can be used by non programmers. I think that developers will always want to have access to straight HTML / JS and do not want to be bound to a certain platform. We have a simple MashupMaker online that allows just that. Check it out at http://www.webrpc.com/mm.