Yahoo Pipes, Microformats and Extendability

I think Yahoo Pipes is really cool. The main attraction is its slick user interface and ease of use.

I just created a pipe of all of the Recent Questions of Yedda translated using Babelfish to French and it took less than 5 minutes.

I do have a couple of ideas that I think will make Yahoo Pipes into something very interesting:

  • Accept Regular HTML pages
  • Have a built-in Microformats parser
  • Support for a more complex piping scripting (perhaps in the form of a JavaScript script)
  • Support for state saving (or at least a limited way such as the ability to compare the previous version of the page/feed you are piping)

Accept Regular HTML pages
Currently, Yahoo Pipes (at least as far as I’ve figured it out) accept only feeds (Atom, RDF, RSS, etc). The other building blocks that works with Yahoo Search, Google Base and Flickr eventually output a feed to Yahoo Pipes. Having the ability to retrieve a page instead of a feed and manipulate it will make things a lot more interesting and will allow VERY interesting meshups and ideas

Built-In Microformats parser
If Yahoo Pipes will accept regular pages, having a built-in Microformats parser will allow people to extract various types of structured information stored in the Microformats on the pages, thus, creating a reacher and more interesting abilities with Yahoo pipes.

Pipes Scripting
Having custom scripting abilities to Yahoo Pipes will make it really great and will allow a burst of innovation and interesting things composed with Yahoo Pipes. Of course, this feature is the most complex one from both development and security since having 3rd party code run on your servers is always a problematic thing. But, I’m sure the fine people at Yahoo can limit that.

One idea that comes into mind is writing such scripts in JavaScript, thus the whole running of the scripts on top of a page will be contained into a JavaScript environment and can only work on the input of the file being parsed.

State Saving
State saving will allow users to create a more complex pipe that can be aware of changes. The simplest one is to compare to the previous version of the page/feed, thus allow the pipe writer to figure out what to output.

An interesting pipe example that uses some of the things I’ve talked about above would be to have a pipe that listen to a certain drivers vendor’s driver page (most of the drivers vendors don’t have a feed that I can subscribe to and know when there are newer versions of a driver and things like that). The pipe would extract the current version and date from the page and compare it to the previous version stored at Yahoo of that page. If it has changed, it will add an item to the feed’s pipe saying that a new version exists, etc.

What do you think? Will this work? Would you be interested in such things?