Google Scholar + Yahoo Pipes = scholar alerts

I was talking to a marine biologist friend who was saying that he really wished Google Alerts would cover Google Scholar results also. Right now alerts only cover news, blogs, web, video or groups content.

It was a chance to play with Yahoo Pipes, and potentially applicable for a project at work, so I threw this pipe together:

Google Scholar RSS/email

You can search for a keyword since a specified year, and you should get the title, link and the source of articles. You can sign up for email alerts for changes to the results as well, so essentially doing the same thing that Google Alerts would do. This will work until Google messes with the <span> tags on the page…the pipe is thrown together with a bunch of regular expression filtering.

update: Yahoo seems to drop the search terms when you try to sign up for the email alerts.

update 2: Google disallowed scraping of the scholar results page, so this pipe no longer works.

Advertisements

15 thoughts on “Google Scholar + Yahoo Pipes = scholar alerts”

  1. Mike:

    This is really cool….. is there a way that it can be sent to a non-yahoo web page. This would be very helpful for people trying to stay current on publications.

    Regards,
    Tom

  2. I’ve built something similar using php but have not advertised it to my colleagues because I’m pretty sure it violates google’s terms of service.

    “5.3 You agree not to access (or attempt to access) any of the Services by any means other than through the interface that is provided by Google, unless you have been specifically allowed to do so in a separate agreement with Google. You specifically agree not to access (or attempt to access) any of the Services through any automated means (including use of scripts or web crawlers) and shall ensure that you comply with the instructions set out in any robots.txt file present on the Services.”

  3. Hey Mike,

    I *love* your Google + Pipes = Scholar alerts pipe. Genius. I can get it to work within the yahoo pipes page, ofcourse, and within Google Reader. Recently though, I was wondering about an “advanced search” version of the thing. It’d take a little more URL crafting on the front end, but wouldn’t change the rest. Sadly I’ve tried, I’m not savvy enough. Ever thing of making the advanced version?

    Cheers,

    Tim W

  4. HA! Great stuff!

    1. It solved my issue with not being able to create alerts for Scholar
    2. It showed me that Pipes exists, great tool!!

    Thanks!

    * bow *

  5. Hi there! Been using this pipe for several feed widgets in a project Netvibes page- my project participants are absolutely delighted as am I- but it doesn’t seem to be working just now. Has Google shut you down do you think? Or is there some other problem?

  6. Anyone can clone and edit pipes. If you were to go to my pipe, edit source, click on Fetch Page, you see this error

    “Can’t fetch pages that robots.txt disallow”

    which means you Google disallowed scraping of their scholar results page.

  7. Might you help us understand why a simple alert service based on keywords in academic journal abstracts, which practically all publishers display at no charge on their own websites, isn’t already provided by Google Scholar? I believe that while the articles are copyrighted, the abstracts themselves, as well as keywords, would seem to be searchable in some manner. Many thanks!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s