|
|||||
Getting Abstracts from SSRN in RSS formThe Social Science Research Network (SSRN) is becoming the top place to find new research. Unfortunately, it doesn’t support RSS feeds except for author pages. That means that if you’re interested in a particular topic like entrepreneurship or finance you have two options. You can either visit the site and click on each individual link or in some cases you can subscribe to a particular network to get email updates. Either way, those options are unacceptable. Who uses email anymore for news updates? That would be like using an abacus or a slide rule to perform calculations. And more importantly, who has time to be checking out a bizzillion websites to stay on top of news and research? If you’re not using an RSS reader you’re wasting your time. So, in the spirit of wasting time, I started working with Yahoo Pipes and Dapper. Using these two tools I was able to extract the URLs for a keyword search on SSRN and use that result to extract the titles and abstracts from individual SSRN pages. I am happy to say it works, but my method is probably not as efficient and elegant as it should be because it causes Dapper to time out sometimes. Once it is out of Beta, I will post the link so that others can use and modify it for their own research. Lastly, I’m sad that more researchers and institutions don’t use REPEC. In my opinion, it’s more open and you can find RSS feeds by topic. People who looked at this item also looked at…Related items4 comments to Getting Abstracts from SSRN in RSS form
|
|||||
|
Copyright © 2010 Distant Traveler - All Rights Reserved |
|||||
I’d be very interested in knowing how you went about this. I’m attempting to do something similar, but I haven’t had a lot of luck so far.
Hi Micah,
I will post some screen shots this weekend and a detailed outline of what I did. Until then, I’ll give you a brief one.
1. I used ponyfish.com on a page like this:
http://papers.ssrn.com/sol3/JELJOUR_Results.cfm?code=A14
I could have used dapper for this, but ponyfish is quicker for simple extracts. Dapper is good for more complicated pages, but it’s more of a pain to use. Essentially, I just wanted links.
2. I then went to Yahoo Pipes and extracted the ID of the abstract. Here is what I am using:
http://pipes.yahoo.com/pipes/pipe.info?_id=WAIKrpo23hG8wji_dPQQIA
3. In dapper, you have to create two dapps. First, you need to create a dapp of the abstract page.
http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1350190
Try to get just the title and the abstract. The more pages you use the better because SSRN does not have entirely consistent abstract pages. This will make the dapp more accurate.
Somewhere along the process of this dapp there will be an option to create a “variable”. This will be the last 7 digit number.
The second dapp then essentially pipes the 7 digit ID number over to the dapp you created to input it into the variable.
Alternatively, you can bring over the entire link and pipe that over as a variable.
If that works for you, great. I often get time out errors from dapper, but if I let it “rest” for a few minutes, it seems to work. Like I mentioned in the post, I’m trying to find a more elegant way to do it, but it works so far….
Hope this helps. I’m currently in finals, so I probably won’t have time to post more until I’m free.
Good luck and let me know if you come up with a better way!
Thanks for your help. Dapper is an awesome tool, I appreciate the pointer. I couldn’t get ponyfish to do anything with the keyword search page on SSRN, so I created a dapp that generates a feed of most of the information I need. I’ll keep at it.
Robert, thanks for the suggestions. Would be great if you could post some more details and screenshots as I am quite keen on getting this to work as well, but haven’t managed so far.