The Technorati Authority Yahoo! Pipe

Over the holidays, I started playing with a new Yahoo! pipe to pull information from Technorati into a spreadsheet. The reasons why I wanted to do this are covered in this post about the quantitative analysis of blogs, and my eventual perl-based solution to the problem is covered in this post.

The problem with the perl-based approach is that it’s a little inaccessible to people who aren’t comfortable using a command line environment. So I really wanted to make something that more people would feel comfortable using, and perhaps play around with.

So, with some help and kind words from Bob Briski, one of whose pipes I’d stumbled across and bookmarked during my research for this project, I decided to finish off the pipe and publish it so that others could use it, or (better still) improve upon it.

What does it do?

The pipe first pulls a list of blog URLs from a Google spreadsheet, then checks Technorati to get the “inbound blogs” count for each blog, then outputs the blog title, link and count to an RSS feed.

Technorati Authority (a useful metric when ranking a list of blogs) is calculated as “the unique blogs linking to a blog over the past six months” — in a way it’s a measure of how many readers may be influenced by a blogger at one remove.

Before you start

You’ll need:

1. An API key from Technorati
2. A Yahoo! account
3. A Google account

I’ve done all the rest, so you could simply clone my pipe, which you’ll find here.

Publishing a Google spreadsheet as a CSV source

In my first pass at this challenge, I used a simple text list of URLs stored on this server. That’s pretty easy if you have (a) a server, (b) a text editor, and (c) some way — like ftp or ssh — of getting the two to talk to each other. But lots of people don’t have servers these days, and still more probably have no idea what I mean by (c), so I decided to follow a new approach before writing this article. Everyone, I thought, must have access to Google Docs.

So here’s my test list as published on Google:

Now all I have to do is Share > Publish as a web page, then click More publishing options and select ‘CSV’ or ‘TXT’ (1) as the publishing option to create a simple text list.

You can see the CSV file here. You need to copy (or make a note of) the link (2) because we use it as the seed for our pipe.

One of the nice things about using Google Docs is that it’s easy to edit, view and share the list. For whatever reason, you may not want to share your list of blogs so if you’d like the original version with the server-based file which is more ‘private’, it’s still available.

Setting up the Pipe

The first thing to do is to build a URL for each request I’m going to send to the API. Technorati takes requests in the following format:
http://api.technorati.com/bloginfo?key=[apikey]&url=[blog url]

The pipe takes two variables that will need to be customized for your own version. One, rather obviously, is the link URL for the Google CSV that you copied or made a note of above (you did make a note of it, didn’t you?). The other is your Technorati API key for which you’ll probably need to sign up. It’s free, but is limited to 500 calls to Technorati’s database per day.

I’ve plugged the link into the Fetch CSV module at top left, and the API key into the Private String module. I use this module so that I can protect my API key — no-one else should be able to see it now.

The URL Builder module placed within a Loop module does this. I’ve hooked the API Key from the Private String into the parameter named ‘key’, and the output of the Fetch CSV module into the ‘url’ parameter.

Getting the data from Technorati

Now it’s simply a matter of looping through the requests that I built during the set-up stage above, and sending each one to Technorati so that I can fetch the data. Technorati responds with an XML file that looks like this:

[code lang="xml"]





    
        http://www.mediaczar.com/blog
             
                Mediaczar
                http://mediaczar.com/blog
                
                http://feeds.feedburner.com/mediaczar/posts
                14
                21
                2009-01-06 12:45:23 GMT
                422833
                
                    
                        mediaczar
                        Mat Morrison
                        Mat Morrison is a digital marketing and
                        communications strategist with over a decade's
                        experience in online advertising, eCRM, and social
                        media.
                        http://technorati.com/people/technorati/mediaczar
http://static.technorati.com/progimages/photo.jpg?
                        uid=139758
                    
                
             
                14
                21
    


[/code]

I can choose which bit of the XML file I want to receive by setting the ‘path to item list’ parameter in the Fetch Data module. You’ll see that I’ve set it to ‘document.result’ — compare that to the XML example above and I think you’ll see how this works: I’ve removed the wrapper information from the file and gone straight to the meat.

Renaming the fields to suit RSS output

Yahoo! Pipes output is limited to RSS, so we need to reformat the Technorati XML as RSS. Now Bob’s shown me how to do it, it’s pretty straightforward: RSS needs a title, link, and description. So I’m using the Rename module to do just that, and choosing the ‘item.inboundblogs’ parameter as the description. I plug the results into the Pipe Output, and I’m ready to go. Save the pipe.

Checking the output of the Pipe

Now I’ve saved the pipe, I can run it to see if it’s working.

Sure enough, there’s the Authority figure coming through nice and clearly (1). Success! But there’s something more we can do. Clicking on the More options link (2) gives us an opportunity to take the RSS feed back into other tools like a feedreader, or into NetVibes (I’m a big fan.) Or, given that we started in Google, we could take it back there…

Importing the RSS feed back into the Google Spreadsheet

Image may be NSFW.
Clik here to view.
Google Spreadsheets has some useful functions that you don’t find in Excel; and which are more geared towards the web. The only one that I’ve really played with is the importFeed function which — as you might expect — imports RSS feeds into your spreadsheet. So here, we paste the RSS link from the pipe into the sheet…

… and it fills in our sheet for us. Now I can change the list of blogs on one sheet, and (as if by magic) the results will appear on the other, without my needing to go into Pipes. I’ve used Yahoo! Pipes to link one Google Spreadsheet to another by way of Technorati. Fun? I should say so. Useful? Most certainly.