Saturday, June 05, 2010

Linux Shell Flickr Batch Downloader Script

For a while I've been meaning to write a Linux Flickr downloader, but I've been putting it off and putting it off as I thought it wouldn't be easy. But I hadn't thought about using curl and the Flickr API directly and, rather embarrassingly, happened upon a script a few lines long to do just that.
curl "###FlickrAPIURL###" | awk '/id=/ {print "http://farm"$6""$5"/"$2"_"$4".jpg" }' | sed '~s/\(farm=\|server=\|secret=\|id=\|"\)//g' > p
wget -i p
There are plenty of options for FlickrAPIURL, and Flickr provides a very helpful URL builder, for search for example.

An improved script:
#!/bin/sh # As is this script takes comma separated tags as argument one and downloads all matching # images from a single Flickr user, probably you # To get your API Key and Secret create a Flickr App at: # APIKey='xxx' Secret='yyy' # To get your token you can manually follow the desktop instructions: # # If you do not want/need to sign your call you can remove the MD5ing and api_sig Token='zzz' # Note that *all* parameters in URI must be in alphabetical order, so this may not work # for your method/parameters as is Method='' Params="tags=$1&user_id=YourUserID" # Image size: _s small square, _t thumbnail, _m small, none medium, _b large, _o original Size="" ( IFS='\ ' Page=1 Pages=1 while [ $Page -le $Pages ] do         URI="api_key=$APIKey&auth_token=$Token&method=$Method&page=$Page&per_page=2&$Params"         MD5URI=$(echo $URI | sed 's/[&=]//g')         MD5=$(echo -n $Secret$MD5URI | md5sum | sed s/\ \ -//)         for f in $(curl "$URI&api_sig=$MD5")         do                 TempPages=$(echo $f | awk '/<photos/ {print $3}')                 if [ $TempPages ]                 then                           Pages=$(echo $TempPages | sed '~s/\(pages=\|"\)//g')                 fi                 URL=$(echo $f | awk '/id=/ {print "http://farm"$6""$5"/"$2"_"$4}')                 if [ $URL ]                 then                           URL=$(echo $URL | sed '~s/\(farm=\|server=\|secret=\|id=\|"\)//g')$Size".jpg"                         wget $URL                 fi         done         Page=$(($Page + 1)) done )
UPDATE: Note that if you want to download original sizes you must request the "originalsecret" by adding extra=original_format to the API URL and use that instead of secret. The awk line needs updating to do this, add -F '"' to separate on double quotes.


Kai Hendry said...

First off the code formatting is so bad. You should make an effort to explain (format) the code, so that shell does not come off as some black magic. And then god forbid, people who don't understand it, end up writing Java or Python instead. Massive fail.

I am not sure why you need to download Flickr images, but I do the backup task from time to time using Henri Sivonen's Flickr backup tool. It is written Java and I wish it was written in shell. :)

Jamie said...

It's not black magic?

Sharath Chandra Guntuku said...

I went through your code on downloading flickr images. Is it possible to download public photos from other users using this script?
Please let me know.

Jamie said...

Yes, just remove "&user_id=YourUserID" from the search parameters.

Jamie said...

If you were only after public photos you also wouldn't need to sign the call.