Linux Shell Flickr Batch Downloader Script

For a while I've been meaning to write a Linux Flickr downloader, but I've been putting it off and putting it off as I thought it wouldn't be easy. But I hadn't thought about using curl and the Flickr API directly and, rather embarrassingly, happened upon a script a few lines long to do just that.

curl "###FlickrAPIURL###" | awk '/id=/ {print "http://farm"$6".static.flickr.com/"$5"/"$2"_"$4".jpg" }' | sed '~s/\(farm=\|server=\|secret=\|id=\|"\)//g' > p
wget -i p

There are plenty of options for FlickrAPIURL, and Flickr provides a very helpful URL builder, for search for example.

An improved script:

#!/bin/sh

# As is this script takes comma separated tags as argument one and downloads all matching
# images from a single Flickr user, probably you

# To get your API Key and Secret create a Flickr App at:
# http://www.flickr.com/services/apps/create/apply/

APIKey='xxx'
Secret='yyy'

# To get your token you can manually follow the desktop instructions:
# http://www.flickr.com/services/api/auth.howto.desktop.html
# If you do not want/need to sign your call you can remove the MD5ing and api_sig

Token='zzz'

# Note that *all* parameters in URI must be in alphabetical order, so this may not work
# for your method/parameters as is

Method='flickr.photos.search'
Params="tags=$1&user_id=YourUserID"

# Image size: _s small square, _t thumbnail, _m small, none medium, _b large, _o original
Size=""

(
IFS='\
'
Page=1
Pages=1
while [ $Page -le $Pages ]
do
        URI="api_key=$APIKey&auth_token=$Token&method=$Method&page=$Page&per_page=2&$Params"
        MD5URI=$(echo $URI | sed 's/[&=]//g')
        MD5=$(echo -n $Secret$MD5URI | md5sum | sed s/\ \ -//)

        for f in $(curl "http://api.flickr.com/services/rest/?$URI&api_sig=$MD5")
        do
                TempPages=$(echo $f | awk '/<photos/ {print $3}')
                if [ $TempPages ]
                then   
                        Pages=$(echo $TempPages | sed '~s/\(pages=\|"\)//g')
                fi
                URL=$(echo $f | awk '/id=/ {print "http://farm"$6".static.flickr.com/"$5"/"$2"_"$4}')
                if [ $URL ]
                then   
                        URL=$(echo $URL | sed '~s/\(farm=\|server=\|secret=\|id=\|"\)//g')$Size".jpg"
                        wget $URL
                fi
        done
        Page=$(($Page + 1))
done
)

UPDATE: Note that if you want to download original sizes you must request the "originalsecret" by adding extra=original_format to the API URL and use that instead of secret. The awk line needs updating to do this, add -F '"' to separate on double quotes.

Short Reveiws: Bam-Bou, Percy / Charlotte Street

A lovely little* Asian fusion place close to perfection, and Goodge Street.

The duck, smeared with plum sauce so think and sweet it was like jam, sprinkled with sesame and drizzled with tamarind was moist and tender and some of the best I've ever had, the sticky coconut rice was extra sticky and coconutty and the prawns and squid were excellent too.

The service was some of the best I've ever encountered, perfectly timed, knowledgeable and polite. What's more, our waitress didn't try to sell us more than we needed, even when prompted.

* Bam-Bou feels small and intimate but is actually quite big, three or four floors.

Recipes: Chocolate Pots

Along with Salmon en Croute this recipe has one of the highest simplicity to deliciousness ratios I know.

1. Melt 200g butter with 200g chocolate*.
2. Whisk 6 eggs with 150g sugar until light (both in colour and density) and fluffy. This can take a surprisingly long time, but be patient.
3. Stir in the butter and chocolate and 150g plain flour.
4. Pour into 6-8 ramekins and bake at 180C for about 10 minutes, until the top begins to crack. In my opinion it's much better to undercook them and have a runny centre than overdo them and let them go solid and a little bland, as I did tonight.
5. Serve with ice cream. The fruit is optional.

*Try using half flavoured chocolate such as Green and Black's Mint or Maya Gold.

Short Reviews: The King's Head, Coltishall, Norfolk

A good gastropub in deepest darkest Norfolk, the King's Head in Coltishall has a fairly standard menu* apart from the fact that about half of it is devoted to fresh and local fish. Make sure you book ahead as I assumed that in a small town in the middle of winter, in the middle of a blizzard no less, there would be no need, but the place was packed and we had to wait until 8:30 for a table, by which time they'd unfortunately run out of the partridge and the lamb. Be careful of the starters, they are huge. After eating my main sized home smoked salmon fillet on guinness "smelt" bread and half the g/f's portion of fried bree? I was already pretty full. On top of that every table gets its own warm loaf**, most of which we took home with us. I thought that prices were quite average for a gastropub, but that would be for London, so I guess it's slightly pricey for Norfolk, the huge starters were around £6, mains started at £10 for pasta and went up to £18 for the steak.

* Not a criticism.
** More of a large bun really. That's not a criticism either.

Short Reviews: Fish Fish, Archway/Highgate

I'm often amazed at how difficult it can be to find a decent chippie; how hard can it be to take a piece of fresh fish, dip it in egg, milk and flour and deep fry it at the right temperature? I was especially disappointed with Toff's on Muswell Hill, which apparently has a good reputation and seems to be shut of a Sunday evening. And so it was an extra special joy to find lovely fresh light golden batter around some tasty fresh fish and good (albeit suspiciously regular) chips about 2/3 of the way to Highgate from Archway on Sunday evening. It's not particularly cheap (£6.95 for take away fish and chips, £11 to eat in) and I was pleased to actuality be able to finish my portion for once, ie, it wasn't a huge portion, small by fish and chip standards.

Short Reviews: Plum Valley, Chinatown

A posh Chinese apparently sanitised for western palettes. The crispy shredded beef was nice and crispy but didn't taste like it had ever seen a chilli, meanwhile I seemed to have ordered the Lancashire hot pot, which I'm not fond of at the best of times. It is possible that we chose badly, our neighbours' set meal looked good, though ours didn't look bad. It was quite pricey too if I remember rightly.

Google Chrome Browser Issues

I've used Chrome a few times since it was released, and it's very fast, but for me, it's just not quite there yet, there are these tiny little niggles:

1. Search as you type. I know I can press ctrl+f, and I know it occasionally mucks up keyboard shortcuts, but I don't care, I want it. Apart from that it makes browsing with the keyboard much easier as you can just hit enter on links, instead of having to hit escape and then enter.
2. Fonts in Linux aren't quite right, see the flickr menu for example, it goes onto two lines.
3. Little website specific bugs, for example, in flickr's organiser, if you press enter after adding a tag to a group of images it seems to hit the cancel button rather than the "thanks" button.
4. I'm getting used to using the history page rather than the missing recently closed tabs feature, but the jury's still out on that one. Ah, there is a recenetly closed list on the new tab page.
5. Would really like to be able to increase the number of thumbnails on the new tab page.
6. While using the debugger, alt+left/right should still navigate back and forward.
7. On Linux it has that same annoying issue that firefox has/used to have: if you kill X chrome thinks that it's crashed and wants you to confirm re-opening your tabs.
8. The spell checker is not as intelligent as firefox's, for example, "exersie" returns no suggestions.
9. The icon is too similar to Visual Studio's, I get confused :)
10. It does not check for a site before googling for what you type into the address bar, eg, if you type "intranet" into the address bar you get google results for "intranet", not your local intranet.
11. The speed boost just doesn't seem as pronounced on Linux.
12. RSS feeds don't seem to show up like they do in Firefox.
13. Chrome seems to "steal" Firefox's saved usernames/passwords and history, which I'm not sure I'm happy about.

I also have an issue with Buzz whereby a Buzz that is in both your inbox and buzz-box doesn't get marked as read in one when you read it in the other. I think that it's just delayed, but it does mean that I click on both boxes to see one message. Also, maybe if you're viewing your buzz-box and a new one comes in it should be marked as read straight away.

AND a couple for gmail: gmail search doesn't split words with underscores and the contact autocomplete doesn't search for words after a period.

Self Certificated Sick Leave

Please say briefly why you were incapable of work for the period stated above, words like unwell or illness are insufficient.

I had a fever, slight headache and very sore throat which made it difficult to speak, admittedly not essential to my job but I wouldn't want to pass the illness to my colleagues who need to use the phone without sounding like they'e doing a Barry White impression.

Using Outlook Web Access

Since my work seem to be refusing to enable IMAP on their new Exchange server and I refuse to use Outlook, which is just too slow and cumbersome for my simple needs, I've spent the last few days getting to know Outlook Web Access, Microsoft Exchange's webmail offering. Here are some tips:

1. If you're using Firefox and are miffed to find that you can only use the "light" version of OWA, don't be, I prefer the light version and would recommend it to IE users too. And no, OWA does not work on Firefox if you fake the User Agent.
2. Don't worry about the missing autocomplete/drop down functionality in the To, CC, etc. fields, type in the names of your recipients separated by semicolons and click Check Names.
3. If you need to import contacts use the desktop version of Outlook.
4. There is no Discard button on the Compose form (my one annoyance so far), so go straight to the drafts folder, use the check box and click Delete.

It's not too bad really.

Facebook Stream_Get, Offline_Access and getFilters Not Working

I am not sure whether I am being stupid, the Facebook API documentation is really bad or the Facebook API just doesn't really work very well, but if you want to display a user's own profile feed on a page (eg, your own feed on your homepage) here are some pointers: (I do not guarantee any of this information is up to date or even correct.)

o) Setup your application on Facebook, it may be easiest to use the sample/example code they give you to start your project. Fill in the Canvas Page URL and point the Canvas Callback URL at your own domain URL. Then visit your Canvas Page URL.

i) To be able to have a page request restricted information from Facebook while the user is not logged in on a particular machine you need to request off line access, you can do this in these two ways, among others:

<fb:prompt-permission perms="read_stream,offline_access">
Grant
</fb:prompt-permission>

<script type="text/javascript">
FB.Connect.showPermissionDialog('read_stream,offline_access', ondone);
function ondone(a){ alert(a); }
</script>

ii) You then need to grab a persistent session key, in PHP this is available like so:

$key = $facebook->api_client->session_key

$key = $_POST[fb_sig_session_key]

iii) Once you've got this key you (or your script) can "become" the user like so:

$facebook->set_user($user_id, $key, 0)

iv) You can then get an array of streams using stream.get:

$streams = $facebook->api_client->stream_get($user_id, $user_id...)

v) I have not worked out how to "filter" stream_get to return just posts, or links, etc. getFilters will return a list of filters, but I have not had any success in changing the return value of stream_get, at all. There are also apps on the filter that is returned that I am sure I have not installed, Doodle Jump for example, wtf?

The $stream[posts][x][type] value in my streams reflects the following:

11 - create/join? group
46 - post
56 - wall post
80 - link
237 - flickr
none - profile pic, note

Changing the content type of my page rid it of some nasty characters:

<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />

Remember that PHP's htmlentities() can also take a charset.

See the result at http://kitten-x.com/#facebook.

The ridiculous thing is that you can save an awful lot of time and effort by using the Facebook RSS feeds, such as:

http://www.facebook.com/feeds/share_posts.php?id=userid&viewer=userid&key=akey&format=rss20

To get your key (and userid) go to your notifications page and click on the "Via RSS" link, you can then change "notifications" in the URL to "share_posts", "status", etc.