Pling!

Testing i18n in Django for the Molly Project

This was originally published on the Mobile Oxford Tech blog

Here in the Mobile Oxford team, we’ve been working with the team at Bangor University to help get the Molly Project, the open source project which grew from Mobile Oxford’s code base, ready to be translated into multiple languages (in Bangor’s case – Welsh!).

As Molly is a Django project, we can use Django’s extensive i18n framework to do most of the work for us, however, this does rely on us marking up the text for translation. When it comes to testing i18n, we needed a quick way of generating a test language file for us to drop in to check we’ve caught all of the strings that need to be translated.

Inspired by Richard Mitchell at Isotoma, who suggested using upside down English to test for Unicode bugs, we decided that testing i18n using upside-down English makes the most sense for us. And so, I wrote a script which did this. It ignores formatting strings and inside HTML, but everything else gets inverted using the lookup table defined at the top. We used this as we marked up each app in a feedback loop to make sure everything is marked up, and I think we got everything.

Data from external services (e.g., the library) isn’t translated, but where our data sources (e.g., the OpenStreetMap planet.osm dumps and the NaPTAN) do contain multi-lingual data, this is now imported into our database and tagged as the correct language, and where the data exists, it means you’ll see it in different languages!

However, we did find a few gotchas with Django’s i18n. First of all, there’s a fairly major bug in Django’s makemessages function, which means that the output of makemessages for JavaScript is unreliable as to whether or not we caught all the strings. The only real alternative is to generate a .po file by hand, however we cheated by creating a ‘dummy.js’ file which manually contains all the JS scripts, to make sure they’re found. However, this does mean we have to update this file every time we add a new string, which is a pain.

We’ve no plans to launch multiple languages on Mobile Oxford at the moment, but this is perhaps something we can look at in the future – translating the site into multiple languages to make it more useful for tourists to the city!

1 comment.

Trackernet and the NaPTAN

At work I’ve been playing around with the TfL TrackerNet API to add real-time tube departure information to the Molly Project, in a similar way to how we provide real-time bus information from Oxontime.

The biggest problem I came across was the fact there’s no mapping (at least, I couldn’t find one – someone may now comment and show me I wasted an hour of work this afternoon) between the NaPTAN and the station and line codes that TrackerNet wants for its detailed predication API. So I had to go make one.

So if anyone wants a prepopulated Python dictionary mapping ATCO identifiers to lists of tuples of line codes and station codes, save yourself some pain and just use the one I just made (under the Academic Free License):

1 comment.

IT Professionals in Schools

Our industry has a problem. It’s not a new problem, and it’s a fairly well known problem. Our industry is suffering from a skills shortage, but this is just a symptom of the problem. The problem starts in schools.

Anna Debenham articulates the problem better than I ever could, and if you’ve not seen this presentation yet, I highly encourage you to watch it:

(more…)

2 comments.

Github to JIRA

I love JIRA quite a lot, ever since I got my first license from Atlassian for the Nouse website (in an attempt to get some form of project management going on), and here at OUCS, where we use JIRA internally. Up until now, we’ve managed the Molly Project using the public Github issue tracker (and private Mobile Oxford stuff on our internal JIRA), but I’ve finally got round to setting up a JIRA instance and getting an open-source license from Atlassian for the Molly Project, but I wanted to get our existing tickets (just under 80) from Github into JIRA, and ideally not by hand.

Fortunately, JIRA supports CSV comment imports, and Github has an API, so it’s a simple task of putting the two together to get what I want.

I don’t think the Github issue tracker is really suitable for any type of real use (you can’t assign tickets, there’s only two states – labels are a bit of a hack compared to actually having component fields/versions, etc), so if you’re thinking about getting a real issue tracker for your open source project, give JIRA a whirl. I’ve put my script up on Github for anyone who’s doing the same: https://github.com/cnorthwood/github-to-jira

No comments.

Reporting Django exceptions to JIRA

I’ve recently started a new job over at Oxford University Computing Services working on Mobile Oxford. I’ve blogged over on their tech blog about integration Django exception reporting into JIRA – you can read the whole article there.

No comments.

Live Blogging version 2

I’m pleased to announce the release of version 2 of the Live Blogging plugin for WordPress.

The Live Blogging plugin offers lots of functionality for WordPress bloggers, especially those who blog following live events. Using Live Blogging, you can create many small, microblogs within a single post, with each update automatically appearing on your readers screens.

For a demonstration on how to use the plugin, and how it works, I’ve created a 4 minute screencast demonstrating it.

For users of version 1, version 2 is an almost complete rewrite, adding many new features – the most important of which is the removal of the requirement of Meteor for automatic updating (although using Meteor offers improvements over the standard polling situation), so the effect demonstrated in the screencast is available to everyone. More customisability is available too, for those who have used Live Blogging v1.

I’d also like to thank the beta testers for helping me debug the release before I released it. Also, thanks to all the users who are using the live blogging plugin.

Watch the screencast

Download the plugin

Please spread the word!

65 comments.

Live Blogging version 2 – looking for beta testers

Over the past few weeks I’ve been working on version 2 of the Live Blogging plugin for WordPress which I originally released last year, and which subsequently won runner-up in the 2009 Weblog Tools Collection Plugin Competition – an acolade very gratefully received!

One reviewer of the plugin did point out that:

I get the impression that it is a plugin that was built for a specific use and released, rather than a plugin that was intended to be released.

which is a very fair, and accurate, assessment. It’s no secret that I developed this plugin in a few days for Nouse‘s coverage of Roses 2009.

Version 2, however, changes all that. It’s been rewritten from the ground up for WordPress 3.0, with no constraints on time for a release date, so it’s coded a lot better than version 1 was. It also addresses a lot of niggles people had with version 1 of the plugin. It no longer relies on Meteor (it can use an AJAX polling method which is suitable for low traffic blogs), the backend user experience is better and the plugin as a whole is a lot more flexible.

When the final version of the plugin launches, I’ll be writing a full blog post detailing the changes and new features – but rest assured, this is not the same plugin as version 1, but contains the same principle – microblogging without your readers having to refresh the page.

As for now, I’m looking for beta testers. Particularly people who have used version 1 of the plugin, as the migration functionality is one part in particular I’m keen to test. If you are interested in testing and blogging using Live Blogging v2, please comment on the post, drop me a tweet, or e-mail me. I’m hoping to have a beta ready within the next week, which I can distribute and elicit feedback on, and hopefully squash a few more bugs before 2.0 hits the WordPress plugins directory.

I think this plugin could be something big, especially for professional bloggers, and making the barrier for entry considerably lower than what version 1 could provide is something, I hope, can achieve this.

The second beta is now out, and can be downloaded here. Any feedback will be gratefully received.

9 comments.

Encryption

So, I’ve decided to migrate my devices towards encryption. I’ve documented the process here, in case you find it useful.

Desktop PC

This runs Windows 7, however only the “Professional” version, so no Bitlocker technology (shame). I’m using TrueCrypt’s volumes for my non-SSD drive which stores the bulk of my personal files. My profile (AppData folder) and programs run off my SSD, and sensitive data (AppData, MSN logs, Desktop and local code checkouts) is encrypted using Windows 7′s EFS (my user profile).

However, my personal data (Documents, Music, Photos, etc) is shared with my laptop for file syncing, and would TrueCrypt volumes being mounted at log in time, this means the shares aren’t automatically recreated every log in. A Windows batch file is used to automate this:

"C:\Program Files (x86)\TrueCrypt\TrueCrypt.exe" /auto favorites /quit
net share Archive=Z:\Archive /GRANT:Chris,READ /CACHE:Documents
net share Documents=Z:\Documents /GRANT:Chris,FULL /CACHE:Documents
net share Music=Z:\Music /GRANT:Chris,FULL /CACHE:Documents
net share Pictures=Z:\Pictures /GRANT:Chris,FULL /CACHE:Documents

Obviously the share names, locations, permissions, etc, will all need changing, but invoking TrueCrypt this way (rather than its automount favourites on startup option) guarantees the availability of the share (as long as you don’t cancel the mount). However, this script needs to be run as Administrator, and with no UAC allowed at login, a scheduled task had to be created to get this working.

This machine is backed up with Mozy, which is supposedly encrypted, but the private key isn’t mine – so I’ve now changed to using my own custom AES key for Mozy. This key as well as the EFS ones is itself backed up (in a TrueCrypt volume) in the event they ever need to be restored.

Laptop PC

Again, with Windows 7 Professional, so no Bitlocker, but Windows’ EFS is used on sensitive data (my user profile), which makes encryption pretty simple to use. My personal files are shared on my desktop (requiring appropriate credentials, obviously), and these shares are mounted on my laptop, using the ‘Offline Files’ feature. This has a built in option to use EFS to encrypt the Offline Files cache, which is used to encrypt those personal files.

Netbook

This machine is Fedora based, and so supports encryption out-of-the-box. However, there appears to be no easy migration path to an encrypted volume if it’s not already encrypted, so I’m putting off implementing this until I need to rebuild my netbook. I don’t store my personal or business files on it (it’s literally just a web browsing machine), so exposure here is limited to basically my browser history.

Phone

The Nokia E71 supports full encryption out of the box, and this was very simple to set up.

iPod

My iPod Classic doesn’t support any type of native encryption (nor is it possible to install full device encryption), however my iPod Classic doesn’t contain any data I consider dangerous if unencrypted (just my music collection), so this isn’t an issue. The iPod Touch is more of an annoyance, as it has things like E-mail, Calendars, Contacts, etc, as well as the myriad apps that seem to stay logged in. Coming to review it at first, I realised how much of an issue it would be if I did lose it or my iPod Touch got stolen. I removed all the auto-sign in password things (however, I will have to remember to manually log out every time I use an app on it), set up a passcode and got rid of the link to my e-mail/contacts/calendar, etc.

iOS 4 did correct this obvious shortcoming with it’s new data encryption features, so I’m back to using my iPod Touch as it’s meant to be used.

Memory Stick

This is a fairly simple case of TrueCrypt encryption again, however, instead of full drive encryption, I am only going to create a TrueCrypt volume – this allows for easy transfer of unencrypted data between any machine that does not have Truecrypt on, if need be (although obviously I must be careful to not put any sensitive documents in there).

And that’s it! With the exception of trivial storage (such as the SD card in my camera with recent photos), my digital storage now has some sort of protection. I realise it’s not perfect – binaries aren’t encrypted (full disk encryption on an SSD is heavily advised against from multiple sources I’ve read), backing up encryption keys in a TrueCrypt volume is possibly a weak link in my system, as are things like hibernation files/leading my laptop in sleep mode, and of course the human link (I know the passphrases) – but hopefully it’s better than nothing, and if my phone or laptop or desktop does get stolen, me (and my customers) can be confident that my records, and my own financial records, are safe.

In other news, I might as well blog a few links as to what I’ve been up to lately:

• I’ve redesigned the website for my parent’s holiday home;
• I moved my web hosting company from Poundhost to Linode sooner than planned, due to a hardware failure on the old server;
• I launched a new site for my web hosting company – the old one was incredibly dated;
• I attended WordCamp UK and took part in WordHack – building a website to show information about how government aid is spent: http://www.openaid.org.uk/.

No comments.

Bug in Facebook privacy settings preview

So, I’ve been updating my Facebook privacy settings lately, just because they seem to change every week, and decided to delete all of my ‘connections’, apart from a select few which are very important to me, and I stumbled across a bug in Facebook’s privacy settings. I should stress that it would appear that your privacy settings are respected, but the ‘preview’ feature simply isn’t showing you correct information.

I’ve set my privacy settings for my connections to friends only, and have no uncategorised ‘Likes’ (due to the bizarre Facebook decision to not be able to set privacy settings for ‘Other’ likes):

I’ve also set the privacy settings to those who can see my friends list to be friends only.

As one would expect, the profile preview doesn’t show these things, but my profile is set to be visible to search engines, and despite my privacy settings (‘Likes’ and ‘Friends’ to be ‘friends only’) and that the profile preview correctly does not show my likes, my search engine preview does. Bizarre, and almost certainly a bug.

Now, actually viewing my profile when not logged in doesn’t show that information, but with yet another bug in Facebook’s privacy settings and the inability to set privacy settings on some aspects of your profile, I’m beginning to trust Facebook less and less.

No comments.

Apache and user-separated PHP FastCGI processes without using Suexec

For most of this evening I’ve been battling against a problem that’s haunted me for a while – how to get PHP scripts to run with sane permissions, rather than “apache/apache” for all the users on my web server. Pretty much every tutorial on the Internet advocates one of two approaches:

• Using Suexec (or some variant thereof)
• Using the ITK or peruser MPM

However, using suexec isn’t feasible for me – my virtual hosts are dotted all over the file system, not served from /var/www as Suexec demands, and I don’t want to start moving my customers files around in case they have hard coded file system paths, and I don’t want to roll my own Apache – I want to stick with the official Fedora RPMs. There’s a third method too – suPHP, which runs a script as the user which owns it, but this also seems a bad idea imo (just like PHP’s “Safe Mode”).

I’ve also been investigating the feasibility of moving to Lighttpd (but decided against it), and stumbled across a tutorial on how to accomplish this with Lighttpd and FastCGI and thought I could take the same tack with Apache. So, I implemented the first part of the tutorial (using spawn-fcgi to start up the PHP FastCGI clients), but it was configuring Apache to use it that flummoxed me. First off, you can not use external FastCGI servers with Apache’s built-in mod_fcgid, you have to use the mod_fastcgi module, so I compiled, configured and installed that. Then I played around with the Apache config to get it to do what I want to do, and the following in my www.pling.org.uk VirtualHost seems to accomplish the job:

FastCGIExternalServer /srv/pling.org.uk/fcgi-bin -socket /var/lib/fastcgi/chris.sock
ScriptAlias /fcgi-bin /srv/pling.org.uk/fcgi-bin
AddHandler php5-fastcgi .php
Action php5-fastcgi /fcgi-bin virtual

The first path in FastCGIExternalServer is a “virtual” path, and anything that gets mapped to there gets sent to the PHP FastCGI server. The ScriptAlias simply maps that virtual directory to be inside of the root (for some reason putting the virtual directory directly in my document root confused my .htaccess rules sending it into an infinite rewrite loop). The final 2 lines simply tell Apache to deal with .php files as “php5-fastcgi” and to map this “php5-fastcgi” type to our fcgi-bin script alias, so all .php files get sent to FastCGI.

3 comments.