Datamining the school run

On some initial surveys, Clifton High appeared to be free of parents exploiting the school-keep-clear section. But it was just their school starts early, at 0830.

Here, on Oct 8, at 08:39, One car doing a dropoff in the defacto-dropoff-zone: WU56LUD

Here, on Oct 9, at 08:28, a car doing a dropoff in the defacto-dropoff-zone: WU56LUD

This car is are now, as they say, "in the database". Not some toy little local government database running on a ropy server in an undercooled attic. Not some secret central goverment server farm somewhere of car number plates -no, they are in a bigger database: the Web. In one of Google's datacentres.

Photos are uploaded to Picasa, running on the Google File System. That also hosts the blog, routing the click log back into GFS for Google Analytics to analyse the visitors, creating stats that get stored in BigTable. All words in the blog, including registration numbers get picked up in their PageRank scans, probably implemented as a big MapReduce job, with the results again stored in GFS and BigTable. Which is why, a few hours after this posting, a search for the word will not return "Your search - WU56LUD - did not match any documents.", as it does now, it will return this page -and any others we add later.

That's our database. A combination of photo, tags and text marks up all cars whose driving/parking skills are felt to be significantly entertaining to merit attention, and then, when the time comes to visit the particular schools, we just retrieve all entries tagged with that school, paste the photos into powerpoint, and we have the list of repeat offenders we expect them to deal with.

Which is why it is important to thank people for their contributions. That's the photographers, but also the subject. Thank them, after they've had their car/bus/van snapped, because without their actions, we wouldn't have anything interesting to write up, and the database would be that much less useful. And Google's BigTable would have less number plates in its columns.