Taking a break from traffic issues, we should note that Bristol Traffic team has long admitted to building a mass surveillance police state in conjunction with google and facebook —our Datacentre State.
In fact the main difference between us and GCHQ is ours is run from an Ubuntu laptop in the comfy sofa bit of the Canteen. That's just down the road from this painting, behind the riot police in the distance
Now that the new Mass Surveillance State bill is up, we should do a post on how we would implement it and cost it out accordingly. Some request logging -> Apache Kafka -> Hadoop HDFS pipeline with hourly scheduled MapReduce or Spark jobs compressing the time-series logs a compact and fast-to-scan format like Parquet or Orc. This could then be queried direct via Facebook's Hive, or imported into NSA's open sourced Accumulo column table DB for even faster lookup. Each ISP/mobile telco may host their own "facility", but sticking them all in the same datacentre would ease low-latency cross-ISP queries issued from government computers, while still pretending they were "separate"
In the meantime, let's pick on some talking points that are being used on the radio and TV to justify the bill and make it look like the government listened to feedback
The nature of technology has changed and we must adapt.
People have been browsing the web for 20 years, even skype is about twelve years old. What has changed is the cost of storage. Back in 2008 we were quoting a few hundred dollars for aterabyte. seven years later and the cost is $30/TB and density shrunk to the extent you can get a couple of petabytes in two wardrobe-sized server racks. That's the big change: governments can afford to store all your personal data.
The pages you visit won't be recorded, only the sites.
With the migration of the main web sites to HTTPS, the ISPs couldn't log the pages anyway. There's no concession here: if your browser shows a little green lock in the URL line, the government couldn't record the page. What they can do now is go to facebook and say "Someone at 10.0.1.1 went to fb.com at 21:14 on Tuesday: what did they do?" Facebook, will have the rest of the information for them.
This is just like an itemised phone bill.
No. It's like a log of every game you played on your PS3, every program you watched on BBC iPlayer, every photo you took which your smart phone backed up (and where). If you read books on an Amazon kindle —or with the app— its a log of whenever you turned a page or turned back. Spent too much time reading "extremist" bits of the Koran between bouts of Call of Duty and facebook posts? That'll be something they'll be able to work out by looking at the URLs and then asking the service providers for the details. Here Sony may come out the best —unless they start recording chat sessions. Amazon? They'll probably record the ambient light and tablet rotation while you were reading those chapter of the Koran.
We won't ban encryption
They'd only be laughed at if they asked for this. The algorithms (RSA, Elliptic Curve Cryptography) are well known. You can't stop RSA working without banning prime numbers. ECC is potentially even harder. though the fact that NSA are no longer recommending is use implies they don't trust it any more. Either they've found some new math or built some new hardware ... so longer key RSA is back in fashion. All the homeoffice can do is go to FB, google, Whatsapp and say "please store the communications so we can ask for it", then drive round to Apple and say "add a back door to iPhone encryption —we promise we won't abuse it, lose the secret key or otherwise destroy its value.
There is some mention of "informal arrangements" perhaps the government has had meetings with all these people, and said, "give us access and we won't review your tax status". But that isn't going to work with those companies that don't have a UK outpost who can hang up the phone when Theresa rings them. Note especially that some of the best cryptography libraries, Bouncy Castle are explicitly developed in Australia to avoid US regulations on RSA key lengths. And guess what's been ported to Android? Building an Android device-device app with unbreakable encryption is straightforward enough to make it a final year project for a Computer Science course at any of our local universities —how could that be criminalised?
We're only formalising what's been going on.
Ignoring the fact that this implies that previous governments have clearly been granting warrants to log the actions of every citizen, the fact that they've been doing this is a key part of the UK-side of the Snowden leaks. In the US this has led to a rethink of state/citizen rights. Here its leading to the government not only formalising the existing state of affairs, but expanding it.
We won't monitor MPs communications
Bulk data collection renders this impossible. How you know that the person posting to twitter from an internet cafe is an MP or a possible enemy of the state? You can't, you just grab it all.
The core concessions aren't concessions, they are the result of the engineering teams of the government and the ISPs telling them what doesn't work, and the politicians coming up with ways to frame this in terms of concessions, rather than acceptance of engineering and cross-border realities. They've also hidden the key implication: they can now afford to record every single interaction you make with a remote computer, and, with informal and formal arrangements with the providers of those services, get the details.
Meanwhile, your civil liberties have been suspended for the duration of the emergency.