Using an attack on freedom to attack freedom

Following the violent attack against the staff at the Charlie Hebdo offices in Paris, we've now got David Cameron calling for more powers to snoop on people.  Here's a transcript of his speech:

In our country, do we want to allow a means of communication between people that even in extremis, with a signed warrant from the home secretary personally, that we cannot read? Up until now, governments have said no, we must not.

That is why, in extremis, it has been possible to read someone’s letter, to listen to someone’s call, to mobile communications … We have a better process for safeguarding this very intrusive power than probably any other country i can think of.

But the question is are we going to allow a means of communications which it simply isn’t possible to read. My answer to that question is: no, we must not. The first duty of any government is to keep our country safe. The attacks in Paris demonstrated the scale of the threat that we face and the need to have robust powers through our intelligence and security agencies in order to keep our people safe.

The powers that I believe we need, whether on communications data, or on the content of communications, I feel very comfortable these are absolutely right for a modern, liberal democracy.

Analysis

I want to analyse what he's getting at, so I'll take the above transcript a bit at a time:

Do we want to allow a means of communication between people ... that we cannot read?

Cameron is quite specific here - do we want to allow a means of communication, not "do we want to find ways to spy on communication". It seems to me that this directly implies not allowing means of communication between people that the security services can't read - i.e. banning such mechanisms.

Up until now, governments have said no, we must not. That is why ... it has been possible to read someone’s letter, to listen to someone’s call, to mobile communications

I think it can be argued that previous governments haven't said "no, we must not" at all.  There's an important distinction between the examples he cites and the "means of communication" that he is proposing outlawing:

• A letter is sent in the clear (that is: unencrypted), and is handled by a third party (the post office).  If the security services want to read it, they can present a warrant to the post office to intercept it and simply open the letter and read it.
• A land-line telephone call is similar - it is sent in the clear and handled by the telephone company.  If the security services want to listen to it, again they can get the telephone company to intercept it for them.
• Mobile calls are slightly different - the call is encrypted as it is sent over the air.  But importantly, the telephone company decrypts it and, like a land-line call, they handle it in the clear and it can be intercepted in just the same way.

The distinction I'm getting at is that all of these means of communication are already readable, the government has just legislated access rights to the communications that are being handled by third parties.  You can send encrypted data through the post and the government won't be able to read it - no previous governments have ever legislated to stop this.

But the question is are we going to allow a means of communications which it simply isn’t possible to read. My answer to that question is: no, we must not.

He's not really added any new information here, he's just reinforcing the point that he wants to ban all means of communication that can't be spied on.

The attacks in Paris demonstrated the scale of the threat that we face and the need to have robust powers through our intelligence and security agencies in order to keep our people safe.

The attacks in Paris do indeed demonstrate the scale of the threat that we face - as horrible as the attacks were, the scale of the threat boils down to "pretty insignificant".  Lets compare it to a few other random statistics from the UK in 2012:

• 552 people were murdered
• 5,981 people aged 15 or over committed suicide
• 1,496 people died from drug misuse

And these are just a few fully preventable examples... we can also have a brief look at casualties that could certainly be reduced but aren't totally preventable: 1,754 people died from traffic accidents and a whopping 64,164 from heart disease.

In the same year, it seems there were no terrorism related deaths in the UK.  In fact, in the past 5 years there have only been 2 people killed in the UK through terrorism.

Secondly, the Paris attacks are a pretty bad excuse to extend surveillance powers - all of the attackers were already known to the security services, they just didn't have the resources to keep tabs on all suspects all of the time.  Additional surveillance powers wouldn't help here - more resources would.  But then, given the above figures, how about we instead spend some of those resources on road safety, cardiovascular treatment, drug education, mental health treatment, general police work and the hundreds of thousands of other things that are a more significant threat than terrorism?

The people at Charlie Hebdo were attacked because they were exercising their right to freedom, and here we have the British Prime Minister using them as justification for removing our freedoms.

Solutions?

So given that David Cameron wants to outlaw any communications that the government can't read, he's got a few options.  Its important to realise that encryption is used by pretty much everyone in their day-to-day lives, and most of it is actually pretty strong and not something the government can routinely break.

1. Just outlaw encryption

Lets say Cameron outlaws encrypted communications entirely.  So we can no longer do any online financial transactions - no online shopping, no online banking - doing these without using encryption would leave everyone way too vulnerable to criminals.

There are a lot of technologies, such as Chip & Pin, Oyster cards, etc. that also require encryption, but maybe he would make exceptions for that kind of stuff?

I wouldn't get much work done either - a big chunk of my work involves administering servers all over the UK from the comfort of my office.  But that involves me using encrypted connections to those servers - using unencrypted connections would leave the servers very vulnerable to being taken over by criminals.  So most of my time would be spent driving all over the country to do administration tasks that would usually just take me 5 minutes to do from my desk.  I don't really want to think of the pollution implications of this amount of driving either.

Of course, using most foreign online services wouldn't be possible since many of them require encryption and the operators wouldn't be subject to UK law requiring them to implement an encryption-free version (with all the horrible security nightmares that would entail).

So anyway, we'll assume all the law abiding citizens are not using encryption at all.  I'm sure the terrorists who think that it's ok to commit mass murder are going to comply with that law and avoid encryption too so that the authorities can spy on them.  Or maybe not...

A great idea though - maybe the authorities can spot the terrorists by seeing who is using encryption!  Well no, as it turns out, they can't - by using a combination of steganography and an old encryption method called a one time pad, anyone could send encrypted communications that are mathematically provable to be undetectable... so bang goes that theory.

In a world where encryption is outlawed, only outlaws will have encryption.

2. Outlaw strong encryption

We could have legislation that allows encryption, but only encryption that is weak enough for the security services to break.  This has most of the same problems as outlawing encryption entirely - you can be sure that if the security services can crack it, so can the criminals, and once again the outlaws themselves can continue to use strong crypto since they don't care about the law.

3. Key escrow

Probably the best option is key escrow - this is where you can continue to use encryption as normal, but everyone is legally required to hand their encryption keys over to the government.

If the keys ended up being leaked out to third parties then everyone would be screwed, of course.  It would take a massive rewrite of pretty much every piece of software that employs encryption, which seems quite unfeasible.

And again, since the terrorists don't care about the law, there's no reason why they would comply with it and hand over their keys; and as previously mentioned, there would be no way to detect that this is happening.

Summary

So there we go, Cameron has said what he wants to do, but it seems that it is all pretty much unfeasible.  Which I guess is a relief, but I suspect that this won't prevent them passing ill-conceived legislation that erodes civil liberties whilst providing no additional security.

Corollary: this is what you get when you have politicians legislating about technologies they don't understand.

The Npower saga continues

So it turns out that Npower informed the ombudsman that they had complied with the order on November 27th.  They said:

In accordance with the remedy I can confirm that we have

• Credited account for missed discounts.
• Credited disputed amount to the account to clear balance.
• Credited your account as a gesture of goodwill.
• Refunded remaining credit by cheque, you will receive this in the post in the next 7-10 working days.
• Please accept my apologies for the delay in issuing you with a correct statement and for the issues surrounding your discount.

Except I haven't received the letter or the cheque and Npower's customer services have confirmed that no cheque has been sent out.  They do claim to have complied with the resolution by crediting my (closed) Npower account, even though they have completely failed to send me an actual refund!

So as far as I can tell, Npower acknowledged the Ombudsman's orders, decided not to comply and then lied to the Ombudsman to say they had complied.  At least, I can't think of any other reason of saying "we have" complied instead of "we will" comply if they hadn't actually already done it.

So to recap, here's why I'm never going to do business with Npower again:

1. They never actually sent me any kind of contract when I opened an account with them.
2. In January 2013 they put me on the wrong tariff... of course they didn't actually tell me this immediately.
3. They set up the direct debit but didn't actually withdraw any funds from it... and again, I didn't get informed of this immediately, and hadn't noticed.
4. In July 2013 I got my first bill, which came as a shock since it was for a more expensive tariff and none of it had been paid by the direct debit they were supposed to be using.
5. Their complaints people initially said that (i) yes they had screwed up and put me on a more expensive tariff, (ii) no they weren't going to fix it or refund me because their internal systems wouldn't let them, (iii) the direct debit not being used was clearly my fault as apparently the customer is responsible for checking their bank account every month to make sure Npower have done their job correctly.  After some argument they agreed that I could cancel my direct debit until things got resolved so I didn't have to over-pay them.
6. They said they would call me back within 10 days... they didn't.
7. They started sending me nastygrams because I hadn't paid the overbilled account (for which my complaint was still being "processed").
8. They informed me that I wouldn't get my direct debit discount since I had cancelled my direct debit (with their permission).  Eventually they agreed I could have it.  I paid off the hundreds of pounds of debt that had been racked up by their failure to use the direct debit.
9. In January 2014 I found that they had "forgotten" to apply my direct debit discount.  I called them and they confirmed it had been forgotten and said they had now credited it to my account but I would need to wait 6 months for my next bill for it to be applied.
10. In May 2014 I got a demand to pay hundreds of pounds within 9 days... Yet again they had decided to simply stop using the direct debit for no reason, and waited until this had been going on for months before informing me with a "pay within 9 days or else" nastygram.
11. The still hadn't credited the direct debit discount that they said they had in January... they said it was still being "processed" - apparently it takes 6 months to "process" a credit.
12. They now swore blind that I wasn't due a direct debit discount at all, despite having previously said that they had credited it.
13. I raised a formal complaint with Npower and got a automated reply saying they would get back to me within 10 days.
14. They didn't get back to me at all, and in fact ignored all my further attempts to raise a complaint too.
15. They started sending increasingly threatening nastygrams because I refused to pay the overbilled amount.
16. The debt collection department said they couldn't do anything to stop legal action and that I would need to raise a complaint about the disputed amount.  When I pointed out I had tried to raise a complaint and had been repeatedly ignored, I was told that I should probably raise a complaint about that (!)
17. I referred the whole thing over to the Energy Ombudsman, who quickly decided on a "remedy" and ordered Npower to comply by December 24th.  Npower sent a response to the Ombudsman to say they had complied on November 27th.
18. Part of the resolution (which Npower told the Ombudsman had been actioned) was to send me a refund.  Npower have confirmed to me that this was never done.
19. Npower still claim they complied with the Ombudsman's orders on time, even though they also confirm that they never sent the refund (!).
20. They don't really seem to be in a rush to fix things - apparently I have to wait another 10 days for them to send a cheque because they can't do a bank transfer.  So assuming they actually manage to do what they say they're doing (which would surprise me, given everything else that's happened), that means I'll get my refund on January 19th, 26 days after the Ombudsman's deadline and a year after I should have received it!

Npower refusing to comply with orders from the ombudsman

I have to say I've been quite impressed so far with the Energy Ombudsman's handling of my Npower case.  On 26th November they ordered Npower to:

• Send me a letter of apology
• Credit my account with the discounts that I was entitled to, which they failed to apply (the Ombudsman actually calculated this to be slightly more than I had)
• Ensure that an unexplained bill adjustment is corrected
• Credit the account with a "goodwill" payment
• Refund me all of the remaining credit on the account

The goodwill payment wasn't as much as I'd hoped, given the amount of time and inconvenience that they had cost me, but all in all this seemed not a bad resolution to the problem.

Npower were required to comply with this order by Christmas Eve...  so today I've written to the ombudsman to report that they haven't bothered to comply...  I've had no communication from Npower at all since the ombudsman issued the order 5 weeks ago.

With levels of competence like this, I do wonder why the regulator doesn't just rescind Npower's licence.

Are exclusivity deals good for the consumer?

Opendium has been going for over 9 years now, and over that time we've gained a number of schools as very happy customers through word of mouth (and as a testament to their satisfaction, no school has ever left us!)  We've spent that time working with our customers to build a very capable product, which is also somewhat cheaper than most of our competitors, and we're actively working to promote our product to more schools.

We're primarily marketing to independent schools, since they have much more freedom to make their own decisions, and there are a number of organisations that represent the British independent schools which we're actively engaging with.  This should be good for both us and their members.  Just last month we sponsored and attended the Welsh Independent Schools Council's annual conference, which was a good experience for us and brought up some interesting ideas for our product roadmap.  In fact, we've already implemented some of those ideas, and they are now going through the testing phase of our development cycle.

However, I was surprised by the Independent Schools Association's attitude when we contacted them - they refuse to work with us as they say they already have exclusive contracts with "preferred suppliers".  They are supposed to be working for their members, which seems at odds with any kind of supplier exclusivity.  Competition is almost always good for the consumer - it leads to innovation and lower prices, and conversely exclusivity almost always leads to stagnation and high prices.  Surely if they truly are working for the interests of their members, they would be trying to foster as much competition as possible and giving their members a wide variety of suppliers to choose from to meet their individual needs?

Thankfully, ISA's attitude doesn't seem to be shared by the other people that we have been in discussions with, so we're looking forward to working with them and benefiting their members.

Small suppliers picking up the tab for support

I've been having some thoughts about how the support load is distributed between large suppliers such as Microsoft, Apple and Google vs. smaller suppliers such as ourselves.

It seems that people buy in services from big providers, but when problems hit they can't get the necessary level of support from them, so turn to the smaller providers with whom they have a not-entirely-related support contract.

This happens to us all the time - we have all manor of kludges in our software to make Apple devices work reliably with it due to bugs in Apple's software, for example.  In an ideal world, the customer would call Apple and Apple would diagnose the problem (possibly with our help) and fix their software.  In a less than ideal world, we would do a temporary work-around to get our customers up and running, then report these bugs to Apple who would fix them.  Back in the real world, Apple never fix the bugs and the work-arounds become permanent bodges that are an ongoing minefield for us.

We used to report all the Apple bugs we found to Apple with the expectation that they would be interested in fixing them.  These days we don't bother - they have never shown any interest in fixing a bug we've reported.  Usually reporting went like this: after spending hours diagnosing a problem, we send them a comprehensive report of what's happening, how to reproduce it, often with network traffic dumps clearly showing the problem.  They respond asking for exactly the same information as we just provided, but in a different format.  So we spend hours reproducing the problem again, send them everything they asked for and never hear anything back.  I've got no problem with spending some time collecting information for them if they are actually going to use it, but it's a complete waste of our time to do debugging that they will ignore every time.

We're currently having problems with Microsoft's web servers - they have a bug in them that means certain clients can't connect (pretty much anything using OpenSSL on Scientific Linux 6.5).  In particular, our proxy server software can't connect without some work-arounds.  There is no well publicised address for reporting bugs, but we found a promising looking address and sent a comprehensive bug report.  We even prefaced the report with a "if this is the wrong address, please forward it on to the right department" note.  Instead, we have simply been bounced from department to department, many refusing to hand out email addresses and instead insisting on us phoning - the phone operators are completely ill-equipped to handle this kind of bug report and inevitably bounce us on to another department.

So much like Apple, Microsoft seem disinterested in actually looking at bug reports.  Limited experience of dealing with Google is much the same - they are just too big to be interested in resolving problems that don't affect hundreds of thousands of customers.

So we're back to my initial thoughts - customers buy expensive products from the big guys, who pocket the profits and refuse to support them properly.  Leaving us to have to pick up the pieces despite it not really being part of our remit, because telling a customer "we're not going to help you" isn't really an option for us.  Yet somehow, when we are unable to work around the problems it is somehow seen as our fault and reflects badly on us - no one ever stops buying from the big guys because of this stuff.

ICO correspondence

Many people don't realise, but sending unsolicited email is unlawful here in the UK.  There are several ways companies go about doing bulk email marketing:

1. Collect the recipient's details via an existing business transaction, giving them the opportunity to opt-out of marketing emails at the same time.
2. Collect the recipient's details via an existing business transaction, giving them the opportunity to opt-in to marketing emails.
3. Buying/acquiring a mailing list from someone else.

The Privacy and Electronic Communications (EC Directive) Regulations 2003 states that you're on dodgy ground if you do (1).  If you do (2) then you can only send marketing regarding "similar products and services".  Doing (3) is never lawful.  Also, any marketing is required to contain a valid address that recipients can contact the sender on.

Whenever I give a company my details, I always ensure that I opt-out of marketing if there's an option to do so, and I never opt-in, so in theory I should get no spam.  Unfortunately, these regulations are widely disregarded, even by big corporations, so I send a standardised response to spam email that I receive from British companies (usually to several of their email addresses):

This is an unsolicited communication by means of electronic mail transmitted to an individual subscriber for direct marketing purposes. This is contrary to section 22 of The Privacy and Electronic Communications (EC Directive) Regulations 2003.

http://www.legislation.gov.uk/uksi/2003/2426/regulation/22/made

Please do not send any further unsolicited emails. A charge of £25 per email will be made for any further unsolicited emails received and your sending of any such emails will be deemed as acceptance of these terms.

I am also making a subject access request under Section 7 of the Data Protection Act 1998 for all the data / information you hold on me and from where you obtained it.

http://www.legislation.gov.uk/ukpga/1998/29/section/7

I suggest you remove me from your list and review your marketing methods with a qualified lawyer. Please confirm the receipt of this email. Failure to respond will result in your organisation being reported to the Office of the Information Commissioner.

I want to know how they came upon my details and why they think I've opted in to their spam, so as part of the above email I make a subject access request (SAR) under the data protection act - companies have a maximum of 40 days to respond.  Usually I get no response at all, and usually the spamming continues.

At the weekend I tidied up my email a bit, and took the opportunity to actually file complaints with the information commissioner's office.  They have the power to follow up these complaints and fine the companies responsible.

In total I made 5 complaints under the PECR - these were companies who had spammed me, been sent the above warning and had continued to send spam regardless.  I also made 6 complaints under the DPA - companies who had spammed me and had not responded to the SAR.

All of the complaints I filed followed the same format - I filled in the appropriate form provided by the ICO and attached it to an email as a PDF.  o the same email I attached all of the relevant emails I had sent or received as message/rfc822 attachments - this means they include all of the headers added by the email client and email servers.

Today the ICO sent me their first response - they tell me they can't investigate my DPA complaint against Halfords because I didn't include any of the email headers and therefore they don't know what date I made the SAR on...  I'm not sure if they're incompetent or looking for an excuse to not do their job - all the emails I forwarded to them had the complete headers.

Diagnosing Sharepoint Breakage

Every so often you get a proper puzzle to solve, and this morning is one of those times. One of our customers reported that they were unable to contact the Microsoft Sharepoint servers through their proxy server, a quick test on my test system confirmed the same issue so we spent about 3 hours delving right into the nitty gritty to figure out what was going on.

The proxy was reporting "connection reset by peer" during the TLS handshake - TLS (Transport Layer Security) is the cryptography protocol used to secure HTTPS web sites, and TLS problems tend to be a pain since the OpenSSL library usually doesn't give especially verbose error messages.  It was clear this wasn't going to be a trivial problem to solve so we immediately disabled HTTPS interception for the Sharepoint site to get it up and running again.  Customer confirmed that this had resolved the issue, so that takes the pressure off a bit but raises a question: why is it working ok when the browser is negotiating the encryption, but not when the proxy is negotiating?

The first port of call was to capture some network traffic and load it into Wireshark for analysis.  This showed that the proxy is sending a TLS "Client Hello" handshake, the server was returning a TCP ACK, but no TLS response.  30 seconds later the server tears down the connection with a TCP RST.  The ACK confirms that the server got the "Client Hello", and you'd usually expect the response to be sent in the same packet as the ACK so it looked like the packet wasn't being dropped by intermediate network hops - the server simply was never sending a handshake response.

Time to make things simpler - instead of using the proxy server, lets ask OpenSSL to connect directly:

openssl s_client -showcerts -connect 157.55.229.87:443

This failed in the same way when we tried it on the test server, but succeeded when run on my Fedora workstation.  Comparing the network traffic between the working and non-working tests showed that the most obvious different was that the non-working handshake presented a few more ciphers for the server to choose from - maybe one of those extra ciphers was confusing the Sharepoint server.

We tried adjusting the list of cipher suites, but each time we tried we found that the request succeeded and we couldn't pin down anything specific that would break it.  We needed to start with the broken handshake and edit it bit by bit until it started working - that would let us figure out specifically what needed to change to make it work.

So we took the captured network traffic and dumped it out as hex:

tcpdump -r capture.pcap -x > capture.hex

We're not interested in the TCP layer stuff, so the first three packets can be ignored (SYN, SYN ACK, ACK) - these are the normal TCP three-way handshake.  The next packet contains the "Client Hello" which we're interested in, but it also contains the Ethernet, IP and TCP headers.  Using Wireshark it's trivial to identify the start of the payload, and we just trimmed everything before that off the hex dump.

Now to replay it and make sure it still fails:

(sed -e 's/#.*$//' capture.hex | xxd -r -p ; sleep 5) | nc 157.55.229.87 443

The sed bit at the start just strips off anything after a # so we can put comments in the hex file.  xxd converts it back into binary and we used nc to connect to the web server and send the data.

We checked the traffic in Wireshark - all looks as expected and the web server still didn't respond, so far so good.

Again, using Wireshark we can identify the various parts of the packet, and set about modifying them.  Of interest are four headers indicating the length of various sections - the TLS Record Layer has an overall length header, within that there is the "Client Hello" data which has its own length header, and within the "Client Hello" are a cipher suite list and an extension list, which again have their own headers indicating their respective lengths.  Each length header is 16 bits long, so can contain a value of up to 65535.

As mentioned, we were interested in the cipher suites - in particular the extra ones that were presented in the broken handshake but not in the working one.  So we set about removing them one by one - each cipher suite is 16 bits long, so removing it involves deleting it from the cipher suite list, and then reducing the cipher suite length, client hello length and tls record length headers by 2 each.

Each time we removed a cipher suite, we replayed the data to the server and looked to see what happened.  After removing two cipher suites, the server suddenly started responding with a "Server Hello"!  We put these ciphers back and removed two others so see if it was specifically one of those ciphers confusing the server, but that didn't break anything again - the server was still happy.

The broken handshake that we started out with had a TLS record length of 258 octets and removing two ciphers (16 bits each) reduced it to 254 - a number that will fit in a single octet, whereas 258 requires two octets.  So we tried adding all the ciphers back in and removing one of the records from the extensions list (5 octets) instead.  Again, the server responded and was happy.

So there we go.  It looks like Microsoft's Sharepoint server has a bug in it that breaks any client that tries to handshake with a TLS record more than 255 octets long.  Evidently the proxy presents a larger selection of cipher suites to the server than most web browsers, so it works fine from the browser but not from the proxy.

We have contacted Microsoft, although I have no idea if we've contacted the right department but hopefully it will get passed on to the right people.