Npower refusing to comply with orders from the ombudsman

I have to say I've been quite impressed so far with the Energy Ombudsman's handling of my Npower case.  On 26th November they ordered Npower to:

• Send me a letter of apology
• Credit my account with the discounts that I was entitled to, which they failed to apply (the Ombudsman actually calculated this to be slightly more than I had)
• Ensure that an unexplained bill adjustment is corrected
• Credit the account with a "goodwill" payment
• Refund me all of the remaining credit on the account

The goodwill payment wasn't as much as I'd hoped, given the amount of time and inconvenience that they had cost me, but all in all this seemed not a bad resolution to the problem.

Npower were required to comply with this order by Christmas Eve...  so today I've written to the ombudsman to report that they haven't bothered to comply...  I've had no communication from Npower at all since the ombudsman issued the order 5 weeks ago.

With levels of competence like this, I do wonder why the regulator doesn't just rescind Npower's licence.

Are exclusivity deals good for the consumer?

Opendium has been going for over 9 years now, and over that time we've gained a number of schools as very happy customers through word of mouth (and as a testament to their satisfaction, no school has ever left us!)  We've spent that time working with our customers to build a very capable product, which is also somewhat cheaper than most of our competitors, and we're actively working to promote our product to more schools.

We're primarily marketing to independent schools, since they have much more freedom to make their own decisions, and there are a number of organisations that represent the British independent schools which we're actively engaging with.  This should be good for both us and their members.  Just last month we sponsored and attended the Welsh Independent Schools Council's annual conference, which was a good experience for us and brought up some interesting ideas for our product roadmap.  In fact, we've already implemented some of those ideas, and they are now going through the testing phase of our development cycle.

However, I was surprised by the Independent Schools Association's attitude when we contacted them - they refuse to work with us as they say they already have exclusive contracts with "preferred suppliers".  They are supposed to be working for their members, which seems at odds with any kind of supplier exclusivity.  Competition is almost always good for the consumer - it leads to innovation and lower prices, and conversely exclusivity almost always leads to stagnation and high prices.  Surely if they truly are working for the interests of their members, they would be trying to foster as much competition as possible and giving their members a wide variety of suppliers to choose from to meet their individual needs?

Thankfully, ISA's attitude doesn't seem to be shared by the other people that we have been in discussions with, so we're looking forward to working with them and benefiting their members.

Small suppliers picking up the tab for support

I've been having some thoughts about how the support load is distributed between large suppliers such as Microsoft, Apple and Google vs. smaller suppliers such as ourselves.

It seems that people buy in services from big providers, but when problems hit they can't get the necessary level of support from them, so turn to the smaller providers with whom they have a not-entirely-related support contract.

This happens to us all the time - we have all manor of kludges in our software to make Apple devices work reliably with it due to bugs in Apple's software, for example.  In an ideal world, the customer would call Apple and Apple would diagnose the problem (possibly with our help) and fix their software.  In a less than ideal world, we would do a temporary work-around to get our customers up and running, then report these bugs to Apple who would fix them.  Back in the real world, Apple never fix the bugs and the work-arounds become permanent bodges that are an ongoing minefield for us.

We used to report all the Apple bugs we found to Apple with the expectation that they would be interested in fixing them.  These days we don't bother - they have never shown any interest in fixing a bug we've reported.  Usually reporting went like this: after spending hours diagnosing a problem, we send them a comprehensive report of what's happening, how to reproduce it, often with network traffic dumps clearly showing the problem.  They respond asking for exactly the same information as we just provided, but in a different format.  So we spend hours reproducing the problem again, send them everything they asked for and never hear anything back.  I've got no problem with spending some time collecting information for them if they are actually going to use it, but it's a complete waste of our time to do debugging that they will ignore every time.

We're currently having problems with Microsoft's web servers - they have a bug in them that means certain clients can't connect (pretty much anything using OpenSSL on Scientific Linux 6.5).  In particular, our proxy server software can't connect without some work-arounds.  There is no well publicised address for reporting bugs, but we found a promising looking address and sent a comprehensive bug report.  We even prefaced the report with a "if this is the wrong address, please forward it on to the right department" note.  Instead, we have simply been bounced from department to department, many refusing to hand out email addresses and instead insisting on us phoning - the phone operators are completely ill-equipped to handle this kind of bug report and inevitably bounce us on to another department.

So much like Apple, Microsoft seem disinterested in actually looking at bug reports.  Limited experience of dealing with Google is much the same - they are just too big to be interested in resolving problems that don't affect hundreds of thousands of customers.

So we're back to my initial thoughts - customers buy expensive products from the big guys, who pocket the profits and refuse to support them properly.  Leaving us to have to pick up the pieces despite it not really being part of our remit, because telling a customer "we're not going to help you" isn't really an option for us.  Yet somehow, when we are unable to work around the problems it is somehow seen as our fault and reflects badly on us - no one ever stops buying from the big guys because of this stuff.

ICO correspondence

Many people don't realise, but sending unsolicited email is unlawful here in the UK.  There are several ways companies go about doing bulk email marketing:

1. Collect the recipient's details via an existing business transaction, giving them the opportunity to opt-out of marketing emails at the same time.
2. Collect the recipient's details via an existing business transaction, giving them the opportunity to opt-in to marketing emails.
3. Buying/acquiring a mailing list from someone else.

The Privacy and Electronic Communications (EC Directive) Regulations 2003 states that you're on dodgy ground if you do (1).  If you do (2) then you can only send marketing regarding "similar products and services".  Doing (3) is never lawful.  Also, any marketing is required to contain a valid address that recipients can contact the sender on.

Whenever I give a company my details, I always ensure that I opt-out of marketing if there's an option to do so, and I never opt-in, so in theory I should get no spam.  Unfortunately, these regulations are widely disregarded, even by big corporations, so I send a standardised response to spam email that I receive from British companies (usually to several of their email addresses):

This is an unsolicited communication by means of electronic mail transmitted to an individual subscriber for direct marketing purposes. This is contrary to section 22 of The Privacy and Electronic Communications (EC Directive) Regulations 2003.

http://www.legislation.gov.uk/uksi/2003/2426/regulation/22/made

Please do not send any further unsolicited emails. A charge of £25 per email will be made for any further unsolicited emails received and your sending of any such emails will be deemed as acceptance of these terms.

I am also making a subject access request under Section 7 of the Data Protection Act 1998 for all the data / information you hold on me and from where you obtained it.

http://www.legislation.gov.uk/ukpga/1998/29/section/7

I suggest you remove me from your list and review your marketing methods with a qualified lawyer. Please confirm the receipt of this email. Failure to respond will result in your organisation being reported to the Office of the Information Commissioner.

I want to know how they came upon my details and why they think I've opted in to their spam, so as part of the above email I make a subject access request (SAR) under the data protection act - companies have a maximum of 40 days to respond.  Usually I get no response at all, and usually the spamming continues.

At the weekend I tidied up my email a bit, and took the opportunity to actually file complaints with the information commissioner's office.  They have the power to follow up these complaints and fine the companies responsible.

In total I made 5 complaints under the PECR - these were companies who had spammed me, been sent the above warning and had continued to send spam regardless.  I also made 6 complaints under the DPA - companies who had spammed me and had not responded to the SAR.

All of the complaints I filed followed the same format - I filled in the appropriate form provided by the ICO and attached it to an email as a PDF.  o the same email I attached all of the relevant emails I had sent or received as message/rfc822 attachments - this means they include all of the headers added by the email client and email servers.

Today the ICO sent me their first response - they tell me they can't investigate my DPA complaint against Halfords because I didn't include any of the email headers and therefore they don't know what date I made the SAR on...  I'm not sure if they're incompetent or looking for an excuse to not do their job - all the emails I forwarded to them had the complete headers.

Diagnosing Sharepoint Breakage

Every so often you get a proper puzzle to solve, and this morning is one of those times. One of our customers reported that they were unable to contact the Microsoft Sharepoint servers through their proxy server, a quick test on my test system confirmed the same issue so we spent about 3 hours delving right into the nitty gritty to figure out what was going on.

The proxy was reporting "connection reset by peer" during the TLS handshake - TLS (Transport Layer Security) is the cryptography protocol used to secure HTTPS web sites, and TLS problems tend to be a pain since the OpenSSL library usually doesn't give especially verbose error messages.  It was clear this wasn't going to be a trivial problem to solve so we immediately disabled HTTPS interception for the Sharepoint site to get it up and running again.  Customer confirmed that this had resolved the issue, so that takes the pressure off a bit but raises a question: why is it working ok when the browser is negotiating the encryption, but not when the proxy is negotiating?

The first port of call was to capture some network traffic and load it into Wireshark for analysis.  This showed that the proxy is sending a TLS "Client Hello" handshake, the server was returning a TCP ACK, but no TLS response.  30 seconds later the server tears down the connection with a TCP RST.  The ACK confirms that the server got the "Client Hello", and you'd usually expect the response to be sent in the same packet as the ACK so it looked like the packet wasn't being dropped by intermediate network hops - the server simply was never sending a handshake response.

Time to make things simpler - instead of using the proxy server, lets ask OpenSSL to connect directly:

openssl s_client -showcerts -connect 157.55.229.87:443

This failed in the same way when we tried it on the test server, but succeeded when run on my Fedora workstation.  Comparing the network traffic between the working and non-working tests showed that the most obvious different was that the non-working handshake presented a few more ciphers for the server to choose from - maybe one of those extra ciphers was confusing the Sharepoint server.

We tried adjusting the list of cipher suites, but each time we tried we found that the request succeeded and we couldn't pin down anything specific that would break it.  We needed to start with the broken handshake and edit it bit by bit until it started working - that would let us figure out specifically what needed to change to make it work.

So we took the captured network traffic and dumped it out as hex:

tcpdump -r capture.pcap -x > capture.hex

We're not interested in the TCP layer stuff, so the first three packets can be ignored (SYN, SYN ACK, ACK) - these are the normal TCP three-way handshake.  The next packet contains the "Client Hello" which we're interested in, but it also contains the Ethernet, IP and TCP headers.  Using Wireshark it's trivial to identify the start of the payload, and we just trimmed everything before that off the hex dump.

Now to replay it and make sure it still fails:

(sed -e 's/#.*$//' capture.hex | xxd -r -p ; sleep 5) | nc 157.55.229.87 443

The sed bit at the start just strips off anything after a # so we can put comments in the hex file.  xxd converts it back into binary and we used nc to connect to the web server and send the data.

We checked the traffic in Wireshark - all looks as expected and the web server still didn't respond, so far so good.

Again, using Wireshark we can identify the various parts of the packet, and set about modifying them.  Of interest are four headers indicating the length of various sections - the TLS Record Layer has an overall length header, within that there is the "Client Hello" data which has its own length header, and within the "Client Hello" are a cipher suite list and an extension list, which again have their own headers indicating their respective lengths.  Each length header is 16 bits long, so can contain a value of up to 65535.

As mentioned, we were interested in the cipher suites - in particular the extra ones that were presented in the broken handshake but not in the working one.  So we set about removing them one by one - each cipher suite is 16 bits long, so removing it involves deleting it from the cipher suite list, and then reducing the cipher suite length, client hello length and tls record length headers by 2 each.

Each time we removed a cipher suite, we replayed the data to the server and looked to see what happened.  After removing two cipher suites, the server suddenly started responding with a "Server Hello"!  We put these ciphers back and removed two others so see if it was specifically one of those ciphers confusing the server, but that didn't break anything again - the server was still happy.

The broken handshake that we started out with had a TLS record length of 258 octets and removing two ciphers (16 bits each) reduced it to 254 - a number that will fit in a single octet, whereas 258 requires two octets.  So we tried adding all the ciphers back in and removing one of the records from the extensions list (5 octets) instead.  Again, the server responded and was happy.

So there we go.  It looks like Microsoft's Sharepoint server has a bug in it that breaks any client that tries to handshake with a TLS record more than 255 octets long.  Evidently the proxy presents a larger selection of cipher suites to the server than most web browsers, so it works fine from the browser but not from the proxy.

We have contacted Microsoft, although I have no idea if we've contacted the right department but hopefully it will get passed on to the right people.

IP Based Controls

Over the years, our Iceni servers have undergone a number of design changes in order to accommodate the changing nature of devices being used on networks.  In particular, authentication of web traffic has needed special attention constantly.

In the old days, software usually had really good support for authenticating with web proxy servers.  Windows clients would silently authenticate each web request using NTLM or Kerberos, non-Windows stuff used HTTP Basic authentication (it pops up a username/password box when you start a session, but you can use the "remember password" checkbox to stop this getting annoying).  Every so often we came across a rare example of software that couldn't handle proxy authentication and we'd have to tweak the proxy configuration a bit to bypass the authentication and filtering, but in general life was good.

We're increasingly seeing software support for web proxy servers getting poorer though - quite a lot of software just plain ignores the system-wide settings and bypasses the proxy, and an increasing amount of software can't handle proxy authentication at all.  In fact, this latter point has often shown just how poorly built some of the modern software is: Windows 8, for example, tries to log in to login.live.com when you log into a machine, and if the proxy asks for authentication the machine hangs and has to be hard-reset!  Apple devices seem particularly bad too - if the proxy asks an iPhone to authenticate when it tries to synchronise its calendar, it just retries immediately, and keeps going indefinitely - hundreds of times a second!

A few years ago we did a lot of work to work around these broken devices:  Firstly we introduced a transparent proxy to deal with the software that completely ignores the proxy settings.  Then we started caching the last known user for each IP address and for client software that's known to be broken, we just reused those details rather than asking them to authenticate.  We also added a captive portal and support for WISPr authentication to help the Apple devices along a bit.

Along the way we his some surprising problems - for example, you would expect every web request to be independent of each other, but we found that if we avoided authenticating certain iPhone traffic, then completely unrelated traffic from that device that would usually work fine suddenly stopped being able to cope with authentication too!

The move to Iceni 2 saw more changes - administrators can now tell the system to only use the captive portal/WISPr authentication for certain problem URIs, or disable authentication entirely in some cases.  For example, by default Iceni 2 servers don't authenticate of filter login.live.com.

All this work has gone a long way to avoiding the problems that were cropping up, but increasingly there's a feeling that things like phones and tablets have such poor support for HTTP proxy authentication that its probably preferable to turn it off entirely for those devices and rely on the captive portal and WISPr.  But how do you do that just for those devices, and not for things like the on-domain Windows machines which still work fine?

This brings me on the the latest stuff I've just finished working on and is now going through QA testing (soon to be released to the customers, all being well!):  We now allow you to define a network - a network address and netmask - and drop it into a user group as if it were a user.  This means you can do stuff like disabling authentication for all devices on a particular network - your wifi network, for example.

The bonus of this is that, if your network is split up appropriately, you can also tweak filtering based on the workstation's location - you can relax the filtering for class rooms that are well supervised, for example.

We've also got rid of the "Guest" user and renamed the "Guests" group to "Anonymous" to better reflect what it means.

Over all I really like the new model, and I have plans to extend it to the mail server component as well.

However, my new job for today is to fix a locking bug in the web filter - joy of joys!

Incompetent billing

I have billing disputes with no less than three separate companies at the moment.  Its depressing that this kind of thing is probably going to go completely unnoticed by a lot of customers and result in these companies actually making money out of their mistakes...

BT

I switched my POTS line away from BT on July 25th.  My annual "line rental saver" contract expired on July 23rd so this should have been ok.  Except BT emailed me to say I was going to be billed £22.22 as an "early cancellation charge" because they thought I was still in contract.  I gave them a call and was assured by the call centre agent that the email had been sent by mistake and I wasn't actually going to be charged.  He said he had made a note on my account to that effect to ensure it got reviewed before actually billing.

Then they took that charge from my bank account by direct debit.  So I called them again and they refused to refund it, stating that the only note on my account stated something very generic such as "customer had a billing question, explained it to him" or words to that effect.  After about 45 minutes of shouting at them they finally agreed to review their call recording.

I got a call back a few days later saying they had reviewed the call recording.  They stated that I had never been told that it was a mistake, never told that I wouldn't be charged and never told that a note had been made on my account.

So I emailed them the recording that I had made of the call...  Suddenly they refund the charge, no questions asked.  (Ok, they miscalculated the refund and I had to tell them to fix it, but still...)

Originally I would've had no problem switching back to BT in the future, but now I've come to regard them as a company that will outright lie to make money fraudulently so long as the customer can't produce any evidence to prove they are lying...  Pretty bad.

(Yeah, I know I could've enacted the DD guarantee to get my money back, but they would've just recorded it as a default so best to get it sorted at the source).

FalconNet / Merula

When I switched away from BT, I moved my internet connection and POTS over to FalconNet (who are a trading name of Merula Ltd).  £22/month (inc. VAT) for the internet connection (40Mbps down, 10Mbps up, FTTC) and £9.50/month inc. VAT for the POTS line.

They are invoicing me £25 ex. VAT and £8.20 ex. VAT respectively.  Emailing them to point out that they're billing me incorrectly has resulted in a grand total of no response at all.  Not impressed.  We'll wait to see what they take by direct debit and possibly ask the bank to back charge the DD if I can't get any response.

Edit: All cleared up and a credit note has been applied to my account.

Npower

The troubles with Npower seem to be continuing, even after I have stopped being their customer.  Over an 18 month period I have received 13 separate bills from them (bearing in mind they are supposed to be billing twice a year...)  Most of the bills are wrong and the next bill (usually also wrong) starts by cancelling the previous bill.  On the whole I've ended up with a massively confusing mess of bills that has taken me a considerable amount of time to go through and understand exactly what they have billed me for.

The latest bill says I owe £144.07, but as far as I can tell they are actually about £166.30 out and they actually owe me £22.23.

Here's the message I just sent to them...  I'm through trying to sort this stuff out over the phone, it's just a complete waste of my time...

My latest bill states that I owe you £144.07.  As I have previously mentioned to you over the phone, I do not intend to pay you until you send me a correct bill - the bill you have sent is incorrect, just like numerous previous bills:

1. My direct debit discount is supposed to amount to £100/year.  In January 2014, the discount for 2013 did not appear on my bill, so I called you and was told that it would be credited to my next bill 6 months later.  This did not happen, so I called again on June 9th and was told the direct debit discount would be refunded.  I called again on June 25th and was told it was "still being handled".  My latest bill, received last week, still shows that this direct debit discount has not been refunded.

2. Although I was a customer until the middle of July this year, I have only received £21.09 as a pro-rata direct debit discount for 2014. It is true that my bill has largely not been paid by direct debit this year. That is your fault though - the direct debit was set up, you just didn't bother to charge it.  I still expect to receive a pro-rata discount of around £50.

3. The statement dated October 22 2013 states that my closing balance is £403.50.  The following statement, dated January 29 2014 lists the opening balance at £419.80.  The £16.30 difference appears to be unaccounted for.

On the whole, I have received an extremely confusing mess of bills, cancelled bills and amended bills since January 2013 - I have received no less than 13 separate bills over this period, most of them wrong in one way or another, and it has been very difficult and time consuming to piece together exactly what you've done.