Wednesday, 30 August 2017

Incompetent ISPs

Some background history


Local education authorities in the UK often operate a "broadband consortium" for the schools in the region.  The cost of providing a decent internet connection has often been very dependent on the school's geographic location - some schools can be connected up fairly cheaply whilst others are out in the sticks and very costly to connect.  So the idea is that the authority can provide internet connections to all of their schools at a fixed(ish) price to make things fairer.  Essentially, the schools that can have cheap connections subsidise those that can't.

Connecting schools into a single WAN offers a few other advantages, such as being able to centralise the internet filtering for the entire region.

As safeguarding requirements have become more stringent, the centralised filtering systems have often not kept pace and some schools have opted to buy third party filtering systems which they considered superior, instead.  So as an online safety supplier, we have picked up a number of customers who still use the authority-supplied internet connection, but buy in their own filtering system.

More recently, bandwidth requirements have been increasing at an unprecedented rate, and internet connections have been getting steadily cheaper, so the original reasons for the broadband consortia are becoming less important.

Some authorities also found that their networks couldn't cope with the increased bandwidth requirements.  Somerset was one of those authorities, with a network that was part of the South West Grid for Learning (SWGfL - operated by RM Plc), and decided to disband their network, leaving the schools to choose between contracting directly the SWGfL, or to switch to a new ISP.

For the schools that were still using the centralised filtering system, sticking with the SWGfL made sense, since they could continue more or less as-is (their internet connection itself was replaced, but the filtering system remained the same).

As an aside, I'll note that my experience of the SWGfL is that they seem to have significantly more problems than other ISPs and it takes them a lot longer to sort them out.  Part of this may be that operating a filtering system introduces a few more points of failure on their network.  But their support is done through a foreign callcentre (so you're already having to deal with a language barrier when trying to get problems sorted) and their staff don't really seem technical enough to do their jobs.  On many occasions I have been told that a problem is "fixed" when it clearly isn't and it seems they simply didn't have the technical skills to actually test it themselves.

The problem

 

One of our customers, who was using the Somerset connection only for internet connectivity (not filtering), asked us to help them choose which ISP to switch to.  We got them a few quotes for leased lines, but they eventually decided to stick with the SWGfL - I'm not completely clear on the costs but I'm under the impression that it wasn't terribly dissimilar to the other quotes, and the reason given for picking the SWGfL was because its "what they know".  I must admit that I don't quite understand that because as far as I can tell "what they know" is that they have had no end of problems with the SWGfL-based authority-supplied connection in the past.

Also interesting to note that most ISPs now do IPv6 as standard over leased lines (although it's still hit and miss for VDSL, etc.) but IPv6 isn't an option at all on an SWGfL connection and it doesn't sound like RM have any plans to implement it any time soon!  You also don't get a public subnet like you would with pretty much every other ISP - instead everything is done on private RFC1918 addresses and the ISP does NAT for you.  This might make sense for schools who rely on RM's filtering, but is a bit nonsensical for anyone who just wants a plain unfiltered connection.

Anyway, the school bought an unfiltered connection from the SWGfL and it was duly connected.  They paid for one of our engineers to go on site to ensure the switch over went smoothly.  One of the things we had to do was expand their external-facing IP network because RM informed the school that they needed to set up VRRP for the backup internet connection.

Very soon, the school complained about problems connecting to HTTPS websites and we did some diagnosis - a high proportion of HTTPS connections just seemed to not be successful.  This was very odd behaviour - you'd try to make a connection and the SYN packets would all just be dropped somewhere in the ISP's network, but traffic for other connections would be working just fine.  We decided the likely explanations were that the traffic was being intercepted by a broken transparent proxy, or that RM's CGNAT system was broken.

We also noticed that some proxy headers were being added to HTTP traffic, indicating that it was being transparently proxied.  This suggested that the HTTPS traffic was probably also being transparently proxied.  Transparently proxying HTTPS traffic doesn't make any sense for an unfiltered connection, and RM eventually confirmed that the traffic was indeed being directed at their filtering system.  They agreed to turn off the filtering and the school submitted a change request for this change to be made.

Several months later, the problem reoccurred and we found that the traffic was once again being directed through a transparent proxy.  The school raised a support request and RM confirmed that the traffic was being sent through their filtering system, and that the school would need to submit another change request to have this corrected.  However, RM flatly refused to turn off filtering for the whole school's network, only for a single IP address.

After an official complaint was made by the school (since RM were not supplying the "unfiltered connection" that the school had been sold), they agreed to move the school onto an unfiltered connection, but that this would require moving them onto a public IP subnet.  They allocated a /28 network and told us that the school currently had 16 public IPs allocated to 1:1 NAT rules which would need to be moved into that new /28.

We pointed out that a /28 network doesn't actually have 16 usable IP addresses (surely this was obvious and should have been taken into account when they allocated it?)  We also asked which IPs were going to be reserved for VRRP - that totally confused the issue because it turns out that, despite specifically asking us to reconfigure things so that they could use VRRP, they never configured it and don't actually need it for anything.

So eventually, RM allocated a new /27 network and we sent an engineer over for another half day to make sure things were reconfigured properly.  Of course, everything got reconfigured and the switch-over occurred and nothing worked.  After some toing and froing it turned out that they had used a completely different network than the /27 network they had allocated.  So more reconfiguration later and everything worked.

Then a few hours later, everything broke again and the school staff came in in the morning to find no internet connection.  We are assured by RM that this was a problem with their core network and nothing to do with the reconfiguration, but I certainly have my suspicions.

Conclusions


  • Using SWGfL may make sense for schools who want to use their filtering system, but they seem ill equipped to provide reliable unfiltered connections (I can't comment on the reliability of filtered connections, but we did see significant enough problems with their transparent proxies that I wouldn't want to vouch for the reliability of their filters).
  • RM's network seems to have significantly more problems than other ISPs.
  • They seem to take a lot longer to resolve problems than other ISPs - the school has had significant problems associated with RM's broken transparent proxies for months.
  • Dealing with foreign call centres is very frustrating, if only because you're constantly battling with a language/strong-accent barrier (I would have similar concerns with any company basing call centres in "strong accent" areas in the UK too).
  • The competence of their technical staff seems very questionable - we shouldn't be told to reconfigure stuff to allow services that they don't need and won't ever set up (VRRP), subnet allocations shouldn't spontaneously change without anyone being informed.
  • We had considered doing the second network reconfig remotely - its lucky we didn't since RM's mistake would have left the school's network unreachable from the internet.

Tuesday, 17 January 2017

Village Hotels, Subject Access Request

So following my complaint about spamming, Village Hotels responded to my Subject Access Request under the Data Protection Act.  In short, the DPA deems personal data to be owned by the person whome it regards.  A Subject Access Request allows the owner of some personal data to request information about it from a company that is holding it.

In this case I asked Village Hotels to:
  1. Confirm whether they are registered as a data controller under the Data Protection Act 1998.
  2. Provide a copy of all data they hold which relates to me.
  3. Confirm to me in writing where they obtained the data from and on what terms.
  4. Provide me with a complete list of companies, individuals, etc, that they have shared my data with and on what terms.
  5. Explain when and how they believe I gave them informed consent to send marketing emails.
They've sent some fairly extensive information, which is good, and confirmed that they are registered.

It seems that they have only my name and email address, which they thought had previously been deleted from their systems.

Notably they haven't said why they think they had consent, or where they got my data.

They enclosed an email chain, which is interesting: it includes a comment from someone at DA Group, which says
Also agree that you need to be careful emailing contacts that have had no interaction with your emails for 18 months - may be better to focus on a re-engagement campaign to these contacts rather than actively marketing to them
DA Group appear to be a marketing company, and the above comment seems a bit questionable.  The Privacy and Electronic Communications (EC Directive) Regulations 2003 don't allow unsolicited emails to be sent to anyone who hasn't given the sender informed consent.  So if they can't demonstrate that such consent has been received, they shouldn't be sending any unsolicited emails, including a "re-engagement campaign".

(Also, yes the email they sent had a confidentiality statement on the bottom, no that isn't a contract I have agreed to, so I am ignoring it.)

Friday, 13 January 2017

Village Hotels, yet again


Despite never being a customer of De Vere Venes, they sent me hundreds of spam emails, ignored my requests to stop and I eventually complained to the Information Commissioner's Office.  De Vere were instructed to stop spamming me and they admitted they had no idea where they got my details from.  Since the Privacy and Electronic Communications (EC Directive) Regulations 2003 require the sender of unsolicited emails to have obtained informed consent directly from the recipient, they're on pretty dodgy ground if they don't have any records to demonstrate that they acquired that consent.  Certainly, they won't be able to demonstrate consent with "we don't know where we got your details."

De Vere Venues then sold their brand and my details to VUR Village Hotels & Leisure.  VUR Village Hotels & Leisure used those details to start sending me spam again.  This is certainly unlawful - consent is non-transferable, so VUR Village Hotels & Leisure are in breach of the regulations by sending unsolicited emails to anyone in the database that De Vere gave them.  Furthermore, since the ICO had already told De Vere to suppress my details, actually using them was completely idiotic.  I'm also not terribly happy that De Vere sold off my data.

Much like De Vere, Village Hotels also ignored the legal notices I sent to them, so the next step was to take court action.  This is pretty easy to do - you just figure out how much the unsolicited email cost you, make sure you can justify it and submit a claim on moneyclaim.gov.  It costs £25 to file with the court.  They coughed up £225 (including the £25 filing fee) and stated that my details were bought from De Vere and that all other information had been removed during the transfer.  They didn't comment on the unlawful use of email addresses which they had bought from De Vere, beyond stating that their spam emails carry an "unsubscribe" link (which the regulations do not consider an acceptable alternative to acquiring informed consent before sending the emails).

5 days after they confirmed that they had suppressed my email address, I received another spam email.  Their defence was that it takes 3-5 days to suppress an email.  This isn't really acceptable - there's no reason that updating a database should take more than a few minutes, and the court papers were issued 10 days previous so sensibly they would have suppressed my address immediately.  In any case, as no further spams arrived I didn't press the point.

At the end of last year, it appears that VUR Village Hotels & Leisure moved to a different spam sending platform.  It looks like they hadn't actually removed my details from their database, merely flagged them as not to be used, and I guess they didn't bother to transfer that flag over to the new platform.  I started receiving more spam.  So off went another Notice Before Action.  An hour later I got an automated message from their spam sending platform (dotmailer.com) saying I'd been unsubscribed.  And 9 days later an email notifying me that they were paying another £200 compensation - I didn't even need to file with the court.

Of course, the concern is that not only do they clearly have no regard for the regulations, which prevent them from just buying a database of email addresses and spamming them, they also don't seem to have the data handling abilities required to ensure that people who unsubscribe actually remain unsubscribed.  I'd certainly recommend that any other recipients of spam from VUR Village Hotels & Leisure (or in fact, any registered UK company who is spamming your personal email address) serves them with a legal notice demanding compensation for their negligence.