Thursday, 5 April 2018

Thoughts on the gender pay gap

I've already managed to upset someone on Facebook because I apparently said that women are worth less than men (I didn't).  I don't regard myself as a feminist, but rather an equalist.  Discrimination of all forms is bad - women shouldn't be discriminated against based on their gender, but similarly men shouldn't be discriminated against in order to give women an advantage.

Discrimination is as old as time itself but I don't think you fix the problem by just changing which group you discriminate against, any more than you prevent war by changing which group of innocent people you're blowing up.

The British government is now requiring all businesses with over 250 employees to publish figures summarising the gender pay gap.  There are a few bits of information required, but essentially this boils down to a simple average difference in pay between men and women.

The press is then using the figures to bash the companies with the biggest pay gap.  No consideration is being given to what types of work are being done since that isn't in the information that companies are expected to publish.  i.e. this is not a like-for-like comparison.

There are a few potential reasons why women may be earning less than men:
  1. Maybe there are less women qualified and applying for the higher paid jobs.
  2. Maybe employers are refusing to employ women in the higher paid jobs.
  3. Maybe employers are employing women in the higher paid jobs but are paying them less than the men who are doing the same work.
All of these points are, of course, a problem.  The two last points are directly under the employer's control and any employer undertaking in this kind of discrimination deserves to have the book thrown at them - I firmly believe that equally qualified men and women should be given the same opportunities as each other and the same pay for doing the same work.

However, it is fairly unclear to me how the first point can be regarded as a specific employers' fault.  Since the information that is being made available doesn't do anything to differentiate between these possibilities, it seems completely unfair to vilify an employer based solely on this data.

So we have STEM companies and the construction industry with a fairly big pay gap simply because it's very difficult to recruit women to do the higher paid jobs in these fields.  I'm sure that in some cases there is discrimination going on, but you can't determine that from the data being used by the press to attack employers.

To demonstrate the issue, lets take a simple fictional employer - they aren't discriminating and they have the following breakdown of employees:

Total Men Women Salary
Engineers 200 180 20 £50,000.00
Admin staff 50 15 35 £20,000.00
Total 250 195 55

The mean pay is £47,692.31 for men and £30,909.09 for women, yielding a pay gap of £16,783.22, even though everyone doing the same job is paid the same.

So, how can the employer fix their pay gap?  Since we're already assuming there is no discrimination going on, we can look at it rationally with maths:

1. Increase the proportion of female engineers
It's very unclear to me how a single employer can increase the proportion of applicants who are female.  In order to do this the proportion of women training in engineering needs to be increased, starting with school kids.  There is some scope for the industry as a whole working to promote engineering to women, but it takes years and a single employer can't do a lot on their own.

At work, when we were last recruiting, we didn't end up hiring a man because we're horrible people who support the patriarchy; we ended up hiring a man because no women applied for the job.  We would usually want to pick the best person for the job, regardless of their gender.

So with far less women in the engineering job market than men, the immediate options are:
  • Increase the women's benefits and decrease the men's.  The women will be "overpaid" with respect to other employers and want to work for you whilst the men will be "underpaid" and get a job elsewhere.
  • Recruit underqualified women to make up the numbers, since there aren't enough qualified women applying.  Recruiting people who aren't qualified to do the job sounds like a bad idea for the business.
  • Restrict the number of applications from men.
All of these options discriminate against men based solely on their gender and are therefore pretty unethical (and probably illegal).

2. Increase the proportion of male administrative staff
So given that we probably can't do a lot to recruit more female engineers, we could tweak the balance elsewhere in the business.  The average women's pay is being dragged down by the fact that a disproportionate number of women are employed in the lower paid administrative roles.  Only 10% of the engineers are women, but 70% of the administrative staff are women.  If the employer reduces the proportion of women in the admin roles down to 10%, that will eliminate the pay gap.

This problem is pretty much the opposite of (1) - the same options apply, but this time the employer must discriminate against women.  Again, doesn't strike me as a good plan.

3. Increase the women's pay or decrease the men's pay
So far, the employer has paid men and women the same amount for the same work, which has led to a big pay gap simply because of the job role demographics.

Adding about 55% to the womens' pay in both job roles eliminates the pay gap.  We're now paying female engineers £77,000 and female administrative staff £30,800.  Of course, the employer may well not be able to afford these kinds of expenses, especially when the men find out that they are earning far less than the women and take their employer to court for sexual discrimination.

Similarly, reducing the men's pay by 65% across the board achieves a similar result - the employer gets sued into the ground for sexual discrimination, and if they survive that, with the men now being paid far below the market rate, they all leave for greener pastures.

4. Pay the engineers and administrative staff the same
Another option is to decrease the pay the engineers receive and increase the pay the administrative staff receive.  By paying everyone £44,000 irrespective of what job they do, the pay gap is eliminated whilst keeping the total wage bill the same.

Unfortunately, the company's engineers are now underpaid relative to the market rate, so they will leave and it will be impossible to recruit replacements.

The entire industry could follow suit, but this would lead to a long term shortage in engineers - it costs tens of thousands of pounds and several years to become an engineer, and how many people would do that if their pay is the same as someone who hadn't spent that time and money on training?


I am a firm believer that equally qualified and experienced women and men are equally valuable and should be paid the same.  For some jobs it is easier than others to ensure that this is happening - where there are fixed non-negotiable pay scales things are obviously clearer than jobs where employees are expected to negotiate their salaries.  In the latter case, people who are poor negotiators are obviously not going to do as well as good negotiators.  I don't know if there's a gender bias when it comes to negotiating skill, but any protection should surely extend far beyond the work place since poor negotiators also lose out when negotiating other things, such as buying a car, etc.

There are also big differences in demographics that need to be accounted for: is a 35 year old who graduated from university at 21 and has been in relevant employment for the whole time (giving them 14 years industry experience) as valuable as someone else who took 10 years off between 23 and 33 to bring up their children (so they have only 4 years industry experience)?  These people could be either men or women, but in the current society the majority of child carers are women and it doesn't seem right to completely discount that when comparing pay.

From taking a rational look at the data being collected, it seems clear to me that it is tackling the wrong thing.  Whilst I'm sure that discrimination is happening, the data being published cannot be used to determine who is discriminating.  Indeed, in many cases reducing the pay gap seems to actually require the employer to discriminate, so the whole thing seems very counter productive to me.

The things that need to be tackled are:
  • Ensuring that men and women are given the same opportunities.
  • Ensuring that men and women receive similar pay for similar work.
  • Ensuring that neither men and women are put off from taking any opportunities that are open to them.
Lets work on those points instead of attacking employers for things that they can't do much about.

Monday, 30 October 2017

ESP8266 CurrentCost Wifi Gateway

I have two CurrentCost power monitors - a CC128 EnviR panel in the living room which monitors our entire electricity usage, and an old white first generation panel in the office which monitors the office's electricity usage.

The CurrentCost panels output real time data in XML format every 6 seconds through a TTL serial connection presented as an 8P8C connector on the bottom, and I used to have a WRT54GL wireless access point connected to that, which sent the data to the server.  The WRT54GL was retired some time ago, so I wanted another way of recording the power usage data.  The ESP8266 seemed like a good choice to provide a bridge to my wifi network and I've chosen to use an ESP-12F board.  I could build a small enough device on stripboard to sit in the cavity under the base of the panel and connect to the CurrentCost's TTL serial output.


The 8P8C connector also includes regulated and unregulated power connections, but the 3v PSU is only rated for 80mA which is nowhere near enough to power the ESP8266.  I didn't want to have a second PSU, so opted to use a single USB power supply to power everything.  I've used an LM1117T LDO regulator to drop the supply voltage down to 3.3v, and connected this to both the ESP8266 and the unregulated supply rail of the CurrentCost.  There's a schottky diode between the regulator and the CurrentCost - this isn't strictly necessary, but will protect against someone plugging the CurrentCost PSU in at the same time as the wifi bridge.

GPIO0 and GPIO2 are pulled up by 10KΩ resistors, GPIO15 is pulled down by a 10KΩ resistor - you can probably get away with omitting the resistors and connecting these directly to Vcc / 0v respectively, although using a resistor to pull up GPIO0 allows the ESP8266 to be programmed in circuit by temporarily pulling it down to 0v.

CH_PD and reset are pulled up all the time by tieing them to the Vcc rail.

The connections to the CurrentCost's 8P8C connector are:
  • Pin 1 - Vcc / unregulated power
  • Pin 4 - Ground / 0v
  • Pin 8 - TTL serial data
Component-side view on the left, flipped for a track-side view on the right
Since the ESP-12F has solder pads with a 2mm spacing and the stripboard has 0.1" hole spacing, the ESP has to stand off the stripboard a little.  I've taken the opportunity to use the space between the ESP-12F and the board for the pullup / pulldown resistors.  I've shown the ESP-12F as a translucent block on the stripboard layout above so you can see the components under it.

The power connector is a surface mount mini-USB socket - obviously that's incompatible with stripboard, so I have just tacked on a couple of wires to connect it to the circuit and soldered its mounting lugs down to the track-side.  I find it useful to hot-glue the wires onto the board for mechanical support to avoid breaking off the socket's pins.
Component-side view

Track-side view
Although the CurrentCost outputs XML, I'm not bothering to parse that in the ESP8266.  Helpfully, each XML sentence is terminated with a newline, so it's easy to use Serial.readStringUntil('\n') to grab the whole sentence and post it to the web server.  Data posted over HTTP needs to be URL encoded and there doesn't seem to be a standard library to do this, so I ended up writing a quick and dirty encoder.  A PHP script on the web server parses the XML and feeds the data into RRDTool, and I'm using Cacti to display graphs of the data.




I wanted my graphs to display a total number of kilowatt hours used.  Unfortunately Cacti doesn't have a generic function to do this.  It can do bandwidth summation for network usage (i.e. displaying a "total megabytes" on a "bytes per second" graph), which is almost what I need.  Using that on the "watts" data gives a total number of watt seconds (i.e. joules) used, but no way to divide this by 3600 to produce watt hours.  In the end I added a COMPUTE data source to the RRD file, which records watts divided by 3600, and Cacti can then use that to calculate the total watt hours of electricity consumed.  Since I'm abusing the bandwidth summation functions, this unfortunately adds a "B" (bytes) unit to the total displayed, but I can live with that.

The code, schematic and layout are available from my Subversion repository.  The circuit works for both the CurrentCost EnviR (also known as the CC128) and the first generation CurrentCost, although the baud rate needs to be changed in the code - the EnviR uses a rate of 57600bps whilst the first generation panel uses 2400bps.  I believe there are also second generation panels around that look similar to the first generation ones but run at 9600bps.


Wednesday, 30 August 2017

Incompetent ISPs

Some background history


Local education authorities in the UK often operate a "broadband consortium" for the schools in the region.  The cost of providing a decent internet connection has often been very dependent on the school's geographic location - some schools can be connected up fairly cheaply whilst others are out in the sticks and very costly to connect.  So the idea is that the authority can provide internet connections to all of their schools at a fixed(ish) price to make things fairer.  Essentially, the schools that can have cheap connections subsidise those that can't.

Connecting schools into a single WAN offers a few other advantages, such as being able to centralise the internet filtering for the entire region.

As safeguarding requirements have become more stringent, the centralised filtering systems have often not kept pace and some schools have opted to buy third party filtering systems which they considered superior, instead.  So as an online safety supplier, we have picked up a number of customers who still use the authority-supplied internet connection, but buy in their own filtering system.

More recently, bandwidth requirements have been increasing at an unprecedented rate, and internet connections have been getting steadily cheaper, so the original reasons for the broadband consortia are becoming less important.

Some authorities also found that their networks couldn't cope with the increased bandwidth requirements.  Somerset was one of those authorities, with a network that was part of the South West Grid for Learning (SWGfL - operated by RM Plc), and decided to disband their network, leaving the schools to choose between contracting directly the SWGfL, or to switch to a new ISP.

For the schools that were still using the centralised filtering system, sticking with the SWGfL made sense, since they could continue more or less as-is (their internet connection itself was replaced, but the filtering system remained the same).

As an aside, I'll note that my experience of the SWGfL is that they seem to have significantly more problems than other ISPs and it takes them a lot longer to sort them out.  Part of this may be that operating a filtering system introduces a few more points of failure on their network.  But their support is done through a foreign callcentre (so you're already having to deal with a language barrier when trying to get problems sorted) and their staff don't really seem technical enough to do their jobs.  On many occasions I have been told that a problem is "fixed" when it clearly isn't and it seems they simply didn't have the technical skills to actually test it themselves.

The problem

 

One of our customers, who was using the Somerset connection only for internet connectivity (not filtering), asked us to help them choose which ISP to switch to.  We got them a few quotes for leased lines, but they eventually decided to stick with the SWGfL - I'm not completely clear on the costs but I'm under the impression that it wasn't terribly dissimilar to the other quotes, and the reason given for picking the SWGfL was because its "what they know".  I must admit that I don't quite understand that because as far as I can tell "what they know" is that they have had no end of problems with the SWGfL-based authority-supplied connection in the past.

Also interesting to note that most ISPs now do IPv6 as standard over leased lines (although it's still hit and miss for VDSL, etc.) but IPv6 isn't an option at all on an SWGfL connection and it doesn't sound like RM have any plans to implement it any time soon!  You also don't get a public subnet like you would with pretty much every other ISP - instead everything is done on private RFC1918 addresses and the ISP does NAT for you.  This might make sense for schools who rely on RM's filtering, but is a bit nonsensical for anyone who just wants a plain unfiltered connection.

Anyway, the school bought an unfiltered connection from the SWGfL and it was duly connected.  They paid for one of our engineers to go on site to ensure the switch over went smoothly.  One of the things we had to do was expand their external-facing IP network because RM informed the school that they needed to set up VRRP for the backup internet connection.

Very soon, the school complained about problems connecting to HTTPS websites and we did some diagnosis - a high proportion of HTTPS connections just seemed to not be successful.  This was very odd behaviour - you'd try to make a connection and the SYN packets would all just be dropped somewhere in the ISP's network, but traffic for other connections would be working just fine.  We decided the likely explanations were that the traffic was being intercepted by a broken transparent proxy, or that RM's CGNAT system was broken.

We also noticed that some proxy headers were being added to HTTP traffic, indicating that it was being transparently proxied.  This suggested that the HTTPS traffic was probably also being transparently proxied.  Transparently proxying HTTPS traffic doesn't make any sense for an unfiltered connection, and RM eventually confirmed that the traffic was indeed being directed at their filtering system.  They agreed to turn off the filtering and the school submitted a change request for this change to be made.

Several months later, the problem reoccurred and we found that the traffic was once again being directed through a transparent proxy.  The school raised a support request and RM confirmed that the traffic was being sent through their filtering system, and that the school would need to submit another change request to have this corrected.  However, RM flatly refused to turn off filtering for the whole school's network, only for a single IP address.

After an official complaint was made by the school (since RM were not supplying the "unfiltered connection" that the school had been sold), they agreed to move the school onto an unfiltered connection, but that this would require moving them onto a public IP subnet.  They allocated a /28 network and told us that the school currently had 16 public IPs allocated to 1:1 NAT rules which would need to be moved into that new /28.

We pointed out that a /28 network doesn't actually have 16 usable IP addresses (surely this was obvious and should have been taken into account when they allocated it?)  We also asked which IPs were going to be reserved for VRRP - that totally confused the issue because it turns out that, despite specifically asking us to reconfigure things so that they could use VRRP, they never configured it and don't actually need it for anything.

So eventually, RM allocated a new /27 network and we sent an engineer over for another half day to make sure things were reconfigured properly.  Of course, everything got reconfigured and the switch-over occurred and nothing worked.  After some toing and froing it turned out that they had used a completely different network than the /27 network they had allocated.  So more reconfiguration later and everything worked.

Then a few hours later, everything broke again and the school staff came in in the morning to find no internet connection.  We are assured by RM that this was a problem with their core network and nothing to do with the reconfiguration, but I certainly have my suspicions.

Conclusions


  • Using SWGfL may make sense for schools who want to use their filtering system, but they seem ill equipped to provide reliable unfiltered connections (I can't comment on the reliability of filtered connections, but we did see significant enough problems with their transparent proxies that I wouldn't want to vouch for the reliability of their filters).
  • RM's network seems to have significantly more problems than other ISPs.
  • They seem to take a lot longer to resolve problems than other ISPs - the school has had significant problems associated with RM's broken transparent proxies for months.
  • Dealing with foreign call centres is very frustrating, if only because you're constantly battling with a language/strong-accent barrier (I would have similar concerns with any company basing call centres in "strong accent" areas in the UK too).
  • The competence of their technical staff seems very questionable - we shouldn't be told to reconfigure stuff to allow services that they don't need and won't ever set up (VRRP), subnet allocations shouldn't spontaneously change without anyone being informed.
  • We had considered doing the second network reconfig remotely - its lucky we didn't since RM's mistake would have left the school's network unreachable from the internet.

Tuesday, 17 January 2017

Village Hotels, Subject Access Request

So following my complaint about spamming, Village Hotels responded to my Subject Access Request under the Data Protection Act.  In short, the DPA deems personal data to be owned by the person whome it regards.  A Subject Access Request allows the owner of some personal data to request information about it from a company that is holding it.

In this case I asked Village Hotels to:
  1. Confirm whether they are registered as a data controller under the Data Protection Act 1998.
  2. Provide a copy of all data they hold which relates to me.
  3. Confirm to me in writing where they obtained the data from and on what terms.
  4. Provide me with a complete list of companies, individuals, etc, that they have shared my data with and on what terms.
  5. Explain when and how they believe I gave them informed consent to send marketing emails.
They've sent some fairly extensive information, which is good, and confirmed that they are registered.

It seems that they have only my name and email address, which they thought had previously been deleted from their systems.

Notably they haven't said why they think they had consent, or where they got my data.

They enclosed an email chain, which is interesting: it includes a comment from someone at DA Group, which says
Also agree that you need to be careful emailing contacts that have had no interaction with your emails for 18 months - may be better to focus on a re-engagement campaign to these contacts rather than actively marketing to them
DA Group appear to be a marketing company, and the above comment seems a bit questionable.  The Privacy and Electronic Communications (EC Directive) Regulations 2003 don't allow unsolicited emails to be sent to anyone who hasn't given the sender informed consent.  So if they can't demonstrate that such consent has been received, they shouldn't be sending any unsolicited emails, including a "re-engagement campaign".

(Also, yes the email they sent had a confidentiality statement on the bottom, no that isn't a contract I have agreed to, so I am ignoring it.)

Friday, 13 January 2017

Village Hotels, yet again


Despite never being a customer of De Vere Venes, they sent me hundreds of spam emails, ignored my requests to stop and I eventually complained to the Information Commissioner's Office.  De Vere were instructed to stop spamming me and they admitted they had no idea where they got my details from.  Since the Privacy and Electronic Communications (EC Directive) Regulations 2003 require the sender of unsolicited emails to have obtained informed consent directly from the recipient, they're on pretty dodgy ground if they don't have any records to demonstrate that they acquired that consent.  Certainly, they won't be able to demonstrate consent with "we don't know where we got your details."

De Vere Venues then sold their brand and my details to VUR Village Hotels & Leisure.  VUR Village Hotels & Leisure used those details to start sending me spam again.  This is certainly unlawful - consent is non-transferable, so VUR Village Hotels & Leisure are in breach of the regulations by sending unsolicited emails to anyone in the database that De Vere gave them.  Furthermore, since the ICO had already told De Vere to suppress my details, actually using them was completely idiotic.  I'm also not terribly happy that De Vere sold off my data.

Much like De Vere, Village Hotels also ignored the legal notices I sent to them, so the next step was to take court action.  This is pretty easy to do - you just figure out how much the unsolicited email cost you, make sure you can justify it and submit a claim on moneyclaim.gov.  It costs £25 to file with the court.  They coughed up £225 (including the £25 filing fee) and stated that my details were bought from De Vere and that all other information had been removed during the transfer.  They didn't comment on the unlawful use of email addresses which they had bought from De Vere, beyond stating that their spam emails carry an "unsubscribe" link (which the regulations do not consider an acceptable alternative to acquiring informed consent before sending the emails).

5 days after they confirmed that they had suppressed my email address, I received another spam email.  Their defence was that it takes 3-5 days to suppress an email.  This isn't really acceptable - there's no reason that updating a database should take more than a few minutes, and the court papers were issued 10 days previous so sensibly they would have suppressed my address immediately.  In any case, as no further spams arrived I didn't press the point.

At the end of last year, it appears that VUR Village Hotels & Leisure moved to a different spam sending platform.  It looks like they hadn't actually removed my details from their database, merely flagged them as not to be used, and I guess they didn't bother to transfer that flag over to the new platform.  I started receiving more spam.  So off went another Notice Before Action.  An hour later I got an automated message from their spam sending platform (dotmailer.com) saying I'd been unsubscribed.  And 9 days later an email notifying me that they were paying another £200 compensation - I didn't even need to file with the court.

Of course, the concern is that not only do they clearly have no regard for the regulations, which prevent them from just buying a database of email addresses and spamming them, they also don't seem to have the data handling abilities required to ensure that people who unsubscribe actually remain unsubscribed.  I'd certainly recommend that any other recipients of spam from VUR Village Hotels & Leisure (or in fact, any registered UK company who is spamming your personal email address) serves them with a legal notice demanding compensation for their negligence.

Sunday, 24 July 2016

Wifi doorbell project

We've got a wireless doorbell which is fairly unreliable.  Also, I can't hear it from my office, and the 433MHz transmitter built into the doorbell button doesn't have enough range for me to put a sounder in the office.  The unreliability problem could probably be fixed by just buying a new wireless doorbell, but that probably wouldn't help with the range problem.

The advent of the ESP8266 microcontrollers opened up a more interesting approach.  These are programmable microcontrollers with a built in wifi interface.  They are tiny and also only cost about a pound each.  So my plan was: build a new doorbell push from scratch, which talks to the wifi network.  The old doorbell has an electromechanical sounder, which I'm reusing by replacing the driver circuit.

Doorbell push

ESP-01 on stripboard
The first job was the doorbell push.  I'm using an ESP-01 module for this because it's tiny, and its running off a pair of AAA cells, which will provide a 3.0v supply when new.  In theory, I could use a buck-boost regulator to maintain a 3.3v supply to the ESP throughout the life of the batteries, but the quiescent current draw of the regulator would probably drain the battery fairly quickly since this circuit is going to spend most of its life asleep.  Although the ESP8266 is supposed to run on 3.3v, they are apparently ok down to around 2.3v, so I'm running it directly off the batteries with no regulator.  The ESP8266's wifi radio has some reasonably high power demands, so there's a 100µF capacitor across the supply rails to absorb any high current bursts which the batteries may not be up to supplying.

The theory of operation is as follows: the ESP-01 module will be put in deep sleep mode when idle.  The button is wired between the ESP's reset pin and ground, so when someone pushes the button to ring the bell, it pulls reset low and wakes the device from deep sleep.  The ESP-01 has an on board power LED which is always on and would drain the batteries quite quickly, so I've cut the track supplying power to that LED - the module now draws 20µA when in deep sleep mode, so I expect the batteries to last a while.  When the ESP resets, it immediately associates with the wifi network and makes two HTTP requests - one directly to the sounder (more on this later) and the other to my home server.  A php script on the server sends out "doorbell rang" instant messages to both myself and Mel via XMPP - this solves the problem of hearing the bell in the office, since the XMPP notification pops up on my workstation and phone.

Everything just fits!
The button I bought contains an LED ring which I'm using to give the person ringing the bell feedback - The ring lights up when the wifi connection has been established (including DHCP, etc.) and flashes after the HTTP requests have successfully completed.  There's a minor complication that the ESP-01 only makes 2 GPIO pins available, they both have to be pulled high when the ESP boots up, and in fact the ESP-01 module has integrated pullups for this reason.  So the LED is wired between the positive power rail and GPIO-0.  This results in inverted logic - to turn the LED on GPIO-0 has to be set low, to turn it off GPIO-0 has to be either high or in high impedance (input) mode.  Once the HTTP requests have completed and the LED has been flashed, the ESP goes back into deep sleep mode.

The web requests that the button makes include the current battery voltage, so I can figure out when to change the batteries.  Its now been in use for about 15 weeks and the reported battery voltage has fallen from about 3.182v to 3.053v, which I think is pretty reasonable.

The finished doorbell push On the door frame

Sounder

As mentioned, I'm re-purposing the old electromechanical sounder by replacing the old circuit board and driving the solenoid directly.  The old sounder was powered by 3 C cells.  Unlike the button, the sounder will need to remain connected to the wifi all the time, so batteries aren't really an option.  I've opted to using a USB power supply, and with no need for the batteries there's now a lot of space inside the sounder for my new circuit.

I'm using an ESP-12F in the sounder - it's a bit overkill really, but I wanted to have an output that didn't have to be pulled up on boot.  The mini-USB port is connected to an LM1117T-3.3 linear regulator to provide the 3.3v supply voltage.  The solenoid is driven by an RFD14N05L field effect transistor and I've included a fairly chunky capacitor to help with the high power requirements of the solenoid.  Of course there's also a reverse biassed diode in parallel with the solenoid to absorb the back-EMF (although no magic smoke was released before I bothered adding it).

The whole thing is soldered up onto stripboard and just pushed into the battery compartment of the sounder (having removed the battery contacts).

The sounder connects to the wifi network as soon as it is powered up and sits there waiting for an HTTP request.  Making a request to "/" returns a status page, requesting "/ring" causes it to fire the striker and make the classic "ding-dong".
The old electromechanical sounder with an ESP-12F installed

Problems to be aware of

Receiving a notification on your phone to say the doorbell is being rung when you're a few hundred miles away is infuriating because you know you're going to have to drive up to the postal depot to collect a parcel when you get home. :)

Source code

Source code for the project is available in Subversion:
https://subversion.nexusuk.org/projects/doorbell/

Friday, 15 July 2016

Adventures in broken apps

Its been a week of frustration, but also success.  We always have a perennial problem of broken apps, which work ok on the average home network, but as soon as you try to secure things you hit problems.  Although there is a good argument for keeping school networks fairly permissive, realistically schools can't just turn off their firewall and let everyone at it - not only would the school be failing in their duty of care, but it would be a security nightmare too.

We're frequently tasked with getting some badly behaved app to work on a school network.  Unfortunately when something works elsewhere but breaks when connected to the school network, the firewall/web filter is often regarded by staff to be "broken", even when we can clearly see that the app is the one doing broken things.  Still, we like to keep our customers happy and be as helpful as possible.

We routinely spend a lot of time diagnosing problems and sending debugging to the app vendors and there are a few vendors that are thankful for our input and will work with us to improve their software.  However, I think its fair to say that the vast majority of app vendors are completely uninterested in fixing bugs in their software.  This attitude is unfortunately prevalent across all kinds of suppliers - from small suppliers, right up to the likes of Microsoft, Apple and Google.  In fact, we no longer submit bug reports to Apple because collecting data for them uses a huge amount of our engineering time and they have never fixed any of the bugs we've reported.  A good example of this recently was CloudMosa, who responded within 24 hours of our bug report, explaining that they weren't going to fix the bugs that we reported in their Puffin Academy app.  WhatsApp have been similarly unhelpful with problems we reported, stating "WhatsApp is not designed to be used with proxy configurations or restricted networks, and we cannot provide support for these network configurations."  What a cop-out!

So with the app developers refusing to properly support their own software, our customers have nowhere left to turn and it is often down to us to do our best to work around the flaws in the applications.  A lot of this comes down to collecting as much information as possible about each connection so we can automatically turn security features on and off on our systems to work around incompatibilities.

We do things like snooping on TLS handshakes - when a device sets up an encrypted connection, it is supposed to include information such as a "server name indication" and we can spot that and use the presented server name to decide whether or not its safe to intercept and analyse the encrypted data.  Some apps aren't compatible with interception, so when we see known problem apps we avoid intercepting their connections.  Unfortunately, every so often you find an app that doesn't bother to include this data, and there's no way for the system to know if it's ok to intercept the connection.

In the second half of this week we developed some new code for the web filter to actively callout to the remote server in these situations.  When the web filter sees an encrypted connection that has no server name indication, it can now connect to the web server, retrieve the certificate and use information in it to figure out what to do next.  We're expecting this to help a lot with the problem apps.  The results of each callout are cached to reduce the impact on the web servers.  This is currently going through testing to make sure it won't cause any problems,

Another frustration has always been Skype - this has always been a real pain to make it work reliably and securely.  We've spent a lot of time this week pouring over network traffic dumps and testing.  There are numerous problems with the Skype protocol, which boil down to:
  1. It makes connections on TCP port 443 (and therefore look like HTTPS), which aren't actually HTTPS, or even TLS.  These connections can go to any IP address, so we can't trivially poke holes in the firewall for them.  They get picked up by the transparent proxy, which treats them as encrypted HTTPS connections and therefore fails to handle them since they aren't actually HTTPS.
  2. It makes real HTTPS connections carrying WebSockets requests.  Unfortunately we don't yet support WebSockets and as Skype doesn't bother to include a server name indication we can't pre-emptively decide not to intercept them.
  3. It sends peer-to-peer voice and video media over UDP using any port numbers between 1024 and 65535.  Since it's peer-to-peer, this traffic can be directed at any IP address on the internet.  Official advice is to just allow that through your firewall - if you do that you may as well not even bother to have a firewall in the first place!
  4. All of Skype's traffic is encrypted so it's almost impossible to figure out what it's actually trying to do and what went wrong when it fails.
  5. If something goes wrong, Skype just breaks in one way or another and provides no indication what actually went wrong.  The Android version of Skype could output some debugging data to Android's standard debugging log, but it doesn't.  The PC version of Skype can be told to produce a debug log, but the log is encrypted so that only Microsoft developers can read it (gee, thanks for nothing Microsoft!)
Fortunately, it turns out that the not-HTTPS-that-looks-like-HTTPS (1) traffic isn't needed if it can successfully set up the peer-to-peer UDP connections (3), so we think we can ignore that problem.

It doesn't actually seem to matter too much if the WebSockets connections (2) fail, but this should be handled by the web filter's new TLS callouts system described above.

So we're left with the UDP traffic, which can be to an IP address on any port (3).  This one is a real problem - blindly allowing all of this traffic would also allow a whole load of other stuff such as VPNs, games, etc.  So we've been playing with the nDPI deep packet inspection library and nDPI-Netfilter.

Normally, firewalling is done based on just the information in the packet headers, such as the source and destination addresses.  Deep packet inspection examines all of the data associated with the connection, including the payload, in an attempt to identify what protocol is being used.  We seem to have got this working pretty reliably now.  The sticking point is that the deep packet inspection system needs to see a few packets before it can identify the protocol - usually you'd allow or refuse the connection immediately, but for DPI to work you have to allow all connections for a while and then terminate any that you don't want to allow.  We're finding that allowing the first 10 kilobytes seems to work reasonably well - after that we chop any connections that haven't been identified as Skype.

Of course, all this was massively complicated by the fact that, unbeknown to us, Skype had a bug which made video unreliable - we found that out on Wednesday when Microsoft released a new version to address the problem.  But not before spending a lot of time trying to figure out what was going wrong (did I mention that Skype problems are almost impossible to debug because absolutely everything, including the debug log, is encrypted so you can't examine it?)

The original intention was to implement deep packet inspection in the new firewall system which we are developing, but by popular demand we've backported this to the existing firewall.  There is currently no user interface to set up the Skype DPI rules, but we can manually set them up for customers on demand for the time being.

Anyway, a moderately successful week - we're still testing the Skype rules, but they should be available Real Soon Now™.