New web filtering module

Today started with fixing a few bugs in the new web filter code: firstly, it wasn't always telling Squid to decrypt SSL connections when necessary, and secondly the overrides didn't appear to be working at all.  Both of these were logic errors - the first one actually already existed in the old code too, although it doesn't seem to manifest itself.  The overrides problem was down to setting the initial filter state incorrectly - previously it hadn't shown up because we didn't aggregate the states of the individual filters until after we'd run all of them at least once (which caused the states to be set right), but now we can aggregate the states before running some of the filters and so the incorrect initial states broke things.

I've also been adding a new filtering module to the web filter - this will scan content for URIs and look them up in our URI categorisation database.  The idea is that websites often link to similar sites, so aggregating some of the information we have about linked sites should allow better categorisation.  Most of the web filter is very modular, and adding a new type of filter should be easy.  However, the content filter code is quite old and the newer more modular design has kind of been shoe-horned around it.  Eventually that code will need refactoring to bring it in line with the rest of the filter, but that's a lot of work for very little gain, so I'm not going to get into doing that just yet.  The new filtering module needs access to all the buffers the content filter currently maintains, so for now the new filtering module is going to be integrated into that existing code.  Its half-done now anyway, and happily extracting URIs from the page content, so it shouldn't be too hard to look those URIs up and feed that data into the categorisation engine.

While all this development has been going on, we've also been trying to tackle an odd issue with Apple's "Find my iPhone" service.  We've got customers using this service with no problem, but it just doesn't seem to be working on one of our customer's networks.  The odd thing is that we can see regular requests to gs-loc.apple.com from the "locationd" service on these devices, and they seem to be succeeding just fine.  But whilst Find my iPhone says the devices are online, it says it has no location for them.  Its quite frustrating, and of course Apple don't provide any information on their protocols, so debugging the issue is a case of reverse engineering it all.  So that's another one to investigate some more tomorrow.