Was it just a simple database query?
So the sensitive personal details of 25m people has been lost and there is a huge political furore over it. Whose fault is it? As far as I can see, (and this is my personal opinion,) blame must lie with IT, specifically the IT contractor and either the contract they work with or the perception of that contract.
The National Audit Office asked HM Customs and Excise for child benefit in “desensitized form”. Sensitive details were specifically asked to be removed, ostensibly to make the file size smaller. This would require a bespoke query to be run. It was deemed too costly so it was assumed that a full extract of the data would do. The fact that this was then burned to a CD, posted unregistered mail and lost is not the point (that is stupidity). What is the point is the IT contract prohibits the business (in this case the governmental offices) to do their job properly.
What sort of contract demands extra payment for a simple database query for “NI numbers, child benefit numbers and children’s names in order to select a risk-based sample of cases to audit as part of anti-fraud work“?
Surely this is an extra request that an experienced database analyst could easily run in the course of a day? If not you must ask why not – is it because the database is badly designed with nested tables and stored procedures and stuff that would make a decent DBA go eugchhh (I’ve seen that happen). If this is the case, the IT contractor has done a bad job; if an electrician worked in your house and left a mess of an electrical installation, would you keep employing them, even if they were cheap?
Maybe however it would not have incurred a cost and this was just the perception; “we must not… run additional scans/filters that may incur a cost to the department”. If this was case it suggests a breakdown in the relationship between the business and IT, with tendency towards the confrontational and transactional rather than co-operation and partnership.
Organisations that outsource their IT often fail to realise what the true costs are. Anything outside the terms of the contract is a change request. It is not unusual for the request itself to incur a cost (someone has to write the documentation, specifiy the design, estimate the effort) before a line of code is written. (At one organisation I worked with that had outsourced their IT function, I was told that to add some basic client-side field validation to a single field on an application form on their website was likely to cost in the region of £60k). The business starts to believe that everything costs and IT becomes a hindrance and a vicious cycle commences.
How could things have worked differently? Let’s say the HMRC IT department was run on more lean/ agile lines. With agile it embraces change. The request comes in (let’s assume such requests are not regular occurances) and in the morning stand-up the BA describes the request and asks the developers for its feasibility in a word. Someone says “yes, I ran a simiar querry last month, it’ll take me ten minutes”. (In reality double or treble that estimate), but it will not have an imact on the developers ability to get thier prioritised work completed. Alternatively the developers say “given the database structure we have inherited that’s a lot of effort” or the project manager says “another request?! pritoritise it like the others!” and it is prioritised in the weekly iteration planning meeting (pushing something else out) and then it gets done.
My hope is that when the inevitable investigation takes place, they don’t just look at the policies and procedures, but also at the underlying structure of the way that IT is managed.