dancingmango

Petition

I’ve written in the past about the government’s abysmal track record on IT development.  I met with the local MP to discuss the issues but he didn’t really get it; he sent me away to write a policy paper for him which I really had time for…  So good news that someone is doing something about it with a petition on the Number 10 website.

In his recent update on the progress of the petition, Rob Bowley mentions the Rural Payments Agency project.  I can’t attest to either have been an ‘expert’ or to have had a salary anything near what he mentions, but I was a consultant on that project so nod in informed agreement.  That experience gave me a benchmark to compare ‘bad’ ways of going about an IT project to compare with the ‘good’ world of lean and agile that I now inhabit.

Please sign the petition.

Are you prepared for the dip?

So you are refreshing or rebuilding your website.  You are introducing new functionality and features, and sweeping away the old. You’ve done usability testing of your new concepts and the results are positive.  Success awaits.   You go live.   And it doesn’t quite go as you expected.  You expect that the numbers and feedback will go on an upward trajectory from day one, but they don’t.  What you should have expected is the dip.

In October 2009 Facebook redesigned the news feed.  Users were up in arms, groups were formed and noisy negative feedback was abound.  A couple of years back the BBC redesigned their newspage, “60% of commenters hated the BBC News redesign“.  Resistance to change is almost always inevitable,  especially if you have a vocal and loyal following, you can expect much dissent to be heard.  What is interesting is what happens next.  Hold your nerve and you will get over this initial dip.  We’ve seen a number of projects recently where this phenomenon occurred; numbers drop and negative feedback is loudly heard.  But this dip is ephemeral and to be expected.  The challenge is in planning for this and setting expectations accordingly.  Telling your CEO that the new design has resulted in a drop in conversion rate is going to be a painful conversation unless you have set her expectations that this is par for the course.

Going live in a beta can help avert the full impact of the dip.  You can iron out issues and prepare your most loyal people for the change, inviting them to feedback prior to the go-live.  Care must be taken with such an approach in the sample selection o participate in the beta.  If you invite people to ‘try out our new beta’, with the ability to switch back to the existing site, you are likely to get invalid results.  The ‘old’ version is always available and baling out is easy.  Maybe they take a look and drop out, returning to the old because they can.  Suddenly you find the conversion rates of your beta falling well below those of your main site.  Alternatively use A/B testing and filter a small sample to experience the new site.  That way you will get ‘real’ and representative data to make informed decisions against.  Finally, don’t assume that code-complete and go-live are the end of the project.  Once you are over the dip there will be changes that you can make to enhance the experience and drive greater numbers and better feedback.

A leap of faith

We recently pitched to a prospective client.  We told a story of how we believed their customers would use the new product they had sketched out in the RFP; we told a ‘day in the life’ and brought it to life with illustrations.  We walked through our approach to tackling projects and described our experience on similar engagements.  One of the bid team described his experiences and what the client could expect and we guaranteed that he would be on the project.  We let them have a rate card, and an indication that, comparing what we knew of the project with others that we have successfully delivered, what they indicated they wanted in the RFP was in the range of their budget (which we knew).  We let have them directly relevant references who were ready to take their calls.

What we didn’t do was produce them a project plan, nor a definitive cost.  Given the paucity of information in the RFP there was just not enough to go by; it would be a lie if we came up with a number, there were too many imponderables.  Yet that was what they craved the most.  Most of the questioning was friendly and we answered well, however one question sticks in my mind: “From your presentation, you are asking us for a leap of faith to engage you”.  Well yes, we are.  But is that any different to the way you engage any new contractor?  It is always going to be a leap of faith; even if you engage one of the big trusted brands with an established reputation.  Sainsburys and NHS took a leap of faith when they engaged my old employer Accenture; they probably felt they had done all the necessary due dilligence and were partnering with a proven, reputable organisation, but in those two instances good things did not result.

About a successful project that was a failiure

On time, on budget, to the scope that was agreed from the outset of development.  A successful project?  Well no actually.  It was a complete failure.

Here is a story about an insurance company with a number of differnet products sold through intermediaries.  Whilst the intermediaries were good at selling single insurance products, they weren’t so good at cross-selling or up-selling other products.  Focus groups with the intermediaries revealed that they didn’t know about all the other types of insurance available through the company.

What if the intermediaries could have a portal where they could access all our insurance products in the same place with customer alerts and sales support prompts identifying further selling opportunities?

From this initial idea a benefits case was pulled together consisting of a product definition and financial projections.  In pulling together the benefits case, the potential revenue uplift numbers surprised everyone.  Signing off the benefits case on the new Intermediary Portal was duly signed off, and the product definition was handed over to IT to build.

Being an agile IT shop, the business and developers sat down together and got their heads around the product definition.  It soon became clear that the challenge was one of “single sign-on”.  Each of the insurance products offered were on a different legacy application that required the intermediaries to sign-on with different credentials.  To bring them all together in a single portal was far harder than the simple problem that the initial product definition suggested.

In pulling together the benefits case, a rough estimate had been supplied by IT. Now it was an in-flight project with an initial list of stories, it became clear that they had significantly under-estimated.   Of the twenty different products that the business wanted on the portal, for the budget the business had set aside would deliver barely four products.

With new estimates a release plan was drawn up.  Release one would deliver single sign-on across four products identified by the business as being most profitable.  All the  sales support tools were de-scoped and scheduled for a third release with the second release delivering single sign-on for the remaining products.

Development started, the business stakeholders worked closely with the developers and the First Release of the Intermediary Portal went live with congratulations all round.  Funding for the next release was lined up depending upon the success of the first release.  But that success never came, take-up was less than expected and the cross and up-selling never materialised.

The proposition to the intermediaries as delivered was flawed; the portal had to be all or nothing, single sign-on across four unrelated products was not compelling to them.  There was no sales support.  The intermediaries thought “so what?”  IT had delivered on the business requirements yet the project was deemed a failure.

This story tells a striking lesson. The project failure was due to a lack of joined-up thinking.  The business and IT both had followed their processes and done the right thing.  The business had identified an opportunity, built a benefits case and had this signed off.  IT had run a model agile project with close engagement with the business.  However whilst both stages of the process were locally optimised, they were done in isolation of each other.  Once the (development) train had left the station both sides were committed to delivering the product portal.  No-one returned to the business case, no-one went back to the end users, the intermediaries and asked whether the cut down scope for the first release would actually be of value to them.  More importantly, IT were engaged too late in the process.  The business had settled on an IT solution to the problem without engaging IT.  Had IT been party to the ideation and visioning process they would have been able to raise the risk of the project complexity earlier on.  Indeed they could have killed the project before it started.

Returning to the initial problem; “intermediaries weren’t so good at cross-selling or up-selling other products… Focus groups with the intermediaries revealed that they didn’t know about all the other types of insurance available through the company.”  The problem didn’t need a portal solution. The issue was one of awareness; almost certainly an off-line marketing campaign would have delivered a greater ROI without the need for IT to build the wrong product.

You need to accelerate and change gear to get up to speed

Velocity is a simple concept to grasp when planning and managing an agile project.  It is one of the first formula you learn in maths (or is it physics), speed equals distance over time.  On a project the distance is scope, measured in the number of ‘points’ you need to complete.  So if the team has a hundred points worth of scope and a velocity of ten, they will complete distance (scope) in ten weeks.  It’s a simple calculation, but is dangerous and wrong.  It fails to take into account acceleration.

Just in the same way that it takes a car time to accelerate to a meaningful speed, so a project takes time to reach it’s planned velocity.  To reach 60 miles an hour takes ten seconds and means moving through the gears.  You don’t put your foot on the accelerator and suddenly find yourself doing sixty.  Nor do you put the car into forth gear and expect to move without stalling.  It is the same in agile projects.  Velocity is misunderstood; you cannot expect to have your planned velocity immediately without acceleration.  Similarly, putting a fall sized team onto a project is like trying to start in forth gear.  You will stall.

When planning an agile project you need to consider acceleration.  The first iterations will be slow as you come up to speed.  Secondly you need to be in the right gear for where you are at.  Start in first (small team) and change gear (ramp the team up) as the velocity requires it.  Better to have the engine screaming in first (change gear) than to have it stall before it’s even got going.

Sadly, this means the simple pictures on burn-up and burn-down charts are wrong.  They need to take into consideration acceleration and appropriate gearing.  And that is advanced maths.

Are you managing expectations beyond the team?

There’s this idea called the Disconfirmation of expectations theory that states that having unrealistically high expectations from the adoption of a new IT application will result in lower levels of realised benefits.  Get customers excited about a new product and fail to deliver on it and you will have unsatisfied customers.  And unsatisfied customers are unlikely to use the product to its full advantage.

There is a risk with products developed using agile approaches that they fail to deliver on their initial promise.  The immediate stakeholders know that the product will evolve incrementally, but is this true of the broader audience? Are they aware of the intended regular heartbeat of delivery or are they expecting a fully featured product at the first release.  How are you managing expectations beyond the immediate product team?

Be wary of what you say early on.  Creating a vision is essential but be mindful of how this is communicated. Early demos, proof of concepts, prototypes, wireframes often show a vision of the end goal, several releases into the future.  Words are easily forgotten, explaining that this is an end goal vision is not enough, you must show a vision of what the cut-down product for the first release is and ensure it is appropriately communicated.

Expectations work both ways, it is easy for the business to tell IT their requirements and assume they will be developed in their entireity in one go.  Similarly it is easy for agile developers to expect the business to understand their incremental approach to delivery. The key to success is effective change management; identifying all stakeholders (both core and peripheral) and create a culture of agility that goes beyond the immediate project team.  In a large organisation that maintains more traditional approaches, agile projects must be supported by a well designed communication plan that builds the relationship between both IT and the business. Identify whose life will be touched by the product and develop a strategy for communicating to them.  This doesn’t mean “they can see what is going on on the project Wiki” this means someone taking responsilibity for listening, engaging and evangelising on the product, the project and its goals.

Have you considered the cost of context switching

One of the great things with running a project with Microsoft Project is that it enables you to allocate resources to activities and you can assign the time they work on each activity.  A resource can then work on several projects at the same time, 50% on project A, 50% on project B.

With this ability it is easier to accommodate multiple projects that the business request.  Rather than having to say we can’t start on a project in six months when the current one finishes, we can run both projects at the same time, using the same resources.  We can do things in parallel rather than in series.

Only it doesn’t really work like that.

One of the great things with running a project with Microsoft project is that it enables you to allocate resources in a way that hinders productivity, effective planning and quality.

The problem is with context switching and the overhead that multi-tasking brings.  This article succinctly describes the problem.

No matter how efficient you think you are, multitasking comes with a high cost. Because we’re people, we don’t swap out the content of our brains as easily as a computer does, and we definitely don’t swap in the old state when we’re ready to return to the original task.

Gerald Weinberg, in Quality Software Management, Vol. 1, Systems Thinking (Dorset House, 1992), estimates the context-switching cost among three tasks to be 40 percent. That means that 40 percent of your available work time is spent on non-task activities. The rest of the time is split among the three projects. So, if you thought that in a 45-hour week, you could spend 15 hours on each of three tasks, don’t kid yourself. You’re really spending eight hours on project A; eight hours on project B; eight hours on project C; and 24 hours context-switching, figuring out where you were and what you have to do next. The time spent on each project works out to about half of what you expected.

So whilst it is easy to appear to please business sponsors by taking multiple projects on at the same time, and the model of working in parallel rather than in series being a politically favourable approach, in fact the costs of multi-tasking far outweigh the benefits.

How long does it take to start. I mean really start?

Let’s assume that you are not bought into the whole agile thing.  That doesn’t mean you can’t look at your IT organisation and identify waste and fat in the process.  How long does it take from the business having an idea and requesting IT to build something to the developers actually starting to code?

This seems to be common: The initial ‘idea’ is fed into the PMO team, it is documented with a high level scope, rough business case and napkin estimate of +/- 100%.  Elapsed time (i.e. the time from the first email or conversation requesting the requirement through to the initial scope document being circulated and approved): two weeks, value added time (i.e. the time actually spent thinking, doing or deciding); two hours.  The project, having gained approval in principle, is then prioritised.  Some organisations actively monitor thier project protfolio, others do it annually, with the business having to put in project requests when the budgets are set.  Mid-cycle and the project is unlikely to take off.

Let’s assume PMO agree the value of the project, next step is high level design.  Analysts capture high level requirements, ascertain what the business really wants and refine the business case further.  IT puts some high level estimates against the requirements, with a +/- 80% confidence.  Elapsed time: eight weeks.  Value added time two weeks.  With a refined business case to take to PMO or the steering commitee all that has changed now is that we have some more detail and a guess on what it might cost.  We are ten weeks down the road and we still have not made a decision whether the project will commence.  This due dilligence is notionally about reducing risk and keeping costs in check, but in reality, what value has been added?

Now it is time for detailed design.  Another (elapsed time) eight weeks of analysis, drilling down into the requirements.  Documentation follows workshops, only now the specification is no longer speaking the language of the business.  Use cases, UML, it’s all getting slightly technical and the business are not really sure what they are reviewing.  Let’s call it another couple of weeks of actual value that is added.

Eighteen weeks elapsed time, countless meetings, momentum and still no decision on whether project will start, let alone a line of code being written.  But the business case is really taking shape and IT have got the estimates down to a 20% confidence limit.

The project gets the go-ahead, but it is not yet time to start coding.  Technical design needs to take place, four weeks of architects architecting and documenting the spec.  Six months has elapsed (of which maybe a month was actually doing stuff that positively contributed to the success of the delivery) and finally the developers start writing code.

What value did that six months deliver?  Requirements, design, business and estimates.  Yet we all know that the requirements will change, and with that the business case.  The estimates will be way out, but we’ll justify the process of estimation because they would have been right it the requirements hadn’t change during the build…

Knocking the process is easy.  What’s the alternative? Start with a burning desire to release something of value at the earliest responsible moment.  Get the business in the same room as the analysts and the architects and get them to articulate their vision.  Use personas and more importantly scenarios and customer journeys to drive out the business vision.  Ask the analyst to capture the business intents on index cards.  Lots of whiteboarding, visualisation and pictures to inform the thinking.  Don’t dwell on the detail, this is about capturing the intent of the system, what are the desired outcomes that it will deliver, how will it impact the lives of all the people who will touch it.  Next step is to prioritise these outcomes.  Collate these into the minimum set that would make a coherent and compelling product that you could go to market with.  For the business case make basic assumptions on the revenue that this feature set will deliver (or what costs it will save) and ask the architects and developers in the room to do some high level estimates.  The whole process should take no more than a week (you could get a first cut done in a couple of days) and there is your initial business case.  If we accept that estimates are little better than guesses and things will change anyway, if we have an initial realisable goal that we have demonstrated will deliver value, why not go straight into development.  By actually ‘doing’ you will minimise risks that you can only predict on paper, and value will be delivered so much quicker, indeed you should be able to have something to market, generating revenue in the same timescales that you otherwise spent planning and analysing.  And if you still want to do waterfall, you’ve got a smaller number of requirements to analyse and design for, again, delivering value sooner.

Someone should talk to the minister about agile

So another government IT project fails to deliver.  The National Offender Management Information System had been budgetted to cost £234m (total lifetime cost) and take four years to complete.  Three years in and the costs had spiralled, with a new lifetime project cost estimated at £690m.  The plug was pulled and a new three year project at the cost of £513m was commenced.  Poor project management was blamed, but I’d go further and blame the project approach as well.  The Minister responsible says why;

“As soon as the extent of the projected costs and delays to the project were recognised, we took immediate steps to halt the project and consider the most cost-effective way forward which effectively preserved the work done to date”

So let’s get this straight:

It took three years to recognise that a project to implement a single database had gone wrong. 

Contrast this to an agile project where progress, costs and risks are continuously monitored.  But what does the goverment do?  Continue with the same approach as before with some new project managers on the job.  And wait another three years before any value will be delivered.

The user journey

“Faster, slicker, easier to use.  That’s how we sold it to the business.”  It is a common theme, IT have a system that is costly to maintain, hard to extend, is on a dated platform and no longer fit for purpose.  The business are persuaded of the need for a replacement.

This is what happened at an organisation I recently visited.  But it didn’t quite go to plan.  A number of years later and they’ve got an application that is slower, uglier and harder to use.  What happened was the business intent was distilled into requirements (based upon the existing functionality).  Each requirement was captured as a control on a screen.  These were bundled up and sent to India for coding, and the developers went and built a bunch of screens.  They considered interface design but not interaction design.  They focussed upon technical processes rather than user journeys and the resultant deliverable was something that functionally ticked all the boxes (it did what the specifications said it should do) but was ultimately rejected by the people it was intended to help. The code may have been great but that meant little to the business who found the application a pig to use.  Another failed multi-million dollar IT project.

User interface design is more than just screen design, it is about the journey the user takes to accomplish their goals.  Remembering that could have saved this particular organisation a lot of money.