project management

It’s not good enough to blame the computer. Blame the lack of tests

  • Friday, 2 March, 2007
  • By:

It’s education lottery time. Parents are receiving letters bearing good or bad news from schools telling them if their little lovelies have got places in their chosen schools or not. Choice being the big thing – although there isn’t really such a thing as choice, it’s more a preference. Parents have have three choices, ranked one two or three. Good schools have strict admission criteria, and will usually only accept “first choice” children. Don’t get your child into your first choice school and you are at the mercy of the local education authority – no good school will accept parents who rate their schools as second or third choices. It’s not an ideal system, but at least all the letters go out on the same day so you get to know where your child has been allocated a place.

Sadly we fall into the unfortunate category of parents whose first choice has not been honoured. Our first choice school has rejected us on the grounds of distance from the school. Their classroom quota has been reached and we didn’t fill it. That is not good for us! So will we be exceptionally lucky and our daughter get a place in our second or even third preferred schools? We don’t know. And Surrey Country Council can’t tell us.

We’ve got a week of anxious waiting before they send their letters out. They’ve got a problem with their “systems.” Well that is the message I get told when I rang them up. “We can’t tell you anything more because we don’t even know” I was told.
Frankly it is not good enough to blame the computer. If something so critical happens in the private sector, SLAs dictate a course of action; this is a severity one problem that will be fixed within hours. Not weeks. But this is good enough in the public sector where faceless bureaucrats can hide behind the computer, cosy outsourcing deals with little accountability mean that no-one really needs to take responsibility. Take responsibility for the pain and distress this is causing me and my family!
So I got their press office number from the website and rang the Head of Communications. She knew nothing of any problems with letters going out, but promised to have someone call me back to let me know what the problem was.

A little while later I got a call back and was told that there was indeed a problem with the computer, specifically in resolving offers, and in particular resolving offers where children have other siblings already in the school. I was also told that they’d earlier had problems in importing data into their systems.

Now the deadline for getting completed applications in is the end of October. That was four months ago. Four months to scan several thousand forms and process the data, and get it right. That is not such a hard task is it? OK, so the process changed for this round of admissions so changes would have been required to the systems, but even more reason to make sure you get it right. It begs the question, was there a test strategy in place? Did anybody do any testing? I believe that the system has been outsourced to Capita. Has anybody at Capita heard of test driven development? It strikes me that this sort of project is ripe for an agile approach to delivery; a business critical system that cannot fail, simple business rules… I wonder if my MP is going to listen to this story…

What if it began with UAT? (Agile and interaction design are compatible!)

  • Thursday, 1 February, 2007
  • By:

There’s an argument brewing that pitches the agile community against the interaction design community. (You get a feel for this here via here ). Personally I don’t get it. The two approaches are a natural fit. You just need a bit of pragmatism and an ability to say “it depends”.

The interaction designers argue for ethnographic research, lots of user research, and only when we have a thorough understanding of the users, their goals and the context of their usage should we start thinking about writing code. The agile developers will say the only thing that is worthwhile is “working software”. Get something out there, quickly, and see what the users think. The agile approach mean that the direction of the software can be changed at the drop of a hat… If you are working in an investment bank, with a small user base who want this approach may be valid. But if you are building a large public facing web site, diving straight into code with little consideration of the users (e.g. their demographics, itentions, motivations etc.) would seem to be somewhat misguided. It need not be like this. At ThoughtWorks we start many engagements with a “QuickStart”. We’ll work with the client to produce a vision of where they want to get to. We’ll listen to end users, sit with them, watch them. Then use models; financial, process and visual to gain concensus and understanding of what the project is aiming to achieve. We do this quickly, focussing upon the outcomes the client are looking for the project deliver.

The visual models are straight out of the interaction designers tool kit: wireframes. Pen and ink sketches of screens, maybe on paper, maybe in PowerPoint or Visio. Alongside this we’ll watch the users, spend time with them and experience how the software will impact their lives. This doesn’t take much time. We don’t look to produce formal documentation – output is scribbled on cards or on flipchart paper stuck on the walls. In two to four weeks we’ve got a good understanding of the as-is situation, and produced a vision of where we want to get to. The benefits of this? The users get a feel for what might be in the application, we can write better stories (requirements) that are more explicit on what is needed and the developers have something more concrete to estimate against. Finally, when it comes to release planning we can visualise what a cut down version of the application might look like (and thus begin to manage expectations that they are not going to get everything in one big bang, and that things will probably change during the journey). This is not big up front design. It is quick and dirty. It is interaction design (e.g. personas, ethnography, low fidelity prototyping, user testing etc) and agile (e.g. rapid, iterative, incremental, tangible… hey! it’s test driven development without code).

Ah! Test driven development? So on my current project we showcased the wireframes and a comment came back from one of the users (someone who will actually use the application, not one of the “business representatives driving requirements).

“What you guys are doing is great,” he said. “UAT at the beginning”.

I rather like this, but with a caveat. All to often the first time the users see an application is when it comes to user acceptance testing- UAT. UAT has always struck me as a bit weird because it is rarely about “acceptance” (even weirder how it is so well embedded in the IT development lifecycle whilst usability testing rarely gets a look in). It is more about testing for bugs that the developers may have missed. It’s not whether I, as a user “accept” that the product is any good. It is probably the first time I’ve seen it. Maybe I’ve got a test script to run through, I can play with the application, but it is so late in the lifecycle that if I don’t “accept” that it is any good, nothing is going to change.

Agile proponents will argue that with small, regular drops, users are able to raise concerns earlier in the software lifecycle. Regular showcases at the end of each iteration will bring to light issues around “acceptability” and acceptance of the software. But these showcases are usually with the “super user”, the “customer” who is feeding the team requirements. Rarely is it the little person who will actually use their application. And then, even in agile, you’ll often have UAT. It will come at the end of a number of iterations, before the first “release”. But already the direction of the application is set. There is less to accept (and therefore it is easier for the direction of the development to change – one of the benefits of agile) but there is still scope for users to dislike and reject what is being released. And that is why doing UAT at the beginning, user acceptability testing, rapidly, user the interaction designers toolkit is such a good thing. You’re getting the users involved. Your testing and refining the vision. And probably getting usability in there by stealth.

Here’s a paper I wrote about this a while ago: Agile and Interaction design paper

Is your IT department like the NHS?

  • Thursday, 11 January, 2007
  • By:

Can Gerry Robinson Fix The NHS? has been compelling TV viewing the last few nights. The thing that really comes across is the lethargy and the inablility to take responsibility to get things done. The two messages that resonate with me are listen to people on the ground and “just do it”. CEOs may chuckle at such incompetence in the public sector but how often are these issues manifest in the corporate world, and in IT organisations in particular. IT is full of reasons why not:

“You may be able to do xyz in Ruby on Rails, and it may only take you two weeks to complete (compared to the six month project we’d take to do it) but we just don’t (and won’t) support anything that’s not on our architecture roadmap…”

“You can’t do abc until we’ve completed our infrastructure refresh project.”

And how often is interviewing / listening to end users on an IT project plan?

What’s the value in changing colour?

  • Thursday, 14 December, 2006
  • By:

As a registered user, I want to change the colour scheme on the web site so that…

Where’s the value in this story? Agile focuses upon business value, and in doing so it commoditise features. The sponsors of the development are invited to prioritise features based on their “business value”. This means that seemingly pointless gimmicks will slip out. And this may impact the overall experience that the sponsors strive for. Yet by commoditising features the sponsor sees how the costs break down. To have functionality that changes the colour of the site will require effort. It’s not going to come for free. And that means that either scope will increase resulting in either increased cost or increased time. The challenge is when you have a sponsor who knows just a little: “changing colour? Pah! That’s a bit of JavaScript and changes to the stylesheet. I could get the code on hotscripts and knock it up in front of the telly tonight…”

And of the requirement to change colour on a site, in fact that whole customisation thing? Until recently I’ve never seen the value of it. After all, how many people have you seen with a personalised theme on their windows desktop, or even just changing the desktop background? One reason people don’t do this is because they are lazy. The call to action to change it is hidden behind a right click, and it’s not exactly straight forward to do. But there’s a bunch of new sites that challenge the user’s laziness. These seemingly pointless customisation features are part of the overall experience. And they work. And by doing that, they add value to the site.

Don’t say UCD is incompatible with agile

  • Wednesday, 13 December, 2006
  • By:

Consultancy is great, you get to see lots of different problems that different organisations and people have. You get to apply the learning’s built up through experience to help solve problems, and sometimes help organisations realise opportunities that they may not have realised possible.

I was recently working with a new client and sat through a presentation by an agile coach from a different consultancy on what agile is. It was great to hear a different slant on the agile proposition and how it delivers business benefit early and regularly. The presenter really honed in on human behaviour. She was suggesting that despite our best efforts and beliefs otherwise, human behaviour is fundamentally unpredictable. The best plans in the world are held hostage by human behaviour…

At the end of her presentation there were a number of great questions from folk more happy in the waterfall space. People who have seen agile and it’s like before – RAD, DSDM etc etc, “yeah, yeah, heard it all before. Agile’s just another passing fad. Then one of the HCI guys asked the question: “in what you have said, it sounds like you would have little time for doing low-fidelity or paper prototypes; a fundamental tool in the interaction designers arsenal of techniques to ensure usability (and acceptability) in design”.

To give her credit, the presenter explained that agile practices are by nature pragmatic and there may be instances where such prototypes may be valid, but in principle she suggested that they should be eschewed.

Something bothers me here. We are quite happy to “spike” technical problems, but possibly the most important part of any software implementation, the user interface, we are happy to let emerge according to the developers’ preference? For it to be refined following feedback from each showcase? What if we call the lo-fi prototype, the wireframes a “spike”. Does that make it acceptable to the agile zealots? What if rather than writing code, testing and refining it, we draw storyboards, test and refine them? Get the UI right quickly and cheaply before a line of (costly) code needs to be developed. It helps you to have a shared vision of how the application will be manifest to the customer. It helps you to show what they will be getting in an iteration, what the goal is for a release.

To say that user-centred design is incompatible with agile (which was certainly the impression the presenter left the UI team I was working with) is nonsense. Which reminds me. I’m supposed to be writing a paper to this effect.

The product is irrelevent. It’s the customer experience that pays the bills

  • Friday, 17 November, 2006
  • By:

Why do organisations exist? To make money.  In corporateSpeak “to add shareholder value”.
Where does money come from to deliver shareholder value? Customers.

Many organisations seem to forget this slight detail. It gets lost in the internal machinations and politics.
Why do individual funcitons exisit within an organisation? Marketing, product development, IT, sales, operations… None of these add shareholder value make money in themselves. Yet in all too many organisations these different functions behave as if they exist for themselves, they are entities unto themselves. Each entity may pay lip service to the “customer” but how joined up is the reality?

How familiar is this.

The product team identify the need for something, based on anecdotes from the sales guys, or something they perceive to be wrong with the existing product or a hunch about a potential niche in the market. A niche in the market but is there a market for that niche? A bunch of requirements are written up and an optomistic benefits case is written up. There are x million widget users in the market place. Y% of these would benefit from NewGizmo. We project 10% of these will buy at a price point of £z. The requirements are thrown over the wall to IT to put some development costs on the project. The project has yet to start and a launch date is set in stone. And a project manager has yet to be assigned. Marketing are brought into the picture and tasked with how are we going to sell the product. Development starts (the new project manager is aghast at the scope and timescales he is inheriting). Operations enter the project and throw water on the fire. There a whole bunch of compliance and business as usual considerations that only now come to light. Relationships become tense as the project manager tells the product team that they can’t have what they want by the date they wanted it. The date has been set in stone, so the product team throw out some of the initial requirements, based upon an argumentative workshop on Thursday afternoon (noone bothers to refer back to the initial benefits case, even less any thought of how their decisions impact the overall customer experience). And inevitably a cut down NewGizmo is launched to the market a month late and sales are nowhere near what was projected. What we need is NewGizmo 2.0 and so the cycle starts again…

What if we put the customer at the heart of the organisation.  What if everything we do is motivated by serving the customer?  What if we forget all about our products and focussed upon what our customers’ need and how we can delight them.  What if the products become an enabler to a compelling customer experience, rather than an end in themselves?

Isn’t shareholder value all about generating revenue? Isn’t revenue driven from customers?  Aren’t customers driven by compelling experiences rather than mediocre hassle?

The project see-saw

  • Monday, 13 November, 2006
  • By:

Most organisations have separated their IT function with IT offering their services at a price. There are good reasons for this, but it is not without its issues. Prashant Ghandi has written some good stuff on this. All too often the separation is more a divorce. This manifests itself in the “business” creating projects (hopefully with clearly defined benefits) and throwing them over the fence to IT to deliver. This would not be an issue if the project is founded on a solid benefits case, with benefits matched against project costs. But all too often the relationship is like a see-saw, with benefits and costs bouncing up and down and focus on the business case is lost. Far better to work together in partnership, building the business case together, focussing upon delivering strategic value to customers rather than balancing the project on a fulcrum of mistrust and compromise.

Project in balance

If you were a bank, which bank would you be?

  • Wednesday, 8 November, 2006
  • By:

In the last year, Barclays have rebranded themselves. This is not just a change in the logo (as apparent with NatWest) but a focus on the customer experience. Go into a branch and there’s a calculator on the desk to add up those cheques you are paying in. And a pen to fill out your paying-in book. A free pen. A nice touch – they don’t assume their customers to be thieves and chain the pen to the desk – how much does a pen cost? And put a logo on the pen and you have free advertising.

Free pen at Barclays

On the counter there’s some technology to enable customers to rate the experience they’ve just had. Capturing customer feedback at the time of engagement, better than a focus group at a later date in a stale market research suite.

feedback at barclays

Outside the branch they don’t call the hole in the wall an ATM. They call it what their customers call it. A Hole in the Wall. All these things make me feel warm towards the brand. Brand Warmth is something that marketeers measure. Customers who feel warm towards a brand a more likely to consume that brand. In marketing research, perceptions of the brand can be tested further using word association. “If AcmeBrand was a car, what car would it be?” “If AcmeBrand was a tablet and you took it, what would happen to you?”

I think that we can borrow some of this thinking when we build software. I’ve written previously about emotional requirements – how do we want users to feel when using what we design and build. Before we start on something new, ask the question “how warm do we want our users to feel when using this product?” It’s a useful level to pull; warmth is probably going to cost more.” “If this product was a car, what car would it be?” If the customer says “Ferrari” and we are thinking Ford Mondeo we need to focus on managing expectations. A Ferrari is going to cost a lot more than the Ford. I’d like to know what I’m going to get before I start bragging about the new sports car I expect to soon be parked in my drive.

Customers are not your enemy

  • Tuesday, 7 November, 2006
  • By:

Paired programming can be daunting to developers who have not done it before. The thought of some someone else looking over your shoulder whilst you work can be met with resistance. And yet the benefits of the transparency that paired programming brings are well documented.

B2B organisations display similar traits to the developer who is resistant to paired programming. They don’t want anyone looking over their shoulders until they are comfortable with what they are doing. And that includes customers. In my experience, the bigger the “B” and the smaller the “2B” the harder it is to gain access to the people we are developing the product for.

It’s not like this in the B2C world, where organisations understand the need to talk to their customers, to do market research, to test propositions. In the B2B world, organisations become far more protective. There is a fear that if we talk to our customers we will be making promises that we cannot keep. We will be building expectations that will inevitably be broken. In the B2C world you can show what you like to customers, if the final product bears no resemblance to the ideas we’ve tested in focus groups it doesn’t matter. The conversation with the customer is part of the process. We do not assume any “customer memory,” indeed the fact that we have explored different ideas and propositions with customers is seen as a positive; it can feature in the PR associated with the product release. If you release a product to market that looks nothing like what you’d explored in early focus groups it really doesn’t matter. You are selling to a segment that is bigger than a focus group of 12. But what if you customer base is small. What if there is a customer memory, that the people you test propositions with are the people we depend on selling to. The people who will keep us in business. In such a scenario it is understandable why you become protective of your ideas and fear saying anything to your customers until you are confident that you can deliver it. Trouble is, by that stage it may be too late. You are committed to a path of action that can be costly to change. What is required is some transparency and honesty; creating a dialogue as early as possible.

So here is the challenge for B2B organisations with a small, high value customer base. Open yourselves up. Be honest, open and transparent. Web 2.0 is all about collaboration and social networking. That shouldn’t just be about me sharing my photos on flickr, my videos on YouTube, my bookmarks on delicious. Product managers should be blogging about what is happening on the product they are developing for their customers. Customers should be commenting on the progress that is being made; they should have a direct channel to the development team, not via layers of corporate Chinese whispers. Here’s a challenge to the business in B2B organisations; get blogging. Stop fearing your customers and worrying about failing to meet what you perceive them to expect. Once you are transparent to your customers and proactively manage their expectations you can venture on a more successful and profitable journey as partners, not as adversaries.

Ditch the feature shopping list. Think customer journeys.

  • Thursday, 19 October, 2006
  • By:

Let’s imagine a mobile phone provider that that has an on-line presence as well as a high street retail network. Their current website was built several years back on legacy technology; it is slow and has a conversion rate lower than industry norms. Along with having poor usability, the current shopping cart functionality does not support the concurrent usage figures that are hitting the site. The business takes the output from their web metrics and throw these at IT and demand improvements. And a new IT project is born. Rebuild the existing site on a new platform. They get a design agency to handle the look and feel. The functional requirements are built upon the existing functionality and chunked into functional silos:

  • Content managed brochureware site
  • Phone selection
  • Tariff selection
  • Shopping cart
  • Order management

A typical project process. That is flawed. It is starting with a functional premise. An alternative is to think in terms of the customer and customer journeys.

We can start by asking who the customers are. Almost certainly the marketing department has a customer segmentation model – this is a good place to start. That may give us basic attitudinal and behavioural details on customers, but we need richer data. How do customers shop for phones? So we go and spend time in the stores and observe how people buy them. It soon becomes clear that the choose phone – choose tariff buy model is a journey that is only taken by a certain number of customers. Other customers come into the store with intentions – they don’t know what phone or tariff they want, but they know what they want a mobile phone for. Other customers come into the store asking lots of questions, they are doing research; they leave the store with the sales guy’s number, and costs for a couple of phones and tariffs. We look at the competition and see how they sell phones. We look at Amazon and eBay and expedia and see how other retailers sell products and experiences on the web. We synthesise our research and suddenly we have a whole bunch of new requirements. Gasp! Scope creep. Indeed if we list them out into functional silos again:

  • Content managed brochureware site
  • Lifestyler phone selection wizard
  • Save a quote
  • In-store quote save for on-line fulfilment
  • On-line purchase for store fulfilment
  • Phone selection
  • Tariff selection
  • Shopping cart
  • Order management
  • Etc

The business is excited and IT is despondent. When the business announce the date all this must go live by, the letters of resignation land on the IT directors desk and they start looking for contractors. It does have to be this way. You can have your cake and eat it.
Rather than thinking about vertical functional silos, we should think about horizontal slices through the functionality. Slices that support customer journeys. We can start with simple journeys to start and build complexity as we prove our process and new platform, mitigating technical risk as we progress.

don't functionally silo, slice by customer journey

The first release probably needs to demonstrate the new platform works: we can deliver on time; that a new shopping cart interfaces with our legacy warehouse order management system; etc. What’s the bare minimum we can do that does this and delivers business value. How about a microsite that sells a single product. Customers are directed to the microsite via a URL published on a flyer handed out in the stores.

As a customer who has picked up a special offer N80 deal flyer, I want to buy a Nokia N80 on a pay as you go contract

Our acceptance criteria:

Given I enter my personal details and credit card details When my credit card details are validated Then send order to warehouse to dispatch phone and activate contract.

We can decompose this requirement into discrete requirements, “stories” of sufficiently small size and estimate them. We’ll soon get a project velocity and because it is only a small release expectations, risks and dependencies will be easy to manage. We’ve not had to wait for months to get something out, everybody is happy.

We identify that a profitable segment of the market are the aspirationally clueless. People who want a mobile phone, realise they are a minority who don’t have one but are too afraid to admit they have no idea what they want. So we build a new customer journey.

As an aspirationally clueless I want say what I do and how I live my life and have a suitable phone selected for me.

OK, not the finest story, but you get the point. This story might take elements of the phone selection and tariff selection functional silos, but just enough to realise the required functionality. Because we are building around customer journeys, just enough to realise customer intentions that support those journeys we build only what is required. We are not building by feature list. We don’t over promise and under-deliver. We are building trust with all stakeholders. Surely this is the better way to build software, thinking about customer intentions and incrementally delivering to support more complex customer goals. Sadly, people all too often get fixated by feature lists. Because that is what they are used to. Because that is how products are sold. But isn’t there a marketing 2.0 where we sell experiences and go beyond the product. Isn’t that what Virgin does? But I’m probably off on a tangent.

4 of 5
12345