This impatient monkey

I’m impatient.  I also expect technology to work.  When it doesn’t appear to be working, when I’m getting no feedback as to what is happening, I get frustrated.  I click-click-click the mouse button.  I hammer the enter key.  I repeatedly thump the keyboard.  “Work damn you!” I curse.

The technology was probably just creaking along, it may have got there in the end, but my hammering is the last straw, it breaks the application. Frozen out, frustration turns to user anger.

The developer tuts, “it’s your own fault” he says, “you broke it with your impatience”.  And that is why Dan North’s recent blog about Monkey Testing fills me with happiness.  Testing software for my monkey behaviour, so that it doesn’t break when I do things that I’m not supposed to do – because I am human.

The Total Experience

The customer experience doesn’t start on the home page and finish on the payment confirmation page. A compelling customer experience comprises of a number of factors with the different factors residing in, and owned by different parts of the business. This can result in parts of the experience being excellent, others being shocking, such as at the Early Learning Centre. Projects that are driven forward by “IT” rarely see the bigger picture; the project manager’s primary concern is with delivery. And this means producing a product that covers all the use cases or stories that were specified in the plan. Agile development practices often reinforce this; an agile team is (rightly) focussed upon delivering technical excellence according to the “customer’s” requirements. The trouble with this is that project success in the on-line world goes beyond just delivering working software. Success is all about the total experience.

I’m sure there are a hundred and one different frameworks for “e” success. The following works pretty well for me.

The Total Experience Model.  Copyright Marc McNeill!

Compelling proposition
Any on-line offering starts with its proposition; what it is offering to the market place. Do we know who the consumers are (segment), how many of them are (market size) and their propensity to buy? What is going to attract them to the site, and what is the glue that will draw them back.


Well if no one can find your compelling proposition it’s never going to make you money. Do you have a strategy in place for ensuring consumers can find your proposition? This starts with search engine optimisation, but will extend to affiliate programs and making a noise about you (Product team blogging?). And then when they are at your site is navigation intuitive? Is relevant and wanted content findable?
What is the product aesthetic, what does it look like? Is it branded? Does it exhibit production values that reinforce the brand? How does the overall experience make me feel – how does it emotionally engage me?

Content is king. But all too often it is a bit of an afterthought. It’s not just syndicated content, or articles that subject matter experts might write, it is all the copy, the words that appear on the site. Do they support the proposition, or are they just placeholders that the developers wrote and never got updated? Related to content, and maybe it should have a bubble unto itself is community. I think that increasingly, successful eCommerce offerings will include an element of community. Fulfilling the promise of the ClueTrain manifesto from several years back?

Technical Excellence
Working for a company that undoubtedly employs some of the finest developers in the market place, this is something I see a lot of. But it means nothing if the other elements are not realised. Technical excellence includes those “non functional requirements” that development projects talk about; reliability, performance, scalability etc etc.

Learnable, speaks the user language, memorable, all those old chestnuts. Probably the maxim is “don’t make me think”. Aligned to usability is accessibility; we don’t want to exclude potential consumers through sloppy implementation.

Operational excellence
So the on-line experience may be compelling, but what happens then. Is the fulfilment channel fulfilling? Is there a promise to deliver goods and is the promised delivered upon? What about support channels? Do they support or do they just pay lip service to the concept? Does the web experience go beyond the web and cross other channels? Can consumers start a journey on the website and seamlessly conclude the experience in store? My blogs on Early Learning Centre, Norwich Union and a seamless experience all touch on operational excellence.

Test – measure – refine

Underpinning these factors is the need to constantly test what we do. We can all learn from Test Driven Development (TDD); write the test first and only have the confidence to proceed when it passes. How do we know it has passed? Well we need to measure it; we can only know success if we can quantify it. Based upon the feedback we refine; test – measure – refine, a concept core to agile software development practices.

A lot of words and a picture that belie a simple concept. When working in the web world, always think about the total experience. Break out of your silo (be it technical development or marketing strategy) and think holistically. What will the end to end experience be for your consumers? A technically excellent website is not success. A polished, well branded look and feel is not success. Success is compelling total experience.

What does red mean to you?

I’ve recently been in Hong Kong and ran a really quick retrospective with the project team. I handed out red and green post-it notes and asked the team to write down things that went well and things that went not so well. They then stuck the post-its on the board, red “not so wells” on the left and green “goods” on the right. Only it didn’t quite work like that. In my western mind I’d assumed that green is good and red is bad. Not so in China where red is an auspicious and lucky colour…

Red and green post-its confused in a project retrospective.  cultural differences were forgotten

What if it began with UAT? (Agile and interaction design are compatible!)

There’s an argument brewing that pitches the agile community against the interaction design community. (You get a feel for this here via here ). Personally I don’t get it. The two approaches are a natural fit. You just need a bit of pragmatism and an ability to say “it depends”.

The interaction designers argue for ethnographic research, lots of user research, and only when we have a thorough understanding of the users, their goals and the context of their usage should we start thinking about writing code. The agile developers will say the only thing that is worthwhile is “working software”. Get something out there, quickly, and see what the users think. The agile approach mean that the direction of the software can be changed at the drop of a hat… If you are working in an investment bank, with a small user base who want this approach may be valid. But if you are building a large public facing web site, diving straight into code with little consideration of the users (e.g. their demographics, itentions, motivations etc.) would seem to be somewhat misguided. It need not be like this. At ThoughtWorks we start many engagements with a “QuickStart”. We’ll work with the client to produce a vision of where they want to get to. We’ll listen to end users, sit with them, watch them. Then use models; financial, process and visual to gain concensus and understanding of what the project is aiming to achieve. We do this quickly, focussing upon the outcomes the client are looking for the project deliver.

The visual models are straight out of the interaction designers tool kit: wireframes. Pen and ink sketches of screens, maybe on paper, maybe in PowerPoint or Visio. Alongside this we’ll watch the users, spend time with them and experience how the software will impact their lives. This doesn’t take much time. We don’t look to produce formal documentation – output is scribbled on cards or on flipchart paper stuck on the walls. In two to four weeks we’ve got a good understanding of the as-is situation, and produced a vision of where we want to get to. The benefits of this? The users get a feel for what might be in the application, we can write better stories (requirements) that are more explicit on what is needed and the developers have something more concrete to estimate against. Finally, when it comes to release planning we can visualise what a cut down version of the application might look like (and thus begin to manage expectations that they are not going to get everything in one big bang, and that things will probably change during the journey). This is not big up front design. It is quick and dirty. It is interaction design (e.g. personas, ethnography, low fidelity prototyping, user testing etc) and agile (e.g. rapid, iterative, incremental, tangible… hey! it’s test driven development without code).

Ah! Test driven development? So on my current project we showcased the wireframes and a comment came back from one of the users (someone who will actually use the application, not one of the “business representatives driving requirements).

“What you guys are doing is great,” he said. “UAT at the beginning”.

I rather like this, but with a caveat. All to often the first time the users see an application is when it comes to user acceptance testing- UAT. UAT has always struck me as a bit weird because it is rarely about “acceptance” (even weirder how it is so well embedded in the IT development lifecycle whilst usability testing rarely gets a look in). It is more about testing for bugs that the developers may have missed. It’s not whether I, as a user “accept” that the product is any good. It is probably the first time I’ve seen it. Maybe I’ve got a test script to run through, I can play with the application, but it is so late in the lifecycle that if I don’t “accept” that it is any good, nothing is going to change.

Agile proponents will argue that with small, regular drops, users are able to raise concerns earlier in the software lifecycle. Regular showcases at the end of each iteration will bring to light issues around “acceptability” and acceptance of the software. But these showcases are usually with the “super user”, the “customer” who is feeding the team requirements. Rarely is it the little person who will actually use their application. And then, even in agile, you’ll often have UAT. It will come at the end of a number of iterations, before the first “release”. But already the direction of the application is set. There is less to accept (and therefore it is easier for the direction of the development to change – one of the benefits of agile) but there is still scope for users to dislike and reject what is being released. And that is why doing UAT at the beginning, user acceptability testing, rapidly, user the interaction designers toolkit is such a good thing. You’re getting the users involved. Your testing and refining the vision. And probably getting usability in there by stealth.

Here’s a paper I wrote about this a while ago: Agile and Interaction design paper

Are you being told the truth?

I’ve spent the last two weeks listening to users, understanding their requirements, drawing boxes and arrows, swim lanes and all that good stuff to show process flows. But is what they say the whole truth?

When we are doing analysis we are building our own mental model of the work, and potential solutions to problems we are hearing. But unless we can see what is going on, how grounded in reality will our solutions be? Anyone who has sat in on focus groups and observed consumer reality will know that what people say and what they do are often at odds with each other.

Finally this morning I had access to the users in their “real” environment. The user sat at his desk in front of his monitors and I sat slightly behind and aside him, watching over his shoulder what he did. Half an hour’s observation brought the boxes and arrows to life. More importantly, it highlighted how long he spent on each task, and the need for efficiency in the interface we are developing.

In a workshop he can ask for stuff, but when you see him working it is clear that that stuff would just clutter and get in the way of him accomplishing his goals.

Most striking is what is left out of the process discussions. Sometimes we forget about those things that are so obvious, so mundane. We assume that they just happen. Only by observing can you get a real feel for the requirements (and also how they should be prioritised).

So go on, take a half an hour break from your development activities and go and watch the users working “in the field”. You never know, you might just learn something.

What is your UI debt?

When developers are up against tight deadlines they introduce the concept of “technical debt”. It is a short cut to delivery; quick and dirty code that will have to be refactored (or fixed) at a later date. Inevitably they will pay the price in the future if this is not done.

So why does no one talk about “UI debt?” When projects are up against tight deadlines and scissors are applied to scope, the first thing that the team will look to cutting are any UI “bells and whistles”. They don’t talk about UI debt – we’ll have to refactor (or fix) the UI at a later date. Yet the price to be paid for UI debt will inevitably be far greater than any technical debt. If the UI is hard to use, and satisfies the technical functionality, not user goals, the users are more likely to shun the software. And isn’t the object of software for people to use it?

Is your IT department like the NHS?

Can Gerry Robinson Fix The NHS? has been compelling TV viewing the last few nights. The thing that really comes across is the lethargy and the inablility to take responsibility to get things done. The two messages that resonate with me are listen to people on the ground and “just do it”. CEOs may chuckle at such incompetence in the public sector but how often are these issues manifest in the corporate world, and in IT organisations in particular. IT is full of reasons why not:

“You may be able to do xyz in Ruby on Rails, and it may only take you two weeks to complete (compared to the six month project we’d take to do it) but we just don’t (and won’t) support anything that’s not on our architecture roadmap…”

“You can’t do abc until we’ve completed our infrastructure refresh project.”

And how often is interviewing / listening to end users on an IT project plan?

Consumer driven development

One of the things that makes me passionate about agile is the way that it places the customer at the heart of everything we do. The trouble is “the customer” is often wrong. For me, the customer is the person whose life is transformed by the software they use. This is not the person who is writing the cheque, nor is it the person who is defining the requirement. In consultancy speak, they are the client. The “users” are the customer. I don’t like the use of the word “user” because it implies a choice to use the software has already been made. I’m probably going to have to accept that “customer” is never going to change, so why don’t we call our users (my customers) “consumers”.

So enough about semantics. The team are building an on-line banking product. The team ask the customer (client) what she wants. She bases her requirement around what she knows. Often this will be swayed by the development team talking about implementation detail. (Few things make me as uncomfortable as a customer (client) from the business (product owner) talking about APIs).

So the requirement is to log-in. The customer is comfortable with customers (consumers) having system IDs; a unique n digit number that identifies the customer on the legacy database. It has, after all, always been this way. She asks whether the customer’s credit card number could be used, but the developers tell her (via the BA) that these often change (customers loose their cards) and it would be hard to do. She nods her head in agreement (she’s put on the spot by the BA – the iteration is due to start next week…) And the requirement is manifest on the login screen (“please enter your 16 digit customer number”).

In reality, the “customer” bases the requirement on her experience with the organisation and what she knows about it. The consumer (if they were to be invited into the conversation) wouldn’t come to the problem with any of that baggage. She needs a simple login. She needs a user name that is accessible or memorable. She does not want a 16 digit number.

It may take a little longer to listen to the consumers’ need (we need to find consumers to talk to them). It may mean a little up front analysis (I’d suggest pre-project analysis, doing due-diligence if you like). But it will pay dividends downstream. In the login example above, the developers have swayed a simple and cost-effective solution in the short term, but have failed to consider the down-stream costs. How will the customer find their 16 digit number? Do we have to post it out (cost) on a card (cost) and have a support team (cost) and process (cost) for letting customers know their number if they’ve forgotten it?

As agile grows up, I’d ask for us to spend more time thinking about the real customer, the consumer, the person whose life will be changed by what we build. The inconvenience we may incur during development may be nothing compared to the pain we will experience after we have “delivered”.

Success is more than just features

All too often there is an assumption that we can deliver a bunch of features and they will provide a benefit to the customer. A simple model and it underlies much of what agile is perceived to be about. Deliver those features that provide greatest (business) benefit at the earliest opportunity.

Features leads to benefits

It is worth looking at the updated DeLone and McLean model for Information Systems (IS) success [pdf]. This is a causal model; if you want to accrue benefits with a system it is not sufficient just to deliver “features”. There are qualities that must be realised; these will drive an intention to use and actually use the application. This is tightly correlated with satisfaction, use will precede satisfaction, but if the outcome is satisfaction then use will increase and this is the ultimate test of how beneficial a system is.

DeLone and McLean model for IS success.  Slightly modified.

So taking the “qualities” one by one:

System quality: e.g. reliability, usability, performance, availability, accessibility
Information quality: e.g. usefulness, personalisation, speaking the user’s language, keywords, search terms
Service quality: the overall experience, interface design, emotional impact, trust, security, support

Typically development teams may consider some system quality attributes as “non-functional requirements” but these are generally from a technical perspective (e.g. availability) rather than a human perspective (e.g. usability). Probably this is because this is the language that development teams understand.

Information quality is usually left to someone else, someone from the business, from marketing, to write the copy for example. On the web, where most customer experiences start with search so getting copy right is essential to ensure search engine optimisation. On pages that are not content managed, pages that the developers are coding have meta-tags been identified? Has copy been written with targeted keywords in mind, keywords at the beginning of the page, with repeated use of the keywords etc?

Service quality is something that everybody assumes will be manifest but is rarely explicitly stated as a measurable component. I recently worked with a client that wanted to offer “world class” services. What did this mean? Asking them resulted in different things to different parties. The DeLone and McLean is a little light on “service quality”. Depending upon the application, it is something that looks and feels good, and makes me feel good using it. It is something that delivers a complete and holistic experience from the first point of contact (e.g. hitting the home page) through to realising my goal (e.g. correct products ordered arrive at promised time).

These qualities have a causal impact on use and user satisfaction. These can be measured (e.g. usage patterns/ repeat usage/ surveys) and should be incorporated into the non-functional requirements at the outset of the development.

Get all this right and you will realise net benefits from the project.

What’s the value in changing colour?

As a registered user, I want to change the colour scheme on the web site so that…

Where’s the value in this story? Agile focuses upon business value, and in doing so it commoditise features. The sponsors of the development are invited to prioritise features based on their “business value”. This means that seemingly pointless gimmicks will slip out. And this may impact the overall experience that the sponsors strive for. Yet by commoditising features the sponsor sees how the costs break down. To have functionality that changes the colour of the site will require effort. It’s not going to come for free. And that means that either scope will increase resulting in either increased cost or increased time. The challenge is when you have a sponsor who knows just a little: “changing colour? Pah! That’s a bit of JavaScript and changes to the stylesheet. I could get the code on hotscripts and knock it up in front of the telly tonight…

And of the requirement to change colour on a site, in fact that whole customisation thing? Until recently I’ve never seen the value of it. After all, how many people have you seen with a personalised theme on their windows desktop, or even just changing the desktop background? One reason people don’t do this is because they are lazy. The call to action to change it is hidden behind a right click, and it’s not exactly straight forward to do. But there’s a bunch of new sites that challenge the user’s laziness. These seemingly pointless customisation features are part of the overall experience. And they work. And by doing that, they add value to the site.

7 of 10