dancingmango

Silver surfers

News this week was that Ivy Bean died aged 104.  A good innings, but there is more to the story than that, Ivy got on Facebook aged 102 and was a regular twitterer with more than 62,ooo followers.  One of my colleagues at work announced as he got off the phone with his mother that she had just upgraded the ram in her computer, herself.  My father has just got an iPhone and is hooked on apps.  Maybe they are late to the game, but the over 65s represent the fastest growth in take up of digital technologies.  Whilst mobile ownership for the 25-44 year olds remained static (saturated) between 2007-9, for the 65+ it grew by 17% (source: Ofcom Communication report 2009).  And internet adoption grew by 11%.  The 2007 Ofcom report found that the 65+ spend 42 hours a month online, four hours more than the most active users who are aged between 18 and 24,

It’s an old finding but the Henley Centre reported that the over 65s most felt part of a virtual community thanks to the internet; with the growth of Facebook that statistic is probably out of date, but still worth reflecting on.

The ’silver surfers’ are not the techno-fearing, techno-illiterate luddites you may percieve them to be.  They are a segment of the market that cannot be ignored, and an opportunity that are craving to be served.  Do they figure in your plans? Do you have any personas for the over 65s? Have you tested your propositions with this demographic?  Is your design optimised for the 18-34 demographic who have less disposable income that the older demographic who have a greater propensity to spend?

I’m not sure where this quote is from so I can’t credit it, but it is worth reflecting on:  “The wealthiest generation in the history (and possibly the future) of the earth are in the process retiring. And they don’t intend to do it quietly”.

Thinking inspired by the Agile UX retreat

In the quest to get agile and UX to get along better, and following successful retreats in the US, last weekend Johanna Kollmann brought together a bunch of agile and UX folk for an Agile UX retreat in London sponsored by.  Giving up a weekend was hard, but was worth it, meeting a great bunch of people and sharing thoughts and experiences from the agile and UX camps.  So what did I learn.

Rethink what we do

Coming out of the retreat it is clear that the way we do UX today needs a fundamental rethink.  As UX professionals we have fought long and hard to gain credibility and traction in organisations for what we do, but we need to be ready to evolve and embrace the changing world around us.  A world where IT no longer needs to have detailed specifications signed off before development start.  We no longer have the need (or the luxury) to do the up-front research that we are used to doing.  We no longer need to sign-off detailed wireframes before handing them over the fence to the developers to implement.  Software today really is soft.  It is more about creativity than engineering (see below).  The serialisation of activities is inefficient and wasteful.  It is time to ask how do we focus upon doing what is needed and when, working in parallel and infecting the whole system with user-centric thinking rather than siloing it into the upfront design.  This after all is what systems ergonomics is about; a forerunner to UCD that we know today, thinking about the macro (a broad system view of design, examining organizational environments, culture, history, and work goals) as well as the micro (fitting the task to the human).

But I am digressing from what I wanted to blog about, the Agile UX retreat.  Some key takeaways for me included Anders Ramsey’s analogies to the restaurant and the theatre.

Thinking analogies

Think of a restaurant.  We have the kitchen, the back room world that is focussed upon delivering consistency of servings.  Everything in the kitchen is utilitarian, serving the purpose to meet this goal.  At the front of the restaurant we have the dining room where the dishes (of consistent(ly good) quality) made in the kitchen are served.  The dining room is all about the ambiance.  Quality here is far more subjective, but a successful restauranteur will be as passionate about the dining room as she is about the food that is cooked in the kitchen.  This is the way that software is all too often built, with the kitchen and dining room being separate entities, however the way they are organised, paid for and owned, there’s little communication between the two.  To quote another Ramsey, Gordon, it is a Kitchen Nightmare.

Anders’ second analogy to consider was the Theatre.  An overly simplistic representation is that the director starts with a script.  From the script he iterates the production.  The producer’s role is to provide the director with what he needs to make the production successful.  Just be ready for the premiere which is on a fixed date.  In the lead up to the premiere the director assembles the cast, the crew and they rehearse.  They’ve got a strawman plan to work from – MacBeth, but how they implement it will evolve according to the stage, actors and artistic direction the director wants to take.  The producer does not care how or when they rehearse, she is only concerned with the success of the end goal.  As they rehearse they increase the fidelity of their performance until they premiere (go live).  but even then they are not done.  They are happy to accommodate changes to the performance, and if something different happens that clearly delights the audience they will happliy incorporate that into future performances (releases).  Sure, the audience is seeing a performance of Shakespere’s MacBeth, but it is a unique performance that has taken the initial plan and evolved as it has been created.

And so should we approach software development.  Not as an exercise in engineering, where our raw materials are fixed and highly stable, but as a creative artform, where our iterations are rehearsals for the premier and ongoing performances.

Thinking tensions

I’m sure there are more, but some examples of tensions that emerge when we try to work together:

AUX promotes rapid open communication and sharing but designers fear sharing.  (They worry early designs will be seized upon before they are ready)
AUX promotes visualisation and use of walls but corporate policies prevent this
AUX promptes doing just enough, just in time but a legacy of deliverable expectations gets in the way (research is rarely bought by the developers who will ultimately consume it).

Thinking people

At the end of the day, success comes down to people.  Agile zealots have done Agile no favours when they bang on about business value and see anything other than code as waste.  Good product design needs vision, it needs research to ensure you are building the right thing for the right people.  No one has the right to tell a UXer that testing ideas or building a prototype or undertaking research is waste if it is right for their context.  But it doesn’t need to take the time it does today.  The UX community needs to get out and spend time with the development community and understand how software is built today.  UXers need to start seeing developers as partners rather than consumers of what they do.  What if we aligned our teams around the products we build rather than the functional silos that the roles describe?  Bringing agile and UX together is more fundamental than arguing about the process (one iteration in front, washing machine cycles etc), it is about fundamentally changing the way we build software; see it as a team activity that works collaboratively rather than a factory production line with process gates and separation of responsibility.

More on the #auxretreat twitter feed

Usability and the $1 trillion mistake

Is this a case of fat fingers, a usability flaw or poor design that enabled a Citigroup trader to have placed an order to sell $16 billion, instead of $16 million? P&G shares plunged by 23% because of this individual erroneous trade. What followed was the algorithms kicked in and automated trading saw the Dow loose a tenth of its value in less than half an hour. (And Accenture dropped to 4 cents down from $42!)

Before we go blaming human error, questions should be asked why that error occurred. How can someone make such a simple mistake so easily? Was it a case of entering two many 0s? (Don’t stop to look or think, answer the question as soon as you’ve read it – how many zeros are there in this number? 160000000. Same thing again, how many zeros are in this number 12,000,000. That’s a bit easier isn’t it. Only an ‘N’ separates the B from the M on a qwerty keyboard, in a hurry, easily mistaken?)

I’d start by looking at human factors and experience design, and question why (assumption here) the IT team who implemented the system didn’t have before a UX designer on the team to think about the human factors. Could this be the most costly example of poor design?

Article: Big drop, was it all a mistake?

Hey usability dude, join the devs (Usability rant part 3)

The last couple of posts here and here have been critical of boutique usability companies that offer user testing services. In the final part of my rant, here’s an alternative approach.  Seed into the development teams someone with a usability background (probably with a psychology degree), someone who can create as well as critique.  Align them to Marketing (or ‘the business’) but physically locate them in IT.  To be successful they must be a passionate advocate for the business vision (hence business alignment), but unless they have a daily relationship with the developers (on hand to support immediate design decisions) they will fail.  (The only way to do this is for them to be part of the development team, same building, same floor, same location).

It is better than they Eschew the formal usability testing process for more informal guerilla usability testing.  In an agile project treat this as akin to the showcase.  Testing sketches and wireframes prior to the development iteration, and putting working software when it has sufficient UI coherence in front of users and well as business stakeholders, with the developers as observers in the process.  One person can cover multiple workstreams.  This is a far better use of a limited budget than commissioning ad-hoc reports too late in the day.

Usability reports (usability rant part 2)

Following on from yesterdays post about the usability process, today I’ll focus on the deliverables, the usability report and my contention that they are rarely grounded in any understanding of the project reality. Here are a couple of examples of usability findings from a (well respected) usability company’s report:

Finding: It was said that the ability to filter [the search results] would be important.
Recommendation: Add check boxes so the customer can choose between [result types]

“Add check boxes”.

That’s easy to say, three words “Add. Check. Boxes”. But what if the particular search engine the team are using does not allow such functionality?  Or such functionality will take significant effort to build.

Finding: When probed about the use of breadcrumbs on the site, 2 participants were confused by the structure that was displayed.
Recommendation: Consider using chevrons [for the breadcrumb] to better covey to the customer that these words show the journey they have been on [rather than '/']

Let’s leave aside the basis of this recommendation; two participants commenting that they weren’t sure about the use of the ‘/’ (this sounds more like it is reinforcing the authors prejudice against the use of / in a breadcrumb and their preference for the ‘>’ symbol).  And let’s also leave aside the fact that it has taken three weeks to let the development team to know that.

It is presented on a powerpoint slide with a screen shot of the breadcrumb and a mockup of a preferred solution, e.g. “Home > UK > South East > News” (Rather than Home / UK / South East / News”).  I’d estimate this took twenty minutes elapsed time to produce this slide. It will take a further ten minutes to discuss when the page is presented to the product owner. And the product owner will spend ten minutes explaining it to the IT project manager who will take ten minutes to explain it to the developers to make the change…

Save your money

Usability testing is not a science. Investing in one or two formal usability tests is almost certainly money badly spent. The Cue reports give a good insight into this.  For example, they asked seventeen experienced professional teams to independently evaluate the usability of the website for the Hotel Pennsylvania in New York.

The teams reported 340 different usability issues. Only nine of these issues were reported by more than half of the teams, while 205 issues (60%) were uniquely reported, that is, no two teams reported the same issue. Sixty-one of the 205 uniquely reported issues were classified as serious or critical problems.

They go on to state…

The study also shows that there was no practical difference between the results obtained from usability testing and expert reviews for the issues identified.

This suggests getting a UI expert into the project team is probably money better spent than employing the usability company (and supports my assertion that usability testing is often just validating the work of a professional).  And when you do get a usability company to report back, as I’ve discussed above, don’t hold your breath for the quality or usefulness of the results:

They found that only 17% of comments in usability reports contained recommendations that were both useful and usable, and many recommendations were not usable at all [source]

So if the recommendations you get from one company are likely to be different to the recommendations from another; if the report is going to be full of recommendations that are impractical and not implementable; if an expert can pick up usability problems that usability testing can, why bother with the usability company testing at the back end of the project?  Indeed, why bother with the usability company at all?  Get an interaction designer on the project from the outset (call them an information architect, user centred designer ,UX person if you want), get them testing ideas and interfaces informally and regularly throughout, and truly embed usability into the project itself, not as an add-on process and report.

Usability rant part 3>

Critiquing the critics (usability rant part 1)

Michael Winner may be a good food critic, but if you were looking for someone to cook you the finest meal for your budget, I doubt he would be your first choice.Same with film critics, they may be able to write an insightful and critical review, but would you want them directing a film for your budget? Would you want Jakob Neilsen, who is essentially a usability critic, to design your website? I mean, take a look at his site!

When you are building a product, you get a usability company in because you know that usability is a good thing that you want to have. If usability companies are the critics, what are you expecting?

The first usability test I ran was in 1991. I’ve set up usability labs, I’ve observed hundreds of people interacting with technology and products. My passion has always to do things at speed, turn around results ASAP and engage all stakeholders in the process.  But I’ll talk about that in a later post.  For now I’ll draw on experience of working with organisations that have commissioned usability companies to review their products.  I’ll breakdown the process I have often observed from usability testing vendors, considering both the elapsed time and the actual ‘value added time’ taken.

Day one

The client (usually the business) engage the usability company to audit the usability of the product that is being developed. The consultants will come in and understand the user tasks, roles and goals; the target audience will be identified for recruitment. ‘Value added time’ = 1 hour.

Day two

The team go away and produce a test plan and a recruitment brief for a research agency to find participants. They promise to get it back to the client in a couple of days. They contact their preferred agency who set about recruiting people (let’s assume this is a simple brief for a retail website targeted at young mothers).  Produce test plan (value added time = 3 hours). Send to client for review.

Day three

Client return test plan with a few comments. Update test plan. Value added time = 30 minutes.

Days six-ten

Twelve usability sessions, each an hour long, they do three a day, that is four days of testing. Value added time = 12 hours

Days eleven – thirteen

The team spend three days analysising and synthesising the results, pull supporting video clips and produce a detailed report. Value added time – 15 hours

Day fourteen

The client sees the report for the first time. (Value added time = 2 hours). Interesting results. (IT representation were not invited, they did not commission the report, the product owner wants to see the output first before sharing it with IT).

Day sixteen

The product owner informs the dev team of the changes that need to be made in the light of the usability report. Project manager sucks air through his teeth and says “you’ll need to raise a change request for those items… ha! quick wins they say? hardly… Hmmm, OK, change the labels in the field, we should be able to do that…”

Value added Vs. Elapsed time

The usability company has delivered and their engagement is complete.  From the start of the process to the recommendations hitting the developers who must ultimately action these, for this not-too-fictitious scenario sixteen days have passed, of which only four were spent on value-added tasks, actually doing stuff.

Day n

The product goes live. The usability company are aghast that so many of the changes they reccomended have not been implemented. They place the blame fairly and sqaurely at the door of the developers and reinforce their belief that IT just doesn’t listen, or worse, care about usability. The critics have critcised from their armchair, like the pigs and chickens they are the chickens, participated not committed.

Usability rant part 2>

Follow fast

I’ll pick on a random industry. Let’s say you are an airline. Part of your digital strategy includes a refresh to your website (maybe you were inspired by this presentation I did a while ago on digital for airlines!). You’ve built a business case and secured funding for the project.  First things first, you get a design agency in and set them to work.

Some sort of competitor analysis is performed that proabably includes features and functions that “we like”, (for example ‘the tactile sliders in kayak.com. We like!  And an iPhone-like coverflow, got to have one of those…)

The information architect takes these ideas away and starts building wireframes and the creative team produce designs that bring these wires to life.  The team come up with lots of new, innovative ideas.  This is after all a ‘refresh’, and ‘innovation’ was probably one of the words in the brief.  The creative is fresh and ‘of the moment’, the IA has developed some new interaction models that are unique and compelling.  The business is sold on a new, innovative way of interacting with their customers, something that no one else does and will blow all their competitors away.

I’ve been bouncing ideas around with Luke Barrett (and because he doesn’t blog, I’ll write them down for him) around this approach; specifically the value of innovation against ‘follow fast’.

Luke reminded me of a project we worked on together many years ago. Before we started designing webpages we did usability testing. We did usability testing of the competitors, and of sites that were getting a lot of press as ‘innovative’.  This was at a time that boo.com had just started and the client were talking about how cool an avatar would be on their site, just like boo. We put people in front of boo.com and watched them suffer. Clearly the avatar was an idea good on paper, terrible on execution.  So we killed it.  Not on our site.

We observed what worked and what didn’t on a multitude of sites with real users. Then, like magpies, we took what was good and worked. Nothing particularly innovative, (let other people do that), we took the best of what existed and delivered on that.

So back to our airline. How about a different approach where they start by usability testing their competitors. Ask participants to book tickets on competitor websites. Understand what interaction elements work, what don’t.

Those kayak sliders, cool for geeks (maybe), but how about the target audience that flies and buys online with you?  It may not be cutting edge design, but Does a drop down work better?

The coverflow may be cool on your iPhone, but how successful is it for people seeking a holiday?  A static list has worked for websites till now (and it wasn’t so long ago that horizontal scrolling was the Great Taboo), just because Apple do something that looks cool for a particular purpose on their products, doesn’t mean you have to follow by scrapping your navigation.

There are no prizies for (design) innovation (other than for some award that the design agency may covet). The only metric that counts is conversion rates and the ability of the website to deliver the business case. Leave others to do the crazy innovation stuff, let others go through the dip when they launch new features, make sure you have got the platform and expertise right and be ready to follow fast.

Pillars of a compelling experience

This is a model I often see in organisations when it comes to their web presence.  A product owner comes up with a commercial proposition, marketing work up the content, IT build the functionality and it is goes live.  With this model, no-one actually owns the customer experience.

Worse, this little temple model is repeated across different commercial propositions so you end up with something that is not very joined up.  I’ve blogged about this lack of joined up thinking before.

Now let’s construct a model where the roof of the temple is a compelling customer experience.

What are the ingredients of this new temple model?  It is still going to be founded upon commercial propositions, but they are going to be overlaid by a culture of test and learn.  That is a willingness and ability to experiment; to realise that what you have developed is never final and is always evolving.  It is about taking the learnings of experiments to inform and improve the experience, or to rapidly refine or kill propositions that just do not work.

Then we have the five pillars.  I describe these in a paper I wrote ages back (pdf here, google books here).

Unfortunately these pillars tend to sit within organisational silos; content and personality are ‘owned’ by  marketing, functionality by IT, and operational excellence (that’s all about fulfilling on the customer promise and beyond) is a mixture of IT and operations.  Usability is a ‘funny one’ in that might sit alone, sit in marketing or sit in IT.  But ultimately it is best placed to direct the horizonal filter of Quality Control.  Quality control is not an additional layer of bureaucracy, rather a cultural component that all the pillars feed into.  It is about ensuring consistency and meaningfulness of the experience.  It is about balancing the commercial needs of the product, with the marketing needs of the message and the delivery capability of IT.

What would Sally do? Personas for retail financial services

Personas are ‘pen portraits’ that bring to life users or customers of a system, service or product.  Giving a personality and back story to your customers helps keep your thinking true to their real needs and goals.  Rather than using  ‘user’ or a segment descriptor such as ‘empty nester’, or ‘this is what I would do’, what would Sally do?

Here’s a set of personas for financial service organisations, geared towards the retail / B2C market.  Sally is included (Shes skint).

Who would turn off the wrong engine?

In designing user interfaces there’s a lot we can learn from systems where failure to consider human factors has resulted in terrible consequences.

On 8th January 1989 British Midland Flight 92 crashed whilst attempting an emergency landing. There had been a fire on one of the engines which led to its malfunction. The captain reacted by shutting down the engine.  Only he shut down the wrong engine. With no power, approaching East Midlands airport the captain manged to glide the Boeing 737-400 to avoid Kegworth village but crashed short of the runway.  47 people died.

The investigation into the Kegworth air disaster identified engine malfunction (the engine used in the aircraft was an upgrade of an existing engine and had not been field-tested) as causal factor, however the report concentrated upon the failure of the flight crew to respond accurately to the malfunction.  Human error was the primary cause.

The truth is a little more complicated than that.  Why does a captain with over 13,000 hours flying experience and a first officer with over 3,000 hours experience shut down the wrong engine?

A number of human factors contributed to the disaster including organisational issues (refer to this paper for discussion of the role of managerial failures in disasters) and cognitive overload.  But of equal importance (and indeed buried in the appendices of the Investigation Report appendices) is the issue of design. Around 50% of accidents and incidents in the aircraft and nuclear industries have a root cause in design (source).

Take a look at the cockpit controls (taken from the investigation report). The left image is for the earlier 300 series and the right for the 400 series aircraft on which the captain had only 23 hours experience after a one day training course.

The actual cause of the engine malfunction was a broken turbine, itself the result of metal fatigue caused by excessive vibration (source).  Had the Captain noticed the Vibration Warning display he probably would not have made the wrong decision.

The Vibration Warning display on the new 400 series was in a different place to the 300 series, but more critically it was designed to be significantly smaller “the dial on the vibration meter was no bigger than a 20 pence piece and the LED needle went around the outside of the dial as opposed to the inside of the dial as in the previous 737 series aircraft” (Source: Wikipedia).  Take a look at the arrow on the left hand image, the display dials on the 300 series use mechanical pointers. On the 400 series they were redesigned with short LEDs rotating around the numbers. These, as the investigation report noted “are much less conspicuous than mechanical pointers, acting more as scale markers, and providing less immediate directional information).

The report criticised the layout of the instrumentation and helpfully suggested an improved layout.  The layout was (and as far as I can tell, remains in 737s) split into primary instruments and secondary instruments.  The issue with this layout is that the dials are not spatially aligned with their associated power levers.  If the pilot is focussing upon the primary instrumentation, the secondary instrumentation is in peripheral view.  This layout will lead to attention based around specific instruments rather than engines.

Compare this to an alternative design that the report provides where the dials are aligned to their associated power levers.  The report recognises the design trade-offs here but concludes that to break the left-right mental association with the engine position was probably not the most optimal solution.

This paper describes the issue well:

The 737 involved in the East Midlands crash had flight deck engine information that lead to confusion under mental pressure. Placing the secondary information sets for both engines to the right of the primary set broke the implied rule set by all the other engine information, that the left engine had left hand controls and indicators (and vice versa). If one assumes that the optimum positioning of indicators is the one that requires the least mental processing then a simple symmetry about the aircraft centre line seems appropriate. The actual positions required a mental spatial transposition of one set of dials to the other side… The readability of the indicators had been reduced by the substitution of electro-mechanical readouts with electronic readouts, but which simulated the old design. Possibly the redesign to electronic readouts should have taken the opportunity to use a rather different layout, possibly with linear indicators rather than rotary ones.

OK, so lots of words, but what is the point of this to what I usaully blog about?  The issue is one of design and layout and who’s responsibility is it.  In designing user interfaces UCD is often seen as a luxury, developers believe that they can design a GUI as well as anyone, and stakeholders (especially on internal projects) will question the value that a UCD person can bring to the project.  Does a developer or an engineer by training and instinct stop to ponder the human factor and the human consequences of the GUI layout? What are the consequences of this?  As can be seen from Kegworth, seemingly minor changes to the control layout can have a signficant impact on the safety of a complex system.