What do you really need?
A couple of clients I’ve worked with recently have been consolidating their application space, decommissioning old technology and replacing it with a new single application with a core user interface. I’ve written about this before but it is worth revisiting. All too often the starting point is the functionality and the features of the existing applications. The client states we must at the very minimum maintain feature parity, and where the business needs, enhance functionality. Starting with the existing applications and as-is processes is a good start, but never where the focus should lie. The focus should be around the business intent; what are the business goals the system is helping the user achieve? Spending time with the end users of the system is enlightening. It is a crude picture, but this shows what the truth often looks like.
There are three systems that have been developed over the years, commissioned by different business stakeholders with different budgets and different delivery teams. Of each of these systems only a fraction is ever used. (This is especially true of vendor products that have requirements rooted in the market rather than the specific needs of that organization. Think about how many of the features in Microsoft Word you use). If there is significant functional redundancy in the applications, there is also duplicate functionality that results from the different development legacies. It is not unusual in such a landscape for the user to enter the same data into each system. Not something you would wish to replicate in a new world.
In building a new application, the place to start looking for requirements is not so much the as-is processes or existing applications. The place to start is the business intent and what the business actually wants the system to do. More importantly, that means starting with an open mind and challenging notions of process complexity. Is the process complex because it really is, or because the current systems make it that way? In my experience, more often than not, it is because of the former. Spend time with end users, see what they do today. By asking seemingly stupid questions, taking a starting position that the process is simple and challenging ‘why not?’ can yield valuable insights. By all means consider the as-is world, but don’t let it cloud your judgment in designing ‘to-be’.
This struck a chord with me. I’ve worked as a computer auditor, and I’m very interested in usability issues. Something that has troubled me when I’ve reviewed applications is the way existing functionality has been carried forward without serious scrutiny. I suppose my particular concern has been the corollary of yours. I’m interested in how the functionality can be exploited, and what the latent, unintended functionality might be. Ideally I’d like a form of usability testing involving bad guy personas trying to exploit the application, but that;’s getting onto another subject.
I recently wrote this in an article in Testing Experience magazine discussing flaws in an insurance application that had been exploited by fraudulent users.
“The control weaknesses related to an abuse of the authorization process, and a failure of the application to deal appropriately with third party claims payments, which were extremely vulnerable to fraud. These weaknesses would have been present in the original manual process, but the users and developers had not taken the opportunities that a new computer application had offered to introduce more sophisticated controls.
No-one had been negligent or even careless in the design of the application and the surrounding procedures. The trouble was that the requirements had focused on the positive functions of the application, and on replicating the functionality of the previous application, which in turn had been based on the original manual process. There had not been sufficient analysis of how the application could be exploited.”
I like the article!
‘Bad guy personas’… Are you familiar with Extreme Characters? There’s a paper out there where the authors introduced a drug dealer and the pope and new personas in the product development processes to see how their needs could be addressed
Thanks. Did you look up my article? The question of usability was fairly peripheral, but I think the intersection between security and usability testing is an interesting topic that doesn’t seem to have received as much attention as it deserves. Personas and usability techniques should be among the tools that functional and security testers call on when they’re considering how applications can be abused. I wasn’t familiar with Extreme Characters, but I tracked down the paper I think you meant.
http://homepage.mac.com/j.p.djajadiningrat/publications/2000DjajDISInte.pdf. It was very interesting.
Of course using these Extreme Characters would be rather different from what I was thinking of. That paper’s about product design. However, the interesting point is that such characters are needed because of the limitations of stock, stereotypical personas that have just bland, “good” characteristics and who will behave in neat, predictable ways. And if they’re going to be predictable, why bother with personas at all? Surely you have to think a bit more deeply than that about your users and what motivates them.
A big problem for testers is that their rushed functional testing, which looks at what the application is supposed to be doing, is hopelessly superficial compared to the deep understanding that real users will acquire over time of what the application actually can do. It’s impossible for testers to acquire that deep knowledge in a limited time, but if they use well-judged personas, unleash their imagination and start looking at the application in a different and more cynical way then they can expose serious weaknesses.
Unfortunately the documentation heavy approach that software testers have traditionally placed too much emphasis on proving the application can do what it’s meant to and haven’t allowed testers time to use their imagination to discover whether it can do thing it really shouldn’t.