As testers, we are often encouraged to act as user advocates, to take the user’s view and make sure that the product we are currently testing is palatable to the users at large. However, everyone has different tastes and preferences, and something that a tester might find to be fine would be considered by the mass populace to be extremely unfriendly. Confronted by this reality, many of us seek an objective way to evaluate the ease of use for the system. The process of this evaluation is usually called “usability testing”. When given a large budget and an ample amount of time, the best way to handle it is to set up the system in a controlled environment, gather a sizable sample of the target population, and set them several tasks to do on the system. Then, as the populace tries to accomplish the tasks you set for them, record their reactions, measure the time and effort it takes them to accomplish tasks, and analyze the written and verbal feedback they give you. The need for usability testing ranges from a bonus feature all the way to a government mandate, and in many cases is able to make the difference between a successful product and a failed one. As such, it is far from always that the most thorough methods are desired, so sometimes usability has to be handled by the testers, without any actual user input. In this case, we have to try to analyze a subjective measure objectively. To do so, we ask, “What is usability?” Classically, usability is defined through five parameters:
Sounds great in theory, but what do those fine terms actually stand for? Let us examine each of them in turn.
Learnability is the measure of how hard it is to understand how the system works, or the intuitiveness of the design. But intuitiveness is still subjective, right? After all, intuition goes off experience. Ah, but that is not as big of an issue as it might seem. Thus the question is: how relevant is the experience that should be relevant? The answer to that question lies in two areas: external experience and internal experience.
Internal experience is the experience with the system in question, i.e. the system should behave consistently throughout itself, and not change the principles of control from one area to another.
Example:
Suppose we are working with a 3D modeling system. Throughout the model-editing module, the positioning module, and the compositing module, holding down the right mouse button would enter the camera control mode. In the texturing module, however, holding down the right mouse button would give the user direct control over the position of the texture relative to the plane it is currently on. As the user has experienced the other three modules first, this behavior is unexpected and confusing.
External experience is a bit trickier, though. External experience consists of experience with the environment in which the system is set and of experience with similar systems elsewhere. The environment usually refers to the operating system, and to the hardware involved. Systems elsewhere, on the other hand, are fairly straightforward. If the system in question has a significant amount of functionality that is duplicated in other systems, then it should discharge its duties in that area according to the standard set by those other systems. This is always true from the point of view of learnability, even if it is not always true as far as usability as a whole goes.
Examples:
(OS: If the system is a windows-based system, and has a tree structure, if the user can drag a folder, he will expect that dragging a folder into another folder will move it there and not create a symbolic link, or delete it. If the system is Linux-based, the user will expect to be able to run it with the “man” argument and get a manual in return. (Hardware) Suppose the system is a flight simulator. If it can recognize a joystick and work with it, yet can also work with just the keyboard and mouse – great! However, if it requires a joystick, and is unplayable otherwise, it is now requiring a new player to not only learn the mechanics of the game itself, but also how to work with a joystick, adding to the learning curve. This is bad. (Hardware 2) Let us stick with the flight simulator for a minute. Let us suppose that the controls are not re-bindable (a sad state of things that will be discussed later). If the tilting of the joystick forward and backwards controlled speed, and tilting it left-right would pitch your aircraft up and down, this would be confusing, as the relationship of the controls and the aircraft would not be obvious. (Similar systems) Suppose the system is a text processor. Now it is firmly established that in a text processor, hitting the “End” key will send you to the end of the line. If the system in question used that key for “save and exit” instead, it would be a bad thing, as users would hit end out of habit, and have their workflow disrupted in a rather savage way.
This parameter is closely related to learnability. It is the measure of how easy the system is to remember, or alternatively, how long it takes to re-learn once proficiency has been lost. Human memory is fueled by associations, and associations are based on patterns. Thus the UI of a system should strive to create associations and patterns whenever possible. There are four indicators to keep an eye out for: having no useless buttons, grouping controls by function, sequencing those groups in ways which sync with the workflow of the system, making sure any icon present clearly illustrates the function of the button, or alternatively that the caption is clear (more on this in information flow).
The first of the four is also the simplest to explain: buttons that do not do anything are basically taking up screen real-estate, but they also disrupt any patterns around them as they are effectively a blank spot. Also, users tend to forget they exist, and then, when they find it in the middle of a semi-remembered pattern they conclude that their pattern was from elsewhere, which detracts from the process of remembrance.
Putting similar items together makes sense as then the user can remember the general purpose of a block of controls, as opposed to having to remember what each button does. Once he gets to the right place, sorting out which particular flavor of that generic purpose he wants should be faster than hunting for a specific button all over the UI.
If applicable, putting functions in the order they are going to be used also makes sense, as it is intuitive to progress onward as you progress over a task instead of jumping all over the place. However, since some systems do not have a predictable flow, this is not always applicable.
The last thing to keep an eye out for is the labeling of the control elements. Remembering which particular function one wishes to use is hard enough, having to also remember what the button looks like can be straight-up painful. Moreover, evocative labeling helps remind the user of the options he has, and as such helps with the memorization.
Example: Imagine for a moment that a system like Photoshop simply numbered brushes, tools and filters (instead of mosaic, lens glare, and sharpen, you had “filter 1”, “filter 2”, and “filter 3.”) Remembering which filter is what would be positively painful.
When users interact with a system, they need feedback from the system to determine the direction of their actions to follow. This feedback comes in three ways: passively displayed information, popups, and information you need to seek out. Passive information should contain enough info to work with, yet not so much as to clutter the screen. Optimally, the passively-displayed information is detailed enough that the user needs more information to accomplish a specific task only a couple times over a session. Also note that if there are certain tools in the system that require information for optimal usage, every effort should be made to display that information when the tool is activated, and hidden afterwards. One thing that is very undesirable with passive information display is repetition, as that takes up valuable screen space without actually adding any information.
Active information comes in two flavors: the notification and the question. The notification is used to inform the user that a certain event took place, and should only interfere with the user’s work if the information it carries is absolutely critical. Notification for the failure of a process that runs in the background of the program is also acceptable, but notification of completion of such a process is not an appropriate reason to interrupt the user’s train of thought. The proper way to handle messages that are just “FYI” is exemplified by the Windows update dialog. It shows up far from the center of the screen, tells you that there are updates that you might want to pay attention to when you got the time, and, if you ignore it for some five seconds, it goes away. Note that it is fairly unobtrusive, yet if the user mouses over it, it will stay, and then the user can read it at leisure.
There is no way to get around the fact that the question-type popup will interrupt the user. Because of this, the question should always follow two simple rules: any question should only be asked once, and it should always conform to the convention that hitting the escape key aborts whatever procedure spawned the question, and hitting enter should try to continue it. This applies even to the select file dialog that pops up when a user tries to save a file. The reason that a question should always be asked once is that if this rule were to be broken, the user tends to lose patience with multiple questions and just hits either enter or escape whenever something comes up, without reading. As the whole point of posing a question to the user is the get them to make a decision, this result is about as harmful to the original purpose as not.
There is another rule that applies to both kinds of active information: it should be as short as possible. The user usually does not feel like reading a wall of text that pops up in the middle of whatever he was doing. Thus, if your message is not conveyed within the first twelve or so words, there is a very good chance that the user will simply hit something to make it go away instead of actually reading. But it is also bad to ask the user a question to answer which he would need additional information that is only accessible once the question is answered. The solution to this dilemma is the “more information” button.
Example: Suppose, that there is an antivirus program that needs to ask a vital question: “The file ‘very_important_document.doc’ has been infected by a strand of the virus ‘evil_puppies_2000’. This virus can spread quickly and will randomly send pictures of puppies to your printer. What would you like to do with the file? Ignore / Delete / Quarantine”. In this case suppose the maker of the antivirus considers all information essential, but the question comes up much too long, so what should be done? The answer is simple: make the statement short and to the point, with a “more info” button that would display the sordid details. The result would be “’evil_puppies_2000’ infected ‘very_important_document.doc’. More info / Ignore / Delete / Quarantine”, with the details of what the “evil_puppies_2000” actually does in the more info section that expands. This way the user gets the message quickly, and if he already knows about “evil_puppies_2000”, he will not have to waste time reading the details.
The last type of information we need to discuss is information that needs to be sought out. This information follows a slightly different set of rules, since it cannot interrupt the user’s train of thought (they already broke when they went looking for the information). These rules are threefold:
As long as these three rules are followed, there can be no such thing as overkill, and the more information presented the better.
The point of any application is to get something done. The user never will start an application simply to stare at its attractive UI or to admire the macro system but instead do so in order to accomplish some task. Because of this, the more effort the user has to put in to operate the application, the less they will like it. Due to the differing nature of systems, it is impossible to create one rule set for efficiency, but there are a few general guidelines.
The last area of usability is pleasantness. Most of this area is intuitive and can be summed up easily: the system should not annoy or tire the user. The first thing to watch out for is the color palette of the system. It should neither strain the eyes, nor make the different elements of the system hard to tell from each other. The next thing is that the fonts used in the system are easy to read. This includes both the font selection and the font size. Also look out for icons that are hard to make out, and contain clashing colors. Clashing colors on the icons are particularly bad as they will draw the user’s attention to that spot in the UI, instead of whatever they were actually trying to focus on.
The flip side of this coin is to ensure that nothing that is seen is straightaway offensive. Abusive language is of course self-evident, but beware messages that denigrate the user in subtle ways, or messages that offend certain groups of individuals. Ugly pictures and icons are also on this list. Lastly remember: you are human and if, over the course of testing, something repeatedly annoys you, it will most likely annoy others; therefore you should raise it as a concern.
In conclusion, the essence of usability is adherence to the five guidelines: learnability, efficiency, memorability, information flow, and pleasantness. That adherence should be the focus when testing for usability. You will note that some of these requirements conflict with each other, but there is no real way to escape that. Use your best judgment and think of the priorities of the target audience when making those calls. Finally remember: when you are testing for usability you are not developing. Report everything that is wrong, and let the development team sort out which usability bugs they wish to attend, and which ones aren’t worth the effort. With all that being said, on the next page is a super-condensed checklist of things to look for that you can use in your testing. I hope it helps you!
