What
Is Usability Testing ?
The
term usability testing is often used rather indiscriminately to refer to any technique
used to evaluate a product or system. Throughout this book we use the term
usability testing to refer to a process that employs people as testing
participants who are representative of the target audience to evaluate the
degree to which a product meets specific usability criteria. This inclusion of
representative users eliminates labeling as usability testing such techniques
as expert evaluations, walk-throughs, and the like that do not require
representative users as part of the process. Usability testing is a research
tool, with its roots in classical experimental methodology. The range of tests
one can conduct is considerable, from true classical experiments with large
sample sizes and complex test designs to very informal qualitative studies with
only a single participant. Each testing approach has different objectives, as
well as different time and resource requirements. The emphasis of this book is
on more informal, less complex tests designed for quick turnaround of results
in industrial product development environments.
Why
Test ? Goals of Testing
From
the point of view of some companies, usability testing is part of a larger effort
to improve the profitability of products. There are many aspects to doing so,
which in the end also benefits users greatly : design decisions are informed by
data gathered from representative users to expose design issues so they can be
remedied, thus minimizing or eliminating frustration for users.
Informing
Design
The
overall goal of usability testing is to inform design by gathering data from which
to identify and rectify usability deficiencies existing in products and their
accompanying support materials prior to release. The intent is to ensure the
creation of products that :
- Are useful to and valued by the target audience
- Are easy to learn
- Help people be effective and efficient at what they want to do
- Are satisfying (and possibly even delightful) to use
Basics
of the Methodology
The
basic methodology for conducting a usability test has its origin in the classical
approach for conducting a controlled experiment. With this formal approach,
often employed to conduct basic research, a specific hypothesis is formulated
and then tested by isolating and manipulating variables under controlled
conditions. Cause-and-effect relationships are then carefully examined,
often
through the use of the appropriate inferential statistical technique(s), and the
hypothesis is either confirmed or rejected. Employing a true experimental design,
these studies require that :
A
hypothesis must be formulated. A hypothesis states what you expect to occur
when testing. For example, ‘‘Help as designed in format A will improve the
speed and error rate of experienced users more than help as designed in format
B.’’ It is essential that the hypothesis be as specific as possible.
Randomly
chosen (using a very systematic method) participants must be assigned to
experimental conditions. One needs to understand the characteristics of the
target population, and from that larger population select a representative
random sample. Random sampling is often difficult, especially when choosing
from a population of existing customers.
Tight
controls must be employed. Experimental controls are crucial or else the
validity of the results can be called into question, regardless of whether
statistical significance is the goal. All participants should have nearly the
identical experience as each other prior to and during the test. In addition,
the amount of interaction with the test moderator must be controlled.
Control
groups must be employed. In order to validate results, a control group must be
employed; its treatment should vary only on the single variable being tested.
The
sample (of users) must be of sufficient size to measure statistically significant
differences between groups. In order to measure differences between groups
statistically, a large enough sample size must be used. Too small a sample can
lead to erroneous conclusions.
The
preceding approach is the basis for conducting classical experiments, and when
conducting basic research, it is the method of choice. However, it is not the
method expounded in this book for the following reasons.
It
is often impossible or inappropriate to use such a methodology to conduct
usability tests in the fast-paced, highly pressurized development environment
in which most readers will find themselves. It is impossible because of the
many organizational constraints, political and otherwise. It is inappropriate
because the purpose of usability testing is not necessarily to formulate and
test specific hypotheses, that is, conduct research, but rather to make
informed decisions about design to improve products.
The
amount of prerequisite knowledge of experimental method and statistics required
in order to perform these kinds of studies properly is considerable and better
left to an experienced usability or human factors specialist. Should one
attempt to conduct this type of tight research without the appropriate
background and training, the results can often be very misleading, and lead to
a worse situation than if no research had been conducted. In the environment in
which testing most often takes place, it is often very difficult to apply the principle
of randomly assigning participants because one often has little control over
this factor. This is especially true as it concerns the use of existing
customers as participants.
Still
another reason for a less formal approach concerns sample size. To achieve
generalizable results for a given target population, one’s sample size is
dependent on knowledge of certain information about that population, which is
often lacking (and sometimes the precise reason for the test). Lacking such
information, one may need to test 10 to 12 participants per condition to be on
the safe side, a factor that might require one to test 40 or more participants
to ensure statistically significant results.
Last,
and probably most important, the classical methodology is designed to obtain
quantitative proof of research hypotheses that one design is better than
another, for example. It is not designed to obtain qualitative information on
how to fix problems and redesign products. We assume that most readers will be
more concerned with the latter than the former.
The
approach we advocate is a more informal, iterative approach to testing, albeit
with experimental rigor at its core. As the reader will see in later chapters of
this book, experimental rigor is essential for any study that one conducts. Much
can be achieved by conducting a series of quick, pointed studies, beginning
early in the development cycle. It is the intent of this book to present the
basics of conducting this type of less formal, yet well-designed test that will
identify the specific usability deficiencies of a product, their cause, and the
means to overcome them. The basics of this approach are described in the sections
that follow.
Basic
Elements of Usability Testing
- Development of research questions or test objectives rather than hypotheses.
- Use of a representative sample of end users which may or may not be randomly chosen.
- Representation of the actual work environment.
- Observation of end users who either use or review a representation of the product.
- Controlled and sometimes extensive interviewing and probing of the participants by the test moderator.
- Collection of quantitative and qualitative performance and preference measures.
- Recommendation of improvements to the design of the product.
Source
:
Rubin,
Jeffrey., Chisnell, Dana., Handbook of
Usability Testing, 2nd Edition, Wiley Publishing Inc., 2008
Good Article.
ReplyDeletePriti G,
Work at Software Testing Company