Evaluation
Evaluation of a system is an important part of the systems lifecycle, as it allows
designers and implementers to compare the system to the original requirements and
therefore ensure the system is what the user wants.
In terms of interfaces, evaluation will ensure that they meet the requirements of
being usable, that is they are “learnable, retainable, efficient and satisfactory
to use” (Constantine and Lockwood, 1999). The intention of evaluation is to reveal
any problems that may exist in an interface or any difficulty that the users experience.
Evaluating the Forrest Hills site
Evaluation of the Forrest Hills website was done using heuristic evaluation (Nielson
and Molich, 1990; Nielson, 1994), a technique for finding usability problems by
having a set of evaluators examine an interface and check its compliance with certain usability factors or “heuristics”.
Generally, heuristic evaluation with one person will not find all the problems,
but by using more evaluators, a larger number and wider variety of problems will
be discovered. A group of about 4 people is said by Nielson to provide the best
cost:benefit ratio, but the type of evaluators used will also affect the results.
A group of 5 usability experts could be expected to find about 84% of problems,
while an individual evaluator will find only 31% of problems. Because of this, I
used a number of people to evaluate the Forrest Hills site, and these consisted
of Graduates who had previously performed a heuristic evaluation and a farmer who
would be a likely user of the system.
Both groups were given tasks to perform during which they could note any problems
and give feedback on the site. The Graduates noted problems in the form of violated
heuristics while the farmer simply wrote down what he liked/disliked and any problems
he faced.
Tasks
The evaluators were given a number of tasks to complete during which they would
note down any problems or violated heuristics. The full tasks can be seen in the
appendix, but summarised they are:
1. Browsing the site.
2. Adding entitlements using the Admin back-end.
3. Browsing entitlements.
These tasks enabled the evaluator to experience the system from the perspective
of a standard user, and also of an administrative user.
The task-based approach also provides another benefit, mostly to the non-expert
evaluators. Instead of having a non-expert trying to evaluate usability, layout
and heuristics; they were given a task to complete and in the process could note
down any problems. The action of doing this would reveal any usability problems.
