Alertbox
May 1998 Testing page templates
|
useit.com
Alertbox
May 1998 Testing page templates
|
Jakob Nielsen's Alertbox for May 17, 1998:
Before you unleash a page template on a large number of content creators and direct them to design thousands of pages to its specifications, it would be nice to have more data on the usability of the template itself. By "usability of template" I mean whether a page gets more or less usable because of the layout specified by the template. It would also be interesting to study whether it is easy for page authors to design with the template, but that is a different matter.
Thomas S. Tullis (tom.tullis@fmr.com) from Fidelity Investments recently conducted a study to assess the usability of five alternative template designs for their intranets. To test whether the layout itself helped users find the page elements, all text on the page was "greeked" - that is, all words were replaced with unintelligible nonsense (sometimes called mumble text). When they can't read the text, users have to rely on the inherent communicative aspects of the layout to perform the test task. If a layout performs well when users can't understand any of the content, then there is hope that the template will survive substantial abuse from authors who fill it with content of varying quality.
Each test user was given printouts of the greeked pages (in random sequence) and asked to draw labeled blocks around the parts of a page that corresponded to each of nine standard elements. Blocks could not overlap and could not be nested. If a user thought that an element was missing from a page, it was to be marked as "not there." The study included five templates of which two are shown here (reproduced with permission).
Page Template 3
Each page template had the following nine standard elements which you should try to identify in the figures:
The correct solutions are shown in a sidebar, but please try to solve the problem yourself before peeking. You only get one chance to approach a problem as a novice user.
As shown in the table, users performed best with Template 3, where they were able to correctly identify 67% of the standard page elements. Users were also asked to subjectively rate how well they liked the templates on a -3 to +3 scale. As shown in the table, Template 1 was the best liked, even though users performed significantly worse with this template. This outcome demonstrates the danger of simply showing designs to people and asking them which one they like the best: users may not know what's good for them.
| Correctly Identified Page Elements | Subjective Appeal | |
|---|---|---|
| Template 1 | 52% | +1.3 |
| Template 3 | 67% | +0.9 |
| Final Design | 72% | +2.1 |
Since different designs scored best on the two usability metrics (performance and satisfaction), Fidelity decided to combine the best parts of the designs into a new, final design. When this final design was tested with the same methodology, it scored better than any of the original designs, proving that it was indeed an improvement.
The greeking technique is a very fast way of gathering usability data about proposed page templates before a large number of pages have been constructed. Since users can't actually use nonsense pages, there is no need to implement anything: instead, each design can be printed out and marked up on paper.
List of other Alertbox columns