Jakob Nielsen's Alertbox for June 1, 1997:

The Fallacy of Atypical Web Examples

Beware of emulating examples featured in the press. The sites that get the most publicity are often not representative of the majority of sites: in fact, they get covered because they are special. The one true "killer app" of the Web is diversity: the fact that sites are very different and can target very specific user needs. A few sites have exceptional circumstances and can serve customers in spectacular ways, but your site will most likely have to use another approach that is better suited for your specific users and the tasks they want to perform.

Atypical Example Common Fallacy
in generalizing
from the example More Valid Analysis
Yahoo! makes a profit from Web advertising Advertising is a viable revenue model for the majority of websites Only the top 0.002% of websites in terms of traffic will make enough money from pure "eyeball" impressions. The distribution of traffic is extremely skewed so 49,999 out of every 50,000 sites will need another business model. If you are very big, then ads are fine, but most sites will be medium-sized or small.
Wall Street Journal seems to have some success in getting Web users to pay for subscriptions If not advertising, then subscriptions are surely a way to make money on the Web. The Journal has unique brand recognition and the type of content that people can safely predict that they will be needing regularly enough times to make a subscription worthwhile. Most sites have content that people can live without, and the nature of the Web encourages users to move around freely; only possible if the payment model is microtransactions rather than subscriptions.
Disney is the most successful media company Storytelling is essential for Web design Storytelling is indeed a powerful way to connect with users. However, most storytelling techniques rely on linear media such as books, film, or television where the user's experience is fully under the control of the author/director. The Web is non-linear, and users control their own movements, weaving in and out of individual sites as they go. For most sites, it is more important to support user navigation and freedom of discovery than to direct users along a given story line.
The WELL nurtured a community of strongly connected and frequently interacting users Community is the key to generating user loyalty and interest Anybody who was online in 1985 when The WELL started was clearly a member of a small elite on the frontier of computing. Geographic location (San Francisco/Silicon Valley) made the membership even more elitist and leading-edge. Sure, hanging out with the 1% smartest people in the world can be fun, but the average Web site will have average users. Also, once you get a broader and more diverse user population, you get hit by the extreme participation inequality that characterizes all aspects of the Internet: a few people do all the talking and they are rarely the most interesting people (unless everybody in the group is drawn from the world's top-1%).
In the battle between Amazon.com and Barnes & Noble, it turns out that Company X wins If "X"=Amazon then clearly brick-based companies are too slow-footed to catch companies that move at Internet speed
If "X"=B&N then clearly having an established brand and plenty of marketing muscle must be what counts
Many, many details will combine to determine which of the two bookstores win (or a third bookseller may emerge as the winner). Reporters play up the "one big thing" that makes their story interesting, but in real use, the story is in a multiplicity of details. For example: size of discounts, speed of server, frequency of crashes, quality of UI, number of third-party sites linking to each vendor. If Amazon wins, there is one additional reason the victory will atypical: they received huge amounts of press coverage for being the first major success in Internet commerce. Any subsequent success stories will get a fraction of the free publicity and thus less of a boost to their brand recognition.

Atypical Example	Common Fallacy in generalizing from the example	More Valid Analysis
Yahoo! makes a profit from Web advertising	Advertising is a viable revenue model for the majority of websites	Only the top 0.002% of websites in terms of traffic will make enough money from pure "eyeball" impressions. The distribution of traffic is extremely skewed so 49,999 out of every 50,000 sites will need another business model. If you are very big, then ads are fine, but most sites will be medium-sized or small.
Wall Street Journal seems to have some success in getting Web users to pay for subscriptions	If not advertising, then subscriptions are surely a way to make money on the Web.	The Journal has unique brand recognition and the type of content that people can safely predict that they will be needing regularly enough times to make a subscription worthwhile. Most sites have content that people can live without, and the nature of the Web encourages users to move around freely; only possible if the payment model is microtransactions rather than subscriptions.
Disney is the most successful media company	Storytelling is essential for Web design	Storytelling is indeed a powerful way to connect with users. However, most storytelling techniques rely on linear media such as books, film, or television where the user's experience is fully under the control of the author/director. The Web is non-linear, and users control their own movements, weaving in and out of individual sites as they go. For most sites, it is more important to support user navigation and freedom of discovery than to direct users along a given story line.
The WELL nurtured a community of strongly connected and frequently interacting users	Community is the key to generating user loyalty and interest	Anybody who was online in 1985 when The WELL started was clearly a member of a small elite on the frontier of computing. Geographic location (San Francisco/Silicon Valley) made the membership even more elitist and leading-edge. Sure, hanging out with the 1% smartest people in the world can be fun, but the average Web site will have average users. Also, once you get a broader and more diverse user population, you get hit by the extreme participation inequality that characterizes all aspects of the Internet: a few people do all the talking and they are rarely the most interesting people (unless everybody in the group is drawn from the world's top-1%).
In the battle between Amazon.com and Barnes & Noble, it turns out that Company X wins	If "X"=Amazon then clearly brick-based companies are too slow-footed to catch companies that move at Internet speed If "X"=B&N then clearly having an established brand and plenty of marketing muscle must be what counts	Many, many details will combine to determine which of the two bookstores win (or a third bookseller may emerge as the winner). Reporters play up the "one big thing" that makes their story interesting, but in real use, the story is in a multiplicity of details. For example: size of discounts, speed of server, frequency of crashes, quality of UI, number of third-party sites linking to each vendor. If Amazon wins, there is one additional reason the victory will atypical: they received huge amounts of press coverage for being the first major success in Internet commerce. Any subsequent success stories will get a fraction of the free publicity and thus less of a boost to their brand recognition.

I am not saying that we should ignore interesting examples. It is hard to discuss something as overreaching as the Internet in purely abstract and generalized terms. People need concreteness to internalize the meaning of trends and concepts. My point is simply to warn against basing your Web strategy on atypical examples that are not representative. It should be a warning signal if you hear the same example used over and over again by different analysts: quite possibly the example is striking, interesting, or even sensational, but it may well be an outlier that does not predict the rest of the field.

June 15: Top ten mistakes of Web management

See also: List of other Alertbox columns