Why you shouldn't cry bias merely because the reviewer works for a mainstream outlet.
|Independent reviewers, as a sort of catch phrase, often bash mainstream outlets. The outlets, they assert, merely shovel out biased reviews to keep their corporate masters on good terms with sponsors. In contrast, the indies stand free from the corrupting influence advertisers impose. While their independence is certainly admirable, they go too far when they assure their audiences they offer unbiased reviews. Simply put, every review is biased. Anyone who looks at, say, a video game, comes to the table with a preconceived notion of what is good or bad, developed by decades of experiences that shaped the person's general understanding of quality. We shouldn't fight this. Our personal biases make us unique. A hypothetical world of unbiased reviews would theoretically contain a "correct" review—any differing opinion would therefore be wrong. But here, every reviewer offers a unique perspective. Even for works that are universally panned, reviewers differ on exactly why the work is bad, or whether it's "so bad it's good." On a more basic level, reviewers will value and emphasize a work's aspects differently. For example, a video game with an excellent soundtrack but bland graphics can lead to wildly contrasting opinions. If you're musically inclined, for instance, then you might decide to tolerate the dull visuals, but if you value what you see instead of what you hear, then the game is garbage. Neither opinion is wrong.
To be fair, these anti-bias crusaders mean to say they are not unfairly biased. The people they target allegedly decide in advance what score to give a work and analyze it accordingly. Perhaps the reviewer despises the author and takes pleasure in tearing down his work. Or, the reviewer dons the dreaded rose-colored glasses and refuses to acknowledge even the most blatant flaws. However, spotting this bias isn't as easy as some people claim. For starters, an abnormally high or low score, by itself, proves nothing—remember, there is no "correct" review, so there can be no correct score. Even a history of unusually high or low scores, no matter how prevalent, won't guarantee unfair bias as to a specific review.
Take, for example, PSXtreme, a magazine that published reviews of PlayStation One ("PS1") games. Metacritic, which averages scores from professional reviews, has entries for 256 PS1 games, with an average score of 69% and a median score of 71%. PSXtreme, on the other hand, reviewed 238 PS1 games, with average and median scores of 83% and 85%, respectively. Additionally, only 29 games have a Metacritic score of at least 90%, whereas a whopping 79 games earned at least 90% from PSXtreme. Notably, PSXtreme rated Bubsy 3D at 93% and awarded it the "Gold X Award," in stark contrast to most other reviewers who found the game mediocre at best. The PSXtreme review praised the game's controls and level design, two aspects that drew the ire of nearly every other critic. But the reviewer admitted to generally disliking previous Bubsy games. So a critic who should have been biased against the game instead stood as one of the few who felt the game deserved great praise. Thus, scores alone do not tell the whole story.
Interestingly, indie reviewers who complain about bias in mainstream reviews sometimes come dangerously close to committing a similar error: presuming a review is unfairly biased simply because it's mainstream. Yet, their arguments have merit. For example, Jeff Gerstmann, a video game journalist, wrote a review for Kane & Lynch: Dead Men back when he worked at GameSpot. He gave the game a 6.5/10. Soon after, he was fired. Coincidentally, the publisher of Kane & Lynch previously purchased advertising space on GameSpot's website to promote the game. Several years later, in an interview with GameSpot, Mr. Gerstmann revealed that the Kane & Lynch score played a large role in his firing. Some publishers exert pressure more overtly. 3DO released a game called Portal Runner, which received an unfavorable review from Game Pro. In response, Trip Hawkins, 3DO's CEO, wrote a letter to Game Pro, reminding them that advertisers, not readers, are Game Pro's customers. He also promised to cut back on advertising as punishment. The reviewer countered by defiantly posting the letter to the public.
People are right to be suspicious of mainstream reviewers. Yet, they often attack the outlet itself. In this regard, IGN is a popular target in the video-game community, but IGN doesn't write reviews—journalists who work for IGN do, and not every journalist will bow to pressure from advertisers. So before dismissing a mainstream review as unfairly biased, at least give the review, and the person who worked on it, a chance.