Video game studies are inherently flawed, according to a writer for a Canadian quarterly, because in most cases researchers are making subjective appraisals of games they don't even understand.
Video game studies - especially concerning violence -are a staple of weekend fare, and every time we put one up, the reactions are pretty typical. It's not to say they're wrong, but Chris LaVigne's examination may help you focus your thinking the next time you read about one of these things, and wonder how it got its results.
Researchers often pair up completely unrelated games but act like they're equivalents. One experiment contrasts sci-fi first-person horror game Doom 3 with falling-bricks puzzle game Tetris. Another pairs dark and suspenseful stealth game Manhunt with a colourful, fast-paced game based on the Animaniacs cartoon. Modern blockbuster titles with lifelike graphics and complex gameplay are compared with shareware versions of Pac-Man. You get the feeling that if video game researchers studied fruit, they'd see no difference between an apple and an orange.
Yes, but don't both appear in Pac-Man? OK, kidding. Let's continue:
Most researchers assume that video games are completely interchangeable with one another, a concept any gamer would find as ludicrous as the idea that all books are the same or all movies are basically identical. One study by two American media researchers acknowledged this limitation. In an article published in the Journal of Communication in 2007, James Ivory and Sriram Kalyanaraman carefully chose to contrast violent and non-violent games with very similar gameplay styles and presentations. Probably not coincidentally, their study found no significant differences in aggression levels between the players of the different games.
Of course, to understand the obvious differences that LaVigne points out would require one to have some exposure to current video games. And if one does, you're probably not arguing against that person in the first place. I think that's why so many studies have such an uptake in the mass media - that lack of understanding, coupled with the credibility of a respected university bootstrapped to the study or its researchers.
Interesting side note: In effect, one of LaVigne's criticisms of the validity of video game studies reads a lot like... a criticism of the validity of video game reviews. "Ranking games with numerical values gives the illusion of precision without really meaning anything," he says. So true.
Why Video Game Research is Flawed [Maison Neuve]