industry news
Former XNA Community Manager Airs Quality Concerns
Posted by Leigh Alexander at 8:40 AM on July 26, 2008
Microsoft's recent announcement that it would let community developers earn money for games they make with the company's XNA toolset came as good news to some, but former XNA community manager David Weller is concerned about the lack of quality standards. On his blog, Weller wrote:
Being an ex-XNA member, I can still say, without a shadow of doubt, that Microsoft is offering a groundbreaking game channel, and that some people stand a chance to make great money from the system. It's an exciting opportunity, but the danger for consumers lies in Microsoft's deliberate steps to avoid discussions regarding game quality, even during peer review.
It's similar, actually, to quality concerns about Xbox Live Arcade that ultimately led to Microsoft's announcement that underperforming titles would be delisted from the service.
Is it as big an issue on the separate community games channel? Said Weller:
I firmly believe that avoiding commentary/ratings on game quality will result in frustrated consumers, who will have no way to discern the quality of a game among (ultimately) thousands. Of course, the game creator has the option of offering a trial game, but it will be interesting to see how many developers take advantage of that path, as it is not required [Edit: Peter Hatch points to a Wired article where Chris Satchell states that a timed trial will be enabled by default, but my concern about te lck of quality indicators still stands]. I doubt consumers will get a refund for buying a sucky game either.
Microsoft's position has always been that the community is self-policing; current XNA manager Chris Satchell told GamesIndustry that, "When you look at strong communities there is always a core that will work really hard to make sure community standards are maintained... a lot of submitted games do get rejected at first, due to either bugs or rating descriptors being inaccurate".
Making money from your XNA game: The good and not-so-good [Let'sKillDave! via GamesIndustry]

Comments (AU Comments · US Comments)
There are currently no AU comments for this post.
Thugmunk
Posted 2:43 PM 26/7/08
So if he rejoins the XNA, that would make him an ex-ex-XNA member, right?
Sounds risque.
Thugmunk
kftgr
Posted 12:06 PM 26/7/08
@ArmiMaan:
From XNA Creators Club faq:
"What is keeping Creators Club members from certifying games that have inappropriate sexual or violent content?
There are checks and balances throughout the peer review system which enables community created games to be submitted, approved and ultimately distributed over Xbox LIVE. Each peer reviewer is rewarded with a better reviewer reputation by the system for accuracy. Conversely, reviewers can get penalized or even banned from the ability to review games should the need arise. If inappropriate content makes it through the peer review process, the process has mechanisms to support reactive takedown measures as needed."
kftgr
Janos2615
Posted 11:57 AM 26/7/08
I just love the internet...
When Sony SUGGESTS doing the same thing, the internet melts down. Yet when Microsoft does the same thing, everything is fine and dandy in a field of roses...
Janos2615
Shin-san
Posted 11:55 AM 26/7/08
@ArmiMaan: I agree. I hope that the process won't be incredibly political with the members of the community. Even though it seems like Microsoft is making it self-regulating, hopefully they will step in once-in-a-while to keep things under control.
Shin-san
Thedavid
Posted 11:38 AM 26/7/08
I could never get that XNA program to even work on my Windows XP, which is kind of ironic.
Thedavid
Heathro
Posted 10:52 AM 26/7/08
@parad0x360: You're right, all you have to go by for many retail games is word of mouth and review. Where you're wrong is how you missed the line drawn between XNA Community games and retail games. The community games have no quality standards. I agree that many retail games are shipped in horrible states but the fact of the matter is that they have passed certification and meet standards. The issue he brings up are a concern for certain. Even if a game has relatively few bugs, imagine if the designer chose to have 'B' always advance menus. This conflicts with the standard that every other retail game has to meet.
With that said, these community games are not meant to rival retail games in quality or standards but regardless, no one wants to have to wade through tons of garbage to find the one gem. We may end up being surprised with the quality community games but I would like to see a rating system at the very least.
Heathro
dunetiger reads kotaku, seems pleased
Posted 10:47 AM 26/7/08
Hey, all I need to know is that XNA creators do have the option to put out a trial version, just like everybody else. If they don't wanna do that, then I'm certainly not going to take a leap of faith just 'cause they say so. Like an "artist" who always has arms and legs leaking off the page to avoid drawing hands and feet, a game without a trial is saying a lot.
dunetiger reads kotaku, seems pleased
Xenigma
Posted 10:16 AM 26/7/08
All I know is that after seeing a small handful of XNA titles that are supposedly being released, I'm fearful for the service. While I think many of the games being released for Arcade look bad, some of the XNA titles I saw looked even worse! Granted, I've seen very little, and I'm sure some solid games will appear down the line, but so far I can't say I'm too excited about this yet.
Xenigma
ArmiMaan
Posted 10:05 AM 26/7/08
I think the biggest problem that will plague XNA games is one of nepotism. With any sort of peer review system, you'll find that a core group of individuals will inevitably take control, and this core group will only give themselves and their close friends the time of day. We'll probably find that lesser-known independent developers will struggle to get their titles through peer-review and onto the marketplace, while the elites will push through whatever junk they put out, which will defeat the whole purpose of XNA.
ArmiMaan
parad0x360
Posted 9:48 AM 26/7/08
His concerns seem kind of silly because the same problem exists for retail games. We dont always have demo's to play so we go on word of mouth and reviews. Why wouldnt the same thing work here? This guy is making an issue where one doesnt exist.
Hell there are rarely demo's for PSN exclusives. I bought Pain and some bowling game only to find out how bad they were.
parad0x360
Slothboy
Posted 9:42 AM 26/7/08
Yeah, it seems like you should be able to do a basic rating of a game. 1-5 stars and then other people can make up their own minds.
Ultimately though this will not be an issue. I estimate that within a month of the launch of XNA games on XBLA someone will put up a website where folks can rate the games. I wouldn't be surprised to see this feature implemented on some of the existing gaming community sites. If Microsoft doesn't want to be bothered with it then the community will step up and offer the feature.
Slothboy
Talleh
Posted 9:36 AM 26/7/08
I know that one of the problems for me when I first got my 360, was I took one look at marketplace and felt overwhelmed. Is a game good? Why should I get it? What do other people think? Demos sorta helped, but even then, the massive number of games made it difficult to tell what was great and what was tossed together in a weekend.
I'd love to see the user-rating system applied to the current marketplace scheme. I'd like to see movies sorted by what's famously great, and not by downloads. I can see that Good Luck Chuck got the most downloads, but how many people liked the movie, or hated it?
This kinda throws in the "Go check your PC" argument, which I could do, but I shouldn't have to. I'm not saying I deserve a rating system, but when the content gets so massive, a user can't decide on what to get and is paralyzed by the volume of choices and no direction, you think it's time to put in SOME system.
Also, categories for each letter, maybe a grouping of 3-4 letters in one category so I don't have to scroll through 5000 pages of the letter I'm not looking for.
Talleh
muu
Posted 9:30 AM 26/7/08
I doubt a lot of people are just going to straight up DL something without a demo -- they're used to having a demo by default from XBLA, and unless they're stupid saps that buy Super Monkey Ball from the App Store without giving a second thought about trying the game out or reading reviews -- oh wait, that's me, and btw Sega could I have my 10 bucks back please -- people are still going to be extremely cautious about buying something just cause it looks good from a SS.
If it's anything like Youtube the brunt of downloads will be coming from word-of-mouth anyway, and not as much from whatever advertising the XNA community decides to pull off. Review scores and such might end up being off, but I think in most cases you'll still be able to discern between a dog and an actual game by looking at purchase rates and such.
muu
armless-phelan
Posted 9:29 AM 26/7/08
This sounds all good and well, but does anyone know if the XNA tools will be used primarily for the 360, or will those of us who own a Zune get some love, too?
armless-phelan
Psudonym
Posted 9:17 AM 26/7/08
Nothing terribly wrong enacting at least some minimum standards of quality. I'd take a single Geometry Wars or Pixeljunk Monsters over thousands of "niche" independent titles hardly playable beyond an ambitious game description.
Psudonym
Datheron
Posted 9:09 AM 26/7/08
It sounds like the main beef Weller has with the current XNA implementation is the lack of feedback tools integrated into the community itself, which is perplexing considering that MS was the one who came up with having such a rating for individual players.
It's not that people can't eventually find out how well the game is; it's more that such ratings and commentary ought to be a part of the system which is presenting these games and not some external forum or review site(s). It'd be like Youtube without the comments, or the iPhone App store without the ratings - not a deathblow for the serivce by any means, but a curious omission given amount of social meta information we all participate in. Hell, it'd be like...Kotaku without comments.
Datheron
verrius
Posted 9:07 AM 26/7/08
@hungry_for_worms: The reason that's acceptable in the mods/fan content community is that its free. The only thing you spend by checking out a bad mod is time. Admittedly, that is a precious resource, but people are a lot more willing to waste time sifting through a bunch of crap than they are to waste both time and hard-earned cash.
"Timed trials" of games could go some distance towards sorting out the crap...but its likely to encourage creating an awesome 10 minute introduction with maybe 50 minutes of crappy filler tacked on the end.
verrius
Striderhayasa - Phillyyakk on PSN and Live.
Posted 9:07 AM 26/7/08
@hungry_for_worms: whatever. Go check out the UT3 mods and get back to me.
Striderhayasa - Phillyyakk on PSN and Live.
Snukadaman_
Posted 9:06 AM 26/7/08
I think people who buy games will see if there is quality in their purchase since most xna games will have demos right...i dont see the argument here.
Snukadaman_
GrrSnort
Posted 9:05 AM 26/7/08
Just to let it be known, my game will NOT have an effeminate male lead with amnesia and spikey hairy and a rotten attitude.
GrrSnort
wild homes loves you but chooses darkness!
Posted 9:04 AM 26/7/08
I have a feeling the final XNA deployment will be a lot like the App Store for the iPhone-- it's hard to straddle that line between we'd like you to make software for us and not our competitors and we appreciate the effort, but your work is crap and we're not going to host this. Microsoft have bet a lot on XNA being compelling, and they know that they run the risk of turning aside talent if they're too stringent about things-- even like bug-testing-- and it's going to result in a lot of really crappy stuff being offered up. Same as the App Store-- at least in its current form-- which is nearly unusable because it's so utterly flooded with garbage.
wild homes loves you but chooses darkness!
Diesel_Power
Posted 8:58 AM 26/7/08
Gonna be a a lot of shitty anime games made by lonely american kids.
Diesel_Power
nip
Posted 8:51 AM 26/7/08
@Arttemis: I agree, dude seems to be reaching if you ask me. I think xbl consumers pretty much know what good games are and the better ones will do just fine if you ask me. Its a no harm situation if you ask me.
nip
Arttemis
Posted 8:46 AM 26/7/08
If the trial games do actually exist for each XNA game, then the blame really does belong on the consumer's shoulders for paying something without any idea of what it is (which they would learn from the trial).
I was under the impression that only the highest rated items in the XNA subscriber's pages would even make it to the general public - so wouldn't the issue of quality already be addressed by that point?
Arttemis
Llost
Posted 8:46 AM 26/7/08
I wouldn't consider that much of a concern unless the game itself were badly advertised and badly priced to reflect the quality. It's not fair for someone who might have been working on a title for quite a long tme to have there game rejected due to somebody deciding it lacked quality.
Llost
GrrSnort
Posted 8:46 AM 26/7/08
I'm going to reattend college and my senior project will actually be a game using the XNA toolset. I hope that the community is as closeknit as Microsoft says, as I know and have faith that my game concept is a great one.
GrrSnort
hungry_for_worms
Posted 8:46 AM 26/7/08
just like user created mods/content for any game 95% will be garbage but the rel good ones will stand out.
hungry_for_worms
Placentasaurus
Posted 3:08 PM 26/7/08
@Janos2615: Sony suggested the same thing? Unless you're talking about the Littlebigplanet thing, which is completely different from this.
Honestly, I think XNA will have one or two good games, but mostly be utter garbage. Hopefully I'm wrong.
Placentasaurus
kingmanic
Posted 4:21 PM 26/7/08
@Janos2615: Free user created levels aren't the same thing as pay for play user created games. The balance can be found and it's an encouraging project but quality filters will have to be found and abuse prevented. There will be abuse.
kingmanic
Ghede
Posted 5:23 PM 26/7/08
The community is self policing. Simply put, if you don't offer tools to allow them to police more effectively, they won't buy any games except those of particular renown. They would get Desktop Tower Defense rehash, but avoid the groundbreaking new [mmo,platformer,fps,rts,shmup] from [INDIEGAMEDEVELOPER].
Ghede
BigWeather
Posted 5:43 PM 26/7/08
XBLCG games will offer trials that should give you a taste for what you are buying. XBLCG will also have a customer feedback system. It is still being finalized and suggestions still being entertained. The submission process (peer review) is not a judgement of quality, but a determination that the developer represents his/her game in terms of violence, sex, and mature content accurately and that the game is stable and not a violation of IP or has embedded porn and the like.
Here's hoping that the earlier post I typed takes, because this one is a bit more terse -- the other one looks like it got eaten. =( If it survives I apologize for the duplicate.
BigWeather
teeps1981
Posted 12:21 AM 27/7/08
The same cna be applied to DL levels for LBP. Allowing users contorl over the pricing of their game/level is dangerous for consumers but in the end...if you buy shit its your own fault.
My main conceren regarding XNA and LBP levels is there is nothing stopping people from creating new accounts and getting friends to rate their shit content highly so that people are roped into buying it.
teeps1981
fuchikoma
Posted 1:22 AM 27/7/08
I actually feel that the cost of entry for selling your games will go a long way to filter out people who are just fooling around or not so sure of what they've put together.
Whether that will be enough is yet to be seen, but while many decried the fees, I think it may raise the bar a little.
fuchikoma
pupaboy3
Posted 3:36 PM 27/7/08
No trial version? I'm not buying anything until I get to try it out simple as that...
pupaboy3
tooji
Posted 12:43 PM 26/7/08
@Llost:
Well the thing is, people aren't going to buy something (XNA tool-kit) that's 100+ dollars just to spam pointless games. People that will be buying it probably have some knowledge of what they are doing and wont dish out shit for a laugh.
tooji
ubermod
Posted 9:39 AM 26/7/08
Long time reader, first time commentator, etc.
How exactly do you define the "quality" of a video game?
Publishers would like to hold developers to ransom regarding Metacritic scores, but that apparently just leads to developers pandering to reviewers tastes. Surely XNA/Community games is there to deliver the kind of niche titles that wouldn't ordinarily attract widespread praise.
ubermod
HykCraft_Returns
Posted 9:14 AM 26/7/08
You are better off publishing your own title via digital distrubtion than having MS take 70% of your revenue.
If your game is popular, MS will put it on the front page but then they take all 100% of the profits for the first few months... then it drops down to 70%.
HykCraft_Returns