A recent paper on Steam’s Early Access program from a trio at Queen’s University in Ontario, Canada goes to show that if you have enough data, you can research anything. What’s great about the paper is that its conclusion includes useful pointers that developers contemplating a stint in EA can learn from.
The paper is available as free download from ResearchGate, if you’d like to peruse it yourself. Alternatively, I’ve done my best to pull out the more interesting bits.
Compiled by Dayi Lin, Cor-Paul Bezemer and Ahmed E. Hassan of Queen’s University, the paper gathered information from over 1000 Early Access games. Some of the conclusions seem obvious, given how EA works, but it never hurts to have the numbers to back those conclusions up:
We observe that on the one hand, developers update their games more frequently in the early access stage. On the other hand, the percentage of players that review a game during its early access stage is lower than the percentage of players that review the game after it leaves the early access stage. However, the average rating of the reviews is much higher during the early access stage, suggesting that players are more tolerant of imperfections in the early access stage.
Pretty standard, right? The good stuff comes along later. For instance, Double Fine’s much-maligned Spacebase DF-9 gets a mention:
On October 27, 2014, the game unexpectedly terminated the early access stage and released a final product that lacked many of the planned features … The abandonment of the game led to the disappointment of a large number of players. As a result, the game received 79% (2,598) negative reviews, and raised a debate between the players and the studio on the discussion forums of the game on the Steam Community … The game is considered to be a failure of the early access model.
It’s not all doom and gloom — the paper includes the following five lessons:
Lesson 1: It is risky to use the early access model as the main funding source.
Lesson 2: Do not release a game on the SEARP too early.
Lesson 3: State promises and plans clearly.
Lesson 4: When a game is abandoned by its development studio, the reputation of the studio as a whole can be damaged.
Lesson 5: Communicate issues and changes to the promised plan on time.
The paper expands on each of these points and I recommend, at the very least, giving this section a read (starting from page 26).
Unsurprisingly, a lot of it boils down to communication and managing expectations. I think most Steam users have been scorched by EA games that promise big and deliver little (or nothing at all).
I’m sure more games will fit this mould in the future, but it’s harder these days for less-than-honest developers to get away with this sort of behaviour.
An Empirical Study of Early Access Games on the Steam Platform [Research Gate]
Comments
7 responses to “Research Paper On Steam Early Access Reveals 5 Lessons For Developers”
Communication is part of it, no doubt.
However, I think most devs get into trouble before they even get to talking publicly about their project.
Most aim straight for the stars, rather than starting with a smaller goal, like landing on the moon, and expanding from there. Of course, going public and telling everyone you are going to give them a brand new shiny Ferrari, then only delivering a Ford with buckled rims, will disappoint a lot.
There are many reasons why this might be though.
In my experience, management, arrogance and pride are big killers.
Management often can’t see past $$$ signs, and discarding something where money and time where spent creating is a huge issue for them because that is all they are focusing on. Obviously money is very important – but if all you think about is money without exception, then your product is going to suffer because you never consider how to make it better.
Arrogance and pride is the other, where someone thinks that because it’s their idea/creation that it will just work, and be great, and that others will automatically love it. But it doesn’t work like that. Sometimes things just don’t work. While some can be adjusted/tweaked/nurtured into something great, others just need to be taken out back and shot – the key is being objective and realistic.
ok, I could go on and on about this subject because it is close to my interests, but I’m gonna leave it there.
*edited to include pride in addition to arrogance (I couldn’t think of the word earlier)
The irony is that schemes like Kickstarter and Early Access were supposed to free developers from the publisher/manager model – the devs were in charge and could do anything, if only you handed over the money! Unfortunately, as plenty of these high-profile failures have shown, sometimes devs either bite off more than they can chew, or they turn in a shitty final product. I really don’t think that “management” is really the issue in terms of only thinking about money, but rather failing to budget properly and consider actual costs.
I think a bigger issue is that KS/Early Access is basically an advertising platform with a buy-in program that people treat like preorders. They tread it like preorders because developers create that expectation in people – that they will get the product they promise. Unfortunately, for a lot of these projects, those unrealistic goals quickly slip away into development hell or a funding black hole – and that pisses people off. Devs know this, will hide it (or at worst deny it) to keep people buying into it, and then give up afterwards.
I actually think Double Fine is one of the worst offenders, especially after Spacebase. There had to be some point that they suspected (and then knew) that it was fruitless, and yet they did nothing except slap an arbitrary “1.0” on it and called it a day.
Double Fine is on my blacklist after what happened with Spacebase. Any trust they had earned was lost when they pulled that nonsense.
Yep, that is what I said in my second paragraph.
Which comes back to my point near the bottom about one of the big killers being arrogance and pride. Dev’s usually have themselves convinced they have a magical idea, and can delivery everything they have planned… so they promise the stars, but are lucky if they can deliver the moon.
I concede that smaller teams are less likely to just focus on the money, and they simply fail to budget and consider costs properly as you said.
But for larger groups failing to budget, or consider actual costs, those things come from management only focusing on the money, without considering the nature of development or really knowing how it works. They keep trying to force exact time frames and budgets onto the creative processes. It just doesn’t work. Which I’ve experienced first hand.
This isn’t exclusive to software development though.
I agree with the whole ‘freeing them from publishers’ side, but argue that it’s naive to think it’s going to get rid of management too. The exception would be small indie teams.
Yep, and sadly that is marketing. I completely agree it makes things worse. Games take a LONG time to develop, but most don’t know that – and kickstarters, etc. never mention it could be years before they get anything. Again, marketing. You don’t mention all the ‘down’ sides if you are trying to convince people to part with their money.
I’ve been involved in a game that went through Early Access basically right when it first began, and I’m going to be entering it with my own game in the next few months.
I don’t know if I agree with the part about not using it as a primary funding source. For the game I worked on it funded a team of around 7 people for a good four years. Things are a little different now though, with people being way more cautious and wanting substantial amounts of gameplay ready before buying in. For this game I did, we didn’t even have gameplay. We literally sold people on a video that outlined our plans. We were in the 2nd round of EA games to be accepted though. That totally wouldn’t work now. But, a lot of teams till fund themselves primarily through EA.
As for releasing too early, I think you can still release fairly early, but you need to handle the communication well. Too many devs are way too formal and vague. I think you need to be completely transparent. And with the right intentions of delivering a good game, I think that can get you far, if not all the way.
The paper is just trying to suggest that “be aware that using the early access model as your main funding source is a risky strategy”, as once it is not able to support the project and the game cannot finish, it would be risking the reputation of the entire studio rather than just a game…
Pretty sure that steam say in there terms and conditions that it can’t be used as a primary revenue source and that the dev commits to delivering the game in full even if they sell nothing.
In part that change was due to a couple of high profile ‘failures’ such as starbase DF-9
The point is that it is risky to base your budget solely on Early Access sales, not that you shouldn’t. It worked in your case, although I wonder if those 7 people had other income sources and their circumstances. It’s one thing to make a game when you’re working already and another to base your entire financial security on the premise that people will continue to buy your game in enough volume to generate thousands of dollars each month. (including tax and development/business expenses)
The main point is that you can’t quit your job and expect to be funded by EA sales. You need to do a risk analysis and make sure that the market is there and that you can deal with any problems like low sales periods or people being injured or quitting.