A recent paper on Steam's Early Access program from a trio at Queen's University in Ontario, Canada goes to show that if you have enough data, you can research anything. What's great about the paper is that its conclusion includes useful pointers that developers contemplating a stint in EA can learn from.
The paper is available as free download from ResearchGate, if you'd like to peruse it yourself. Alternatively, I've done my best to pull out the more interesting bits.
Compiled by Dayi Lin, Cor-Paul Bezemer and Ahmed E. Hassan of Queen's University, the paper gathered information from over 1000 Early Access games. Some of the conclusions seem obvious, given how EA works, but it never hurts to have the numbers to back those conclusions up:
We observe that on the one hand, developers update their games more frequently in the early access stage. On the other hand, the percentage of players that review a game during its early access stage is lower than the percentage of players that review the game after it leaves the early access stage. However, the average rating of the reviews is much higher during the early access stage, suggesting that players are more tolerant of imperfections in the early access stage.
Pretty standard, right? The good stuff comes along later. For instance, Double Fine's much-maligned Spacebase DF-9 gets a mention:
On October 27, 2014, the game unexpectedly terminated the early access stage and released a final product that lacked many of the planned features ... The abandonment of the game led to the disappointment of a large number of players. As a result, the game received 79% (2,598) negative reviews, and raised a debate between the players and the studio on the discussion forums of the game on the Steam Community ... The game is considered to be a failure of the early access model.
It's not all doom and gloom — the paper includes the following five lessons:
Lesson 1: It is risky to use the early access model as the main funding source.
Lesson 2: Do not release a game on the SEARP too early.
Lesson 3: State promises and plans clearly.
Lesson 4: When a game is abandoned by its development studio, the reputation of the studio as a whole can be damaged.
Lesson 5: Communicate issues and changes to the promised plan on time.
The paper expands on each of these points and I recommend, at the very least, giving this section a read (starting from page 26).
Unsurprisingly, a lot of it boils down to communication and managing expectations. I think most Steam users have been scorched by EA games that promise big and deliver little (or nothing at all).
I'm sure more games will fit this mould in the future, but it's harder these days for less-than-honest developers to get away with this sort of behaviour.
An Empirical Study of Early Access Games on the Steam Platform [Research Gate]