Stolen off NeoGAF:
http://pc.ign.com/articles/111/1118657p1.html
When George Orwell wrote the dystopian novel 1984, he laid out a possible future in which the government watches everything in a heavy-handed approach to population and thought control. Big Brother was always watching, a concept that still sends chills down the spines of every privacy advocate out there. Thankfully, 1984 never came to be.
But as it turns out, Big Brother is watching; not in an effort to control you but rather to learn from you. You're not just playing videogames anymore. You're actively providing feedback about what parts you like and which you don't. How you play could ultimately help shape the future of videogame design.
BioWare is just one of numerous development studios and publishers that have begun collecting anonymous player data. No identifying information is tied to the information harvested, so you don't have to worry about things being traced back to you. You're just a data point amongst millions. I spoke with Casey Hudson, Executive Producer for the Mass Effect franchise, about how this is done and what the studio has learned so far.
"The only data that we get are in terms of events -- little things that happen in the game," explained Hudson. "Let's say if we want to know whether players skip lines of dialogue, we can have that become a little event that gets sent up. It's all completely anonymous, so all we get is raw numbers for how many times these kinds of events occur. Then we can start getting ratios and comparing proportions and things like that. It becomes this mass of numbers, and then we have to try to figure out how we would interpret that."
This is a process that is new for BioWare with Mass Effect 2, one that Hudson hopes will aid the team in designing future games. "When we were looking at the Achievements for Mass Effect 1, that's where we realized that there are some really interesting player behaviors there that it would be nice if we could answer more questions for ourselves. [Then] we could understand what's going on and how people play our game."
"Sometimes you'll design something and think that it's going to be used in a certain way and people will use it in a completely different way. And if you didn't know that, then you would just keep making that system the same as you did before. But once you know what players like and what they don't like, based on the way that they're playing it, then you can make more of the good stuff and less of the stuff they weren't interested in."
This data is used for top-level design and isn't anything that could have been used in the making of downloadable content, or even to alter Mass Effect 2 for PS3.
"There are some things regarding difficulty and weapons and things like that and those are easier to tune, but there won't be a huge opportunity to include this stuff for the PS3 version. It's about how you design a game from the ground up."
"Ultimately it doesn't always give you the answers, but it sometimes raises questions or gets you to ask the right questions…More people played the soldier class than all of the other classes combined. If you know that, then you can start thinking about future games. Is that good? Is that a problem? Should we look at the other classes and start thinking about ways to make them selected as often as soldier? As part of asking these questions, we can design games in the future a lot better."
It's not just about raising questions. Sometimes the data gathered reaffirms beliefs. "It helps us to confirm a lot of the assumptions that we make when we design games. Obviously with a game like Mass Effect we're trying to capture the sense of continuity and the cinematic experience. More and more we're trying to create something dynamic and exciting like a really great movie and we're trying to get away from dialogue, though even a movie has conversations. This set of decisions we make for how you would experience that and the fact that we have a dialogue system and things like that, we're kind of assuming and hoping that we're right in that people are interested in this kind of experience."
BioWare found that only 15% of conversations were skipped in Mass Effect 2, with the rate higher in non-critical moments like in the hub worlds and much lower later in the game at the climax. "If we found that 80% or 90% of the lines were being skipped, we would have to reevaluate the work that we were putting into the digital acting."
Other times, the data surprises. That happened with both how there were differences between the PC and Xbox 360 populations, as well as in regards to importing saves from Mass Effect 1. "We were surprised by how many people imported a game from Mass Effect 1…We put a lot into that feature and we could have gotten data back that said nobody was doing it. But actually more than half of players imported their save game from Mass Effect 1. That to me is quite high."
The average time to play through the game was 33 hours. PC gamers spent about an hour longer, while Xbox 360 players did 10% more loyalty missions on average. "In general, pretty much all of the data for the Xbox 360 version and the PC version are quite similar. One difference was the people who did certain loyalty missions on the Xbox versus the PC, which is kind of surprising. On the PC for example, people did Miranda's loyalty mission quite a bit, which is where she is trying to connect with her sister and it's more of a touchy-feely plot. Not a lot of Xbox 360 players did that one. But the Xbox 360 players did do Grunt's mission a lot more than PC players."
The PC players have the edge in dedication. Hudson claims that a lot of people played Mass Effect 2 more than once and about half of all players -- including those who rented or borrowed it -- who started played all of the way through to the end. Two players in particular on the PC played through 28 times. That's the current record. Four people on Xbox 360 played 23 times. If you're any of these people, please write in to us and tell us how you have so much spare time.
Some of these statistics are nothing more than that. As Hudson noted, there isn't any real takeaway in the differences between Xbox 360 and PC players. "Even if you know what some of these player behaviors are, is there anything you should do about it or is it just something interesting?"
There was a great improvement in quality from Mass Effect 1 to Mass Effect 2, and that was without collecting this telemetry data. If this endeavor proves useful, we can expect great things from the final chapter in the trilogy.
"Ultimately what it means for players is that we don't have to guess anymore about what players are doing. We can actually learn about what they like and what they don't like and just try to focus on building the good stuff."
Sounds good to me.
