Johnny Lee talks about Wii and the Mutilation.
NWF: At the TED conference, you talked about the accessibility of the Wii and the projects you're working on, like the interactive whiteboard, the demonstration of head tracking, etc., and how that accessibility motivates you. What do you think this means for other people doing similar research, either formal or informal? Because it's so accessible, will we see similar projects?
JL: I don't know. I guess if you consider the work I've done to be somewhat either motivational or inspirational, then sure. I think that would be great, to see more work like that. In general, I think there's always been room for the type of motivation that I've presented, coming up with really cheap solutions that may not necessarily provide 100% of the capabilities of some of the higher end options, but are good enough for a wider population, and as a result, it becomes attractive technology not because it's the best, but because it's the most accessible. And actually, it's probably less novel of an idea than some people might think. I've been reading a book called The Innovator's Dilemma by Clayton Christensen, and he introduced the concept of a disruptive technology, and for the most part, his definition of a disruptive technology sort of fits that description of a technology which may not necessarily provide the most outstanding performance but does have a much different price point and as a result becomes much more attractive. His book wasn't necessarily my inspiration, but I guess you could say I came to the same sort of conclusion on my own. It's been a motivator in my work, and if it's a motivator in other peoples' work, that would be great, but I don't necessarily see a reason why it would become more or less popular than it was before.
NWF: Most of the other projects that we've seen that involve the Wii remote are just using it to play other games. Why do you think so many armchair developers and researchers are focusing on that aspect?
JL: It's easy and obvious. I think it's sort of the most straightforward thing to do. If you have a controller that you can talk to, what else can you control? This input device has, for example, an accelerometer and an accelerometer is very good at detecting tilt, so you make something that needs tilt control, like a video game. I've also seen some projects which use it to detect the orientation of a screen, and they have a virtual ball that moves around. If it's good for detecting orientation when you rotate it, then people will start out by coming up with orientation-based ideas. It's sort of the first degree idea, and it's going to be the one people do first simply because it's the easiest.
NWF: One of the other things you talked about at the TED conference was that people in schools were already using some of your ideas, like the interactive whiteboard. Do you think that any of these projects will suffer any sort of stigma when it comes to schools because they're affiliated with gaming? At its base, this is technology associated with gaming, because that's what you're using.
JL: That's an interesting question; I haven't encountered that one before. My guess is that, if the technology provides an economically attractive solution, that bottom line will overcome the association with the gaming industry. In some circles, gaming technology is becoming so sophisticated that it's earning respect in more general technological appreciation circles, and people are taking games more seriously -- especially with this generation of kids who are growing up with video games and technology. Some educators would find the ability to somehow turn a gaming technology into an educational product very appealing, because you can maybe appeal to the children a little bit more. I think that, if there's an institution which says "we're not going to look at the Wii remote because it's a gaming technology," they're a little bit short-sighted, and they'll be in the minority in the coming years.
NWF: I certainly agree with that! But on that same line of thought, do you think that a gaming console can help change the way society interacts with computers?
JL: Yes! I think it's done so already. If you think about computing in general, it includes a wide variety of technology. Usually, when people say computers, they're thinking of something with a keyboard and a mouse, and maybe a web browser and a word processor on it. Computers include your phone, your microwave, and most living room devices like DVD players and stereos. I would argue that the video game console is in fact a computer, and in the circles that I run in, that's a relatively non-controversial concept. But the kind of activities people think about doing with a gaming console currently are limited to playing video games, and as a result, if it's not productive, people don't think it's really a useful computer. But in fact, the console is becoming a very, very powerful computing platform, and so for example, things like surfing the web become much more plausible in a console format, as does accessing on-demand video, as well as more sophisticated things like programming your digital video recorder. If you just think of this as a general computing platform, it's a channel into a wide variety of computing tasks. They may not have word processing, but it's no less computing for that. I think the game console tends to evolve a lot faster than the desktop computer, where people are really attached to the keyboard and mouse, and it's really hard to pitch a new input system that may not... well, the keyboard and mouse is so efficient that a new input system has to compete with that efficiency and cost, and thus it's always very difficult to work in that space. But in the gaming space, you can experiment with lots of different input technologies.
NWF: There's so much convergence now with the gaming consoles -- they're doing so many things, like media centers, and those traditional computing functions -- and you can do so much beyond just playing games. Do you think that's connected to the kind of experimentation people are doing with the Wii remote and other consoles? Do you think that the one follows the other? Would people have thought of these things if gaming consoles weren't expanding the way that they are?
JL: To some degree, I think the experimentation with the Wii remote is independent of the integration aspect of the platforms, largely because the integration of the all-in-one living room device is a corporate agenda, and the operating systems that allow the consoles to do that is not open. Sony, Microsoft, and Nintendo all control that. The experimentation with the Wii remote, on the other hand, I think was an accident. Mostly because Nintendo did not necessarily intend to support open experimentation with the controller, and as a result ... I think everything done with the Wii remote was completely unintentional on Nintendo's part, and the fact that they did nothing to stop people has allowed a lot of exploration. The integration of the consoles and the experimentation with user interfaces with the Wii remote are somewhat independent of each other, though they deal with the same commercial domain of video game technology.
NWF: Speaking of what the companies are doing and/or not doing, there was some discussion that EA was going to include your head tracking in
BOOM BLOX, and then suddenly it was announced that head tracking was
no longer going to be available as the discussed
easter egg. We're pretty interested in what happened there, if you can discuss it at all.
JL: I was a little disappointed to hear about that. I have been in touch with Louis Castle, who was involved in that decision, and it's actually a pretty complicated decision, and I don't know if I'm at liberty to explain exactly why, but I understand why they decided to take it out. Rest assured that I know EA is still interested in including it, and so are several other companies, so even if it doesn't come out next month, it will probably be integrated into games within the next year.
NWF: So you do think we'll see it in this generation?
JL: I think so.
If Nintendo doesn't do it, I'm fairly sure that Sony and Microsoft may be interested in doing something similar. It's simply ... the kind of visual experience that head tracking provides is simply an evolutionary step in display technology and interactive displays, and it would be foolish not to embrace that progress in technology. It's not so much a feature, it's just a technological step forward, similar to 3D TV, that will eventually come. Nintendo may or may not decide that this is an okay idea, but even if they don't, it will come in some other form from some other provider.
[NWF note: Sony, in fact, has a head tracking demo of their own] NWF: You know, that's interesting, because there's a lot of discussion any time we see another of your projects. Nintendo fans just blow up with excitement, because what you're doing with the Wii remote is so much more interesting than what a lot of actual Wii games have done with the motion controls. Why do you think that is?
JL: Well, I don't have any rules to conform to. In truth, when choosing to make something for a game console, a developer has to sign a lot of agreements and play within a very well-defined space that determines what they can and cannot do. I have been talking to some Wii game developers and they've said that ... if a game requires too much motion or requires ... they had some word for it, but essentially, if it requires too much movement on the player's part, Nintendo asks them to pull it. There are all these internal guidelines they have to conform to that prevent them from doing anything too -- I hate using the term "outside the box," but this is a box that has been defined by Nintendo and they literally can't step out of it. The video game industry is also extremely market data driven, which is unfortunate. The investment levels keep going up and up, and the certainty of return keeps going down, and as a result, marketing has more control over development. And if marketing says something isn't going to sell, or if marketing has no data on it, so if something is too radical or if it's a new IP with a new story or new characters, and it's untested, marketing tags it as high risk. If there are safer, but more boring [laughs], the decision tends to be to make the more boring title, which is unfortunate. That's actually why I decided not to work at any game companies specifically, like Electronic Arts or Ubisoft, because my ideas would have likely been squashed quickly by marketing.
NWF: Do you think those same marketing people, or maybe anyone at these companies, is really paying attention to the reaction your projects are getting? Obviously somebody is listening, because they're contacting you, but do you think the reaction is enough to change some of those things you were just talking about?
JL: I think so, or at least, that's my impression. The wonderful thing about my videos is that they're widely available, and they've given marketing departments some data that they can work with. It's put something out into the world for very low cost -- because all I did was make a video in my house, so I didn't have to invest much money -- but marketing can now look at the numbers associated with the video, like the number of views, or they can send out a survey or poll games about these particular features, and then marketing can make an assessment about the risk of the feature. So, I think the main contribution these videos have had, at least to the gaming industry, is that they've given marketing some data that indicates this is a worthwhile feature. Getting it integrated into near-term titles is a little difficult, though, because it takes a few years to build a game. If they're going to retrofit an existing title with these features, it adds risk to the title. If they're going to build a new title around this feature, it's not going to hit the shelves for at least two years or so, or a year if they do a really simple title. It's sort of interesting ... I've gotten exposed to the business of video games recently. It's unfortunate that the business of the video game industry almost seems designed to squash innovation, or at least discourage it, because it's risky.
NWF: Well, that explains a lot of our Wii games!
JL: The Wii also has another complicated issue. Actually, I would say there are two. One is that the input control system is so radically different from the other two consoles that the only way to make a good game is to start from the ground up for the Wii. If you port an existing game, it's usually not going to be very good, because the control schemes don't map over very well. Cross-console game publishing has become a popular business model, simply because it's a more attractive return on investment for game developers. So, for the Wii, they have to take on the risk of making a game just for the Wii instead of doing that.
Another problem ... accelerometer data is actually very hard to work with, so most of the games have very simple shake recognition, sort of an analog shaking recognition, sort of like the Rayman games, where you make the person run faster by pumping faster. But really complicated gesture recognition is actually very hard to do. Some dancing games, for example, are very difficult to make trigger reliably, because you want to make sure you're getting positive movement. You want to correctly register movement, but you don't want to be too generous or too critical. If you're too generous, the game is too easy, it's not compelling, you can cheat at it .... If you're too critical, the game doesn't trigger properly and then you get frustration on the part of game players.
NWF: I'd also assume shaking speed and method is harder to translate in a game manual than something like "press A for action X."
JL: Right. And game developers just don't know how to use it yet, because it's pretty new. It's pretty radical, and it has a lot of its own technical issues, not only in coming up with interesting game content, but also in dealing with the control scheme. It may take another year or two before game developers are used to working with an accelerometer. They had a decade to get used to the joypad. The technology in the Wii remote is still new to developers and they aren't necessarily using it in the best way yet, so there are a couple reasons why Wii games have at times been less than stellar, and maybe haven't lived up to the hopes and dreams of Wii owners.
NWF: Are there any that you've noticed that actually do make pretty good use of the controls?
JL: Well, Nintendo has done the best, because they had the longest time to work with it.
Wii Sports is fantastically done ... in fact, I was talking to a Wii game developer, and apparently
Wii Sports is so good that it's a problem, because people buying the Wii almost don't feel compelled to buy another game, so they're having a hard time getting people to buy their games. It's also hard to make a competitor. For example, if you're making a baseball game, you're essentially competing against
Wii Sports, and that's a challenge. There were some games I saw in prototype form, and I don't think I'm allowed to talk about them, but there are some interesting things coming down the pipeline. They weren't "WOW, AMAZING," or world-changing, but they were much better examples of the controls than some of the things we've seen before.
NWF: Hey, that gives us hope! That said, I really just have one more question, and it's probably the one you won't answer.
JL: [laughs]
NWF: I've read in some other interviews that you have more projects coming. Can you give us any hint at all of what we can expect, or maybe when we'll see them?
JL: I've been really busy lately... [laughs] It's sad, because I haven't been able to do any work since December. I've actually been on my job search the past few months, and dealing with my job search and dealing with the attention from the Wii remote projects has consumed my extra time. And then I'm defending my thesis -- I'm graduating next week, so I've had to deprioritize making new videos. I do have a couple more ideas. I don't know ... it's hard for me to tell if people are going to get excited about them. There's also a closing window here. I don't know how much time I can spend on publishing these videos, partially because once I start employment, I may not be able to continue openly publishing and sharing these Wiimote videos. My hope is that I can, but I don't know. Maybe next week or the week after that, I can push out another video.
[NWF note: said graduation is now this week!] NWF: Well, on behalf of our staff, and all of our readers, everyone wishes you the best of luck and probably thinks that, out of everybody in the world, you shouldn't need much of that luck. I'm surprised you even had to search for a job.
JL: Well, it's actually been a pretty easy search, and I had a lot of opportunities I didn't even ask for. And oddly enough, Nintendo was not among them.
Nintendo never contacted me. NWF: Never contacted you at all?
JL: Nope. It's okay. I don't think I'd actually want to work solely at Nintendo. I have a pretty diverse set of interests, only some of which are relevant to gaming, so working at a gaming company isn't very interesting to me.
NWF: Congratulations on graduating, and thanks for taking the time to talk to us during this busy time!
JL: No problem!
