I personally would say leaving it as long as possible...

Kids with too much exposure to games can't learn to concentrate on other things like reading books and such.. Of course, this does not apply to everyone but it is really troublesome to keep children entertained (without games) after they have been exposed to games.

I started when I was 5 years old and I consider myself a normal human being.

I think the more appropiate answer is not when he should be starting to game but what kind of games he is going to play.

Edit: It's also important how much time he is going to spend playing videogames!

It was my mother who introduced me to gaming when I was three. Back then we'd play platformers mostly: Mario and Donkey Kong.

As soon as they understand that they shouldn't get all their fun from playing games. You need to be outside, playing (not games!) with your friends too.

Let his first console be a Sega Genesis
He will thank you in 20 years, trust me

Depends on the kid really. Video Games helped me learn to read. I got into Pokemon (and other JRPG's)  when I was 6/7 while I was learning how to read, and it helped a lot. Then i was reading children novels in second grade (Harry Potter), while in class we had pamphlets with three sentences on each page. Kids will find other things to be distracted by if they really want to.   As long as you just set a time limit on games when you feel that they are becoming overwhelming. 

I started gaming when I was 4 so this should be around the right time to start. Check obviously that the game content is appropriate and turn off the controller vibrations.

P.S: Since this is thing your probably most worried about: I turned out fine and I'm not addicted to gaming or anything.

Make sure he starts with Super Mario Allstars and other old games, don't tell him the WiiU or 3ds exists start that kid on the classics of old. If his first game is Super Mario 3d world he'll never be able to enjoy the greatest games of all time afterwards. He'll just look at the screen and vomit.

Crash 2/3

Super Mario World

Ocarina of time!!!

5 or 6, that's when I started gaming and I turned out fine.

Just make sure to let him play the Snes/Megadrive and PS1/N64 era first.

I started when I was 4 but only serious starred playing when I was about 13. Just don't let him or her get addicted. Only let them play a lot of they're playing a lot of sports and doing their learning work. The last thing you want is an overweight and lazy child. If I wasn't ushered into sports at a young age and left to my gaming devices I'd be in a bad way now. Remember your health is your wealth.