It's less about buttons and more about having some sort of tactile feedback. And by feedback, I don't just mean vibration.
It's the difference between acting out a scene with props and faking it pantomime style. Pointing a gun prop and pulling the trigger or pointing your finger and moving it up and down.
Touch based controls are still "touch" based. Crazy concept, but yes; it's true. You are physically interacting with a display. And of course, plenty of people still complain about it saying that they need a mini-button keypad to text with or a physical joypad to move a character around screen as reason why they are incapable of ever gaming on a touch screen phone or tablet.
Pretty much any game that has a character being moved around on screen by the player works best with a pad or stick of some kind. Gesture based movement basically equates to making a character walk or move on screen by pointing with the hands, which is not VR type motion based movement (where you are "in" the game) so much as replacing directional input with pointing gestures.
See Pac-Man on Kinect if this issue isn't clear.
Controller free interfaces will work best in certain cases though; basically anything that uses full body control in a static position (dancing in place, running in place, walking in place and any kind of action performed from a static stance). That's a pretty broad range of possibilities for games, but not for the vast majority of traditional games (RPG, FPS, any non rail shooters, adventure, etc.) which has the player moving a character around with a stick or pad.
MS more or less said it themselves that Kinect games are designed from the ground up to work with a camera based motion tracking control scheme.
So one should expect this means we won't be seeing developers swapping button presses with simple gestures that do not match the on screen action. But this takes a different approach to designing games, which is where a developer can either succeed with something yet to be seen, or more of the same, only swapping hand waves, arm flapping and kicks for controller shaking/waggle or frantic button presses.
So the jury is still out on this, but we'll see the potential pay off in 2011.