First off, I think it's worth noting that Japan really seems to like the black sheep of the series, Adventure. Adventure focused on combat over puzzle solving and had RPG elements. Considering how people say that Zelda has focused more on puzzles lately, I suppose that can hurt.
Twilight Princess is obvious in at least one respect. The Japanese don't care for the realistic art style. Then again, that can't be all of it: the N64 games did at least as well as WW percentage wise, and their style was much closer to TP than WW. Prehaps the fact that it was a launch title actually hurt TP; by the time there were a lot of Wiis, it started getting competitors, both on the Wii (Monster Hunter 3 has often been compared to Zelda in Japan) and elsewhere (Monster Hunter's enormous popularity on the PSP prehaps?)
Actually, the Monster Hunter thing might be worth looking into. Could the more recent games be sabatoged by them, due to genre overlap? Or maybe the Japanese really perfer their 2D games, which is why the handheld titles have been more popular since the 3D jump. Remember how popular 2D Mario is compared to 3D Mario in Japan.
Just brainstorming.
