I think the core gamer will end up getting both systes provide the Vita continues to get support by next years end.

But to answer the question; Yes. A portion of the audience sony should have won with the Vita probably to decided to pick up a 3DS instead, not to say they won't get a Vita at a later point. Ultimately though when the vita finally comes into its own it will deliver an experience which the 3DS can't. An example of this is AC liberation and COD. However I think the base that such games appeal to may be less concerned with Handheld gaming then JRPG fans. From the gate out sony should have been looking at getting RPG, both western and japanese on the system. I playing AC liberation and sometimes and squint my eyes and pretend I'm playing a game like Fable... :'(

I think the 3DS will surpass DS sales in Japan, developers are already begining to treat it like its the countries main platform. 

The truth is --- in Japan Nintendo have all biggest IP's now. I mean their own multimillion gems - Mario (where is 2D, 3D, Kart and others are BIG), Pokemon (main line surely wil come in 2013, spin-offs), Animal Crossing (just released), etc + 3rd party blockbusters, like Dragon Quest and Monster Hunter. It's oblivious that, for example, DQXI after success of Monsters and announced DQVII Remake will come to 3DS as well.

Nothing bigger than this in Japan. Everything is on 3DS + A LOT of great niche games with sales like 100-150K (which devs are happy about, look at Rune Factory, Harvest Moon, Senran Kagura, Etrian Odyssey etc etc etc).

The main difference with DS is Monster Hunter (everything else was been on DS, including Kingdom Hearts, FF, Dragon Quest) and minus casual sales for games like Brain Training.The market for last is gone.

In first 1,5 years there's more great games released on 3DS than on DS. And more to come. Can't wait Ace Attorney 5, SMT 4, MH4, FE, new Zelda game, Bravely Default etc here.

The word "hardcore" is so funny, lol.

No, 3DS haven't games like COD and Halo. But not only them meant "hardcore", oh lol.

In terms of 3rd party western support, Vita not going to have it in a full force, too. Look at Assassin's Creed debut, which's not moving a lot hardware, and multiplat games like Fifa and Need for Speed are dead ones there.

It depends on what audience.

Nintendo never really lost the "traditional" gamer audience since the DS and 3DS in the west because the portable gamer is mainly the same type of gamer that made the previous portable systems successful, except with a influx of gamers that liked more Eastern gaming. In a way they obviously picked up a few that never had portables but might have been fans of one of the previous console generations seeing sequels or remakes/ports of favorites appear on it.

In Japan the situation is mixed, they are getting some of their normal audiences, plus a good clip of portable fans that were supports of the PSP because the system allows for similar gaming experiences in adventure/visual novel and RPGs. However, according to sales, they are still having trouble getting some of the PSP base that may be Sony loyalists, hence why the PSP and Vita are still getting a lot of games that could easily be on the 3DS. This may be because some developers are infact still loyal to Sony or just don't want to use stronger hardware since the main reason they started developing for the PSP was because it allowed them to use PS2 development tools and cut costs.

Either way, Nintendo is getting some added pull but most of the success is still coming from the same types of gamers that liked GBA/DS games.

If be "hardcore" you mean niche audiences, then the 3DS shouldn't have a problem. Games like Monster Hunter and Shin Megami Tensei have a large following in Japan, and will attract a certain kind of hardcore gamer in the west as well.
The kind of hardcore player that would buy a Resident Evil game, however, doesn't strike me as one who would be hugely interesting in playing such a game on a handheld system. Save for the Japanese.

I expected P4G to have a bigger impact in Japan.

I suppose it wasn't a big enough game to cause a long-term rise in Vita sales (though the game itself sold well imo), or maybe those who are most interested in P4G already owned a Vita to begin. Even so, my point is that the if the 3DS (or Vita) gets more games of that type, the gamers will come. There will always be one major game to sway buyers, but it will be more effective with a host of supporting, smaller games (or remakes/re-releases of games).

A SMT game built from the ground up for Vita might help, all things considered.

Don't forget 3DS is getting Shin Megami Tensei 4 as well. I agree with you 100% though, if we start to see Atlus and Square Enix support the 3DS more, over the PSV then PSV could be doomed in Japan. This really depends on how well the next major titles from each platform does on the 3DS. If Dragon Quest VII does as well as DQIX or even just better then DQVI, I think Sony will stick with Nintendo, same with Capcom if MH4 does well, and same with Atlus if SMT4 does well.

Next year 3DS is going to have an awesome year with all the 3rd party support it has, if PSV can't keep up, it may just crash or stay niche (just like GCN did compared to PS2 despite some excellent first party support).

I think people that self-describe themselves as locked in a hardcore gamer box don't really care about anything outside of that box.

So while the 3DS is and will continue to be successful, it will be so with gamers and not "hardcore gamers"

I probably play 30-50 games each year and I would not call myself a "hardcore gamer"... that term to me is either too limiting or I don't understand how it is defined.