I find the opposite is true.
In fact, people who often buy games that are deemed 'casual' also buy other types of games, such as RPGs, shooters or sports games. Whereas people who consider themselves 'hardcore' gamers are more likely to keep from buying a game if they deem it is 'casual'.
Plus, its usually those people who consider themselves 'hardcore' gamers who call a game 'casual' in the first place. People playing a game like Nintendogs or NSMB don't even make a distinction between games aside from their genre.











