Well there's your problem. How can you expect a fanbase to grow and a game to sell better if it's only aimed at fans of one game in the series and doesn't try to market to anyone beyond that? Maybe they were going for word of mouth spread, in which case it worked a little, because I'd never heard of the series before 4, now I have two games from it.
That is why is Valkryia Azure Revolution was made. That was an attempt to expand the fanbase.
Clearly if I didn't hear about the game meant for VC1 fans, but grew to like it and VC1, there is still more growth to be had and they should have marketed it better.