Well there's your problem. How can you expect a fanbase to grow and a game to sell better if it's only aimed at fans of one game in the series and doesn't try to market to anyone beyond that? Maybe they were going for word of mouth spread, in which case it worked a little, because I'd never heard of the series before 4, now I have two games from it.
That is why is Valkryia Azure Revolution was made. That was an attempt to expand the fanbase.