Any long running series where the games themselves are different will run into this. It's the same with Final Fantasy. Not closely following a formula means you end up with a fractured fanbase, where one group will love Game A and hate Game B, while another group will love Game B but hate Game C, and so on.
Personally, in terms of business decisions, I think reoccurring IP should stick to formulas and new ideas should go to new IP.