Well now you're speaking subjectively. And your opinion is not shared with the majority. Of those games you listed, they are all still considered high quality by fans and critics. (Except for Madden but that doesn't have any competition so people have to buy it to play NFL football). The people that disagree are in the minority.. These people probably aren't even fans to begin with. I think you're referring to Internet fans rather than the global populace of fans. If most players expected change, then those games would not be so successful. That's not the case. People want the same core. It's as simple as that.
Like you said, nothing is set in stone. You have your Halo's and you have your Dynasty Warriors. Both games stayed similar to their core, but have two different outcomes. Also, You have your Dragon Age 2's and you have your Resident Evils. Both games changed, but have two different outcomes. Changing a games's core doesn't ensure success or failure. There are obviously bigger factors at work here.