I don't know about the first 2 since I don't play TPS that much and I never really had a problem with the cover system or camera placement of the few that I did play.
But as for FPS I believe ADS is used to make the experience more realistic. In real life you don't just hip fire a gun and hit a dude 5 miles away lol. But keeping realism aside I think it more lends to the gameplay when you have low health. With the health that you have in most modern shooters, it would simply be game breaking if you could hip fire a weapon and take people out 10 miles away with a few shots. ADS helps mitigate that with increased recoil and slowing your character down to take people out long range while still allowing you to hip fire at close range.
This is something that you can see in Halo, the assault rifle is pretty useless compared to the other guns (like battle rifle or whatever that 3 round burst weapon is called) because you can only hit people close range with it. If it had ADS a lot more people would use it since they could hit people long range with it. Not that it matters in Halo anyway because it takes one trillion shots to down a guy which I personally dislike), so hip firing or ADSing doesn't even matter.
I think the mobility is somewhat of a downside one which was fixed by the stalker perk in MW3, I run it all the time and wouldn't mind it being default. In the end I'm up for whatever works, and ADSing works for me and for most people.
