what's wrong with playing a game on casual/easy mode?

[Never]

If the games an interactive movie the why not.

I played bioshock infinite on normal but would it have been better on easy?

The plot and world creation would have lost nothing and that's the games strong point. I found the combat on normal ok but generic. Maybe even dragging at some points, but also very easy and forgiving. If I made it a little easier and faster to get through maybe it would have dragged less... or maybe the reduction of challenge would have made it more tedious. It could go either way.

Games that are about gameplay rather than plot need to be at the right difficulty for your skill level.

Mario 3d land was too easy for me at first and felt like effortlessly running through an obstacle course. At first I was disappointed with the game. However, on rescuing the princess and unlocking the special (Hard) worlds the game really became engaging. That was when the difficulty became the right level to test my skill. I might have died plenty of time in the normal worlds but only because it was so easy that I was carelessly running through the game without paying attention.