From a game-play point of view, Nintendo made the right decision to go with cartridges in the 5th gen.
Back then, discs were unreliable and the loading times were atrocious. Many PS1 games for me were ruined (both figuratively and literally) because of it. It made the N64 a much more reliable system over the PS1. Didn't have a Saturn but I guess it could compare to PS1. The N64 also didn't have the hassle of memory cards for the most part. Yes, you had the controller-pack which certain games needed, but the big games all had memory within the cartridge. N64 games were just easier to pick up and play and they are in a way better state today than my PS1 games are.
From a business point of view though, they should've went with CD's.
Because of the ease of programming, the amount of data capable of being held on a disc as well as the very low costs would've kept a lot of key third parties in Nintendo's fold. I'm not saying N64 would've won the generation over the PlayStation, but it would've been a lot closer if developers like Square still supported Nintendo. Also, the games being a little cheaper at retail were they on discs would've also made things more competitive, most N64 games back then were more expensive than a lot of games today!
Going further back though, they shouldn't have bullied Sony during the development of the SNES-CD and should have went for this deal in the first place. They created their own worst enemy.