With a CD-ROM format, Nintendo would likely have kept Square and Enix exclusive. That would have killed the PS1 in Japan. The Saturn outsold the PS1 during the two systems' first year in Japan. Final Fantasy VII was what turned the tables for Sony there, and at the time, it was a foregone conclusion that it was going to be a N64 exclusive. Had Square stayed, Enix and Dragon Quest VII would have stayed, and that would have sealed the PS1's fate in Japan. Nintendo might have been able to get games like Mother 3 N64 completed. Capcom would have likely developed Resident Evil cross-platform for the N64 launch. The only thing Sony had going for it early on in Japan was Tekken.
Sony would have done better in the West with a lot of support from EA, and they probably would have been able to secure at least time-limited exclusivity for Tomb Raider. They would have still dominated Europe. Nintendo would have eventually come out the winner in the United States, though, with a double whammy of Final Fantasy VII and Goldeneye for the holiday 1997 season and Zelda the next year. Globally, the N64 would have walloped the PS1 with the N64 winning Asia and North America and the PS1 winning only in Europe.
In this scenario, Sega might have done somewhat better, but they also might have done even worse, to the point where they called it quits as a hardware market before they even tried to get the Dreamcast to market. And then there's the question of how the Xbox would have fared against a weaker PS2 and a stronger Gamecube (or equivalent, since a 5th gen win would likely have set Nintendo on a different course in 6th gen.)