I don't think it would have it another crash, but it definitely wouldn't have grown to what it is today. I mean, if you look at the growth from the 3rd gen to 4th gen, there are signs that growth was very slow, maybe even barely any.
In the 3rd gen, we had the NES (62M), SMS (13M), Atari 7200 (4M), and SG-1000 (couldn't find hard numbers, so let's go with 1M), which sold 80M combined. The market leader, the NES, sold 62M units. In the 4th gen, we had the SNES (49M), Genesis (40M), TurboGrafx (10M), and Neo Geo and CDi (1M combined), selling 100M combined. That shows a 20M growth in overal numbers, however, given that the market leader actually dropped 13M, I think it's safe to assume we have quite a few crossover buyers (those who bought more than one system.)
Now, let's look at the incredible growth during the 5th gen. We had the PS1 (103M), N64 (33M), Sega Saturn (10M), and 3DO (2M) selling close to 150M. That's not only about a 50M growth, but the market leader also grew 54M and sold more than all consoles from the previous gen combined. The 2nd outing of the PS saw even more growth. We had the PS2 (158M), Xbox (24M), GC (22M), and DC (11M) selling 215M. That's another 65M in growth, with the market leader once again selling more than the last gen combined. Now, after the PS2 expanded the market incredibly, I think we have sort of hit a saturation point for how fast the market can grow.
This last gen, we did have the Wii (102M), PS3 (90M+), 360 (88M+), which will go on to sell 280M+. Of course, last gen has to be put into perspective, since we saw an artificial inflated growth thanks to a large number of casuals who joined, but have since jumped onto phone/tablet gaming. And because of Sony's poor decisions before/at launch, Sony and MS pretty much split the core market in half, with Nintendo taking a few. Still, I think we did see some legitimate growth. As far as this current gen goes, I think we will see a "shrinking" in the market. Of course, it won't exactly be a true shrinkage, since it's mostly due to casuals exiting the market. I think we will still see the market grow slightly over the 6th gen. Probably 135M for PS4, 65M for XBO, and 21M for Wii U.