First off I'm going to say thanks to Shadow1980 for the following charts from his NPD post, but I think they illustrate what an amazing first year the N64 had, and I've always maintained this -- the N64 should've won that generation. The N64 is a unique system from a sales POV IMO it's maybe the only so-called "failed" system that actually was really actually popular with consumers.
It was like a baby destined for great things that was crippled by a terrible mistake made by its parents. Had Nintendo compromised on the CD situation (even made a CD drive in lieu of that stupid 64DD that went nowhere) it would've won that generation and potentially sold over 100 million consoles. I've always felt this and this has always been to me as a Nintendo fan "the generation that got away". You could have had either a CD add-on or kept the cartridge slot and had a CD drive. Either way would've worked.
To illustrate this I present the following two charts:
Now is Playstation sales. You can see in the US it wasn't anything special until late 1997 when Final Fantasy VII comes along. A game that it likely would not have had if Nintendo had just compromised on the CD issue. In fact you can see the PSX was quite unimpressive prior to that point selling like a Wii U for certain stretches. In Japan it was even losing to the Sega Saturn for until 1997 I believe. Squaresoft and FFVII basically turned that entire generation in Sony's favor.
Second, lets look at actual N64 sales:
Look at those freaking N64 sales for year 1 in yellow! That is amazing when you consider how much smaller the industry was in 1996/97, it's first year in the US arguably may be more impressive than even the Wii and PS4 considering the industry size difference. And the N64 did that with droughts too ... poor thing was starved for games for months at a time. Like literally one game a month was a good month for the N64.
After all those game droughts and a lack of support titles the N64 finally succumbed to lower sales later in life, but IMO this is all such a shame it had to finish like that.
So take a bow you sexy Silicon Graphics beast, I'll always remember you fondly, Nintendo 64. You were robbed, little buddy.