The N64 might have had the greatest first party lineup of any Nintendo system if you count Rare as first party. What it didn't have was 3rd party support.

In terms of sheer quality, Absolutely. But as far as quantity and variety is concerned, it's easily one of Nintendo's worst showings. Take Rare out of the equation, and you mostly have Mario games, SNES sequels, and a few sports games. Sure Super Mario 64 and OoT were revolutionary, but Nintendo's overall N64 lineup was lacking in variety and quantity compared to their other systems.

If you're going to put in the arbitrary stipulation of "you can't count Rare" for whatever reason, then things like Retro Studios shouldn't be counted for the Wii. 

Super Mario 64

Zelda: Ocarina of Time

Zelda: Majora's Mask

Star Fox 64

Animal Forest (Crossing)

Pokemon Snap

Super Smash Bros. 

F-Zero X

Wave Race 64

Pilotwings 64

Mario Kart 64

Yoshi's Story

Mario Tennis

Mario Golf

Paper Mario

Sin & Punishment

Kirby 64

1080 Snowboarding

Is a pretty diverse set of  games anyway. Then you factor in Rare + other Nintendo 2nd party projects you can add Blast Corps, GoldenEye 007, Jet Force Gemini, Banjo-Kazooie, Starcraft 64, Star Wars: Shadows of the Empire, Star Wars: Rogue Squadron, Killer Instinct Gold, Donkey Kong 64, Diddy Kong Racing, Cruis'n' USA, Command & Conquer 64, NBA Courtside, Ken Griffey Jr. MLB, Ridge Racer 64, Excitebike 64, Pokemon Puzzle League, Tetrisphere, Perfect Dark, Conker's Bad Fur Day.

If you're talking 1st + 2nd party projects, the N64 is quite diverse. 

They have Pikmin games, except the Virtual Boy.

That list actually proves my point. Rare doesn't count because they actually self-published most of their N64 catalog. It's not like Retro, which is not only had Nintendo publish all their games directly, but is also a wholly owned subsidiary of Nintendo. It's not even like Intelligent Systems, where all their games are property of Nintendo. There's a reason why 90% of Rare's IP now belongs to Microsoft.

Getting back on topic, most of that list is Mario and Mario spin-offs, or cute mascot platformers. With 2 sequels to SNES games, a Snowboarding game, and one truly unique game with Sin & Punishement, Animal Crossing is more well known as a GameCube game, as it was the west's first introduction to the series, so I'm also not counting it.

N64 was the only one I liked, loved the rogue squadron games.

Rare should count, Nintendo financed basically all their games during that time and marketed all of them. They were a 2nd party studio for all intents and purposes. Nintendo did btw publish Blast Corps, Killer Instinct Gold, GoldenEye 007, Banjo-Kazooie, Banjo-Tooie, Mickey Speedway USA, Donkey Kong 64, the deal with MS had nothing to do with Nintendo publishing certain games or not. 

The N64 wouldn't have sold 2x better if Nintendo published some weird niche "unique" games, the problem was not the 1st/2nd party output which quite frankly was spectacular and arguably better than GameCube or Wii. The problem was the cartridge only solution cut the legs off the system before it ever had a chance. 

Take all the 3rd party support from the NES or SNES and give it basically all to Sega and the Master System and Genesis beat the NES and SNES and I doubt either the NES or SNES sells even 33 million units that the N64 got to. 

Maybe the SNES. In the US, NES built the market. Super Mario Bros. 3 is one of the few games in video game history that has delivered the hype it promised. I believe third party support on a Nintendo console hurts a console's longevity more than its sales. The case of Wii when Nintendo abandoned the console plummeted in sales.

They all have one thing in common: They didn't sell well. :(

Rare almost singlehandedly carried the N64 and basically solidified them as my favorite developer outside Nintendo in the 90s. Without them the library would have been a Wii U-esque dessert. Rare was actually the initial reason I made the jump to the Xbox line when 360 came out. 

Of course, most Xbox fans know it didn't really work out (although I did somewhat enjoy Kameo and even PDZ wasn't horrible, was sort of their last gasp for me). Regardless, got super into Gears, Bethesda games, and other random third parties on 360 anyway and never really looked back. 

But anyway yeah ultimately much of it just boils down to the quantity, and the relatively lack of appeal of many of the games that WERE there. Rare was fantastic but they were largely carrying the 2nd/3rd party weight themselves.