Banjo Kazooie for me. Absolutely love that game even above Mario 64. Goldeneye/Perfect Dark both round up my top 5 too. Rare were so damn good back then!

Perfect Dark was the best technical show piece for the console.

It was also a banging game with a great story.

Ogre Batlle 64 for me, and its not close.

In fact, Mario64 is not even in my top15, regardless of how iconic/relevant you might think it is, I always found it to be a very mid ugly-looking game, even by the time it came out. 

Uha, tough one. I think it doesn't hold up today but for me it will have to be Conker's Bad Fur Day, just because of what it was at the time- after years and years of getting mocked by the Pro-Playstation crowd for the family friendly games on Nintendo64, Rare's swan song started with Conker cutting the N64 logo in two with a chainsaw and didn't stop for the rest of the game with splatter, feces humor and sexual innuendo- with more voice acting than 98% of other N64 games combined. Just what the doctor ordered at that time. Other mighty honorable mentions: Diddy Kong Racing (oh yes, it was better than Mariokart64), Goldeneye, Perfect Dark, Super Smash Bros and Mario Tennis.

Single player: Resident Evil 2
Multiplayer: WWE No Mercy

Everything Rare made for the system. Goldeneye for multiplayer and Blast Corps for single player. I had so much fun with my neighbor friends playing through Blast Corps several times. We would beat a level and than passed the controller to someone else to beat the next level.

Time to get moving!

Such a unique game, not seen anything similar since.
Another unique one no one has mentioned yet: Pilotwings.
Both of these I would love to see a modern version (or if anyone can reccomend anything similar)

For the original question, I think I would rate things differently then vs now as some games age better than others (i didn't play SM64 until Wii VC version but apart from graphically I think it held up well)
So of the games I played on the N64, after OoT I think I would probably give the top spot to either Goldeneye, Perfect Dark or F-Zero X.

And if i were to rate them now I think F-Zero X would be my top rated game above OoT/SM64 anyway as racing games I think just hold up better. (although I had a lot of fun playing MK64 with my sister, it's my least favourite of the Mario Karts I have played)

I think both the PS1 and N64 are hard to go back to and enjoy the games on their own merits so if you weren’t from that era and playing those games it can be a bit jarring to try to recapture the magic of those times. N64 wasn’t my preferred console outside of the games you’ve scratched off the list.

That means I’ve got nothing. I’ve tried Banjo on the Switch Online and I just can’t get into it. Same with Goldeneye. I was exactly the wrong age for this console and was very thankful Sony came along to widen the scope of gaming at the time.

I guess I’d have to pick Paper Mario, though the Thousand Year Door makes it feel like an artist’s concept. At the time I was upset that we didn’t get Metroid 64, but in retrospect I think it would have looked like a nightmare. I’m sure they were prototyping something at the time. Would love to see what they were working on.

Majora's Mask or Banjo.

Wave Race 64!

The system would not have been the same without this impressive water physics engine. It felt and looked real. Incredibly fun to get so in flow with it that you could bounce out of the water with extra gusto and feel the flight and landing of physical propulsion from the natural mechanix