I didn't know that Goldeneye introduced headshots doing more damage, or zooming in with a sniper rifle.

I think a reason I liked it so much is also due to how you describe it not being a dungeon-crawler. Even though I never considered it at the time, the stages were pretty unusal for an FPS. Some of Perfect Dark's stages as well.

I would argue Diddy Kong Racing is on par with most of the top ones mentioned (I myself used to prefer it to MK64, because of the much much better campaign). Man, 64 had great racers!

Episode one racing.

F Zero no question. I slightly prefer wipouts these days cause of Wipeout in VR and the amazing Omega collection but I still think of F Zero.

Oh and Perfect Dark even though I preferred the Game Boy colour game m9re for some reason. 

If I had to pick a runner up, it'd be a close fight between F-Zero X and Lylat Wars/Starfox 64, the former for being so intense and oozing cool, and the latter for its epic space battles, stellar (heh) soundtrack, quotable VA, and replayability thanks to branching paths.

I will not recommend Ocarina for a newbie or a rockie in N64 games. Aged not well.
But I will recommend SM64 all day along. That game aged very very well.

Anyways, I can't say only one game, N64 have LOTS of amazing games, many of them exclusives, and many people do not even know some gems,

If you like Racing games, you have a HUGE quantity of games, from very serious with official licenses, to good games without licenses, to arcade type games, to Mario Kart types.

I will recommend some games I do not normally see in the lists:

-F-1 World Grand Prix: An amazing experience, with all the official 97 season of F-1, including all the 17 circuits around the world perfectly reproduced, the 22 drivers (excluding the prick of Jacques Villenevue, who also did not like to give his rights to apper in Sony's F1 games either), and even scenarios to play yourself the best moments of the year. That game is fully official: from "Tag Heuer" times, to Goodyear and Bridgestone tyres for every team, their logos are seen in the actual tyres of the cars (well, everything official... but tobacco brands, like Malboro, because yes, by 1997 that was already unacceptable in a videogame! XD). You also have F-1 WGP II for N64, with full season 98, but only released in PAL markets.
The game is a very serious simulator, but you can also play it as a total casual game with easy difficulty, automatic changes, no damages, no weather changes, no mechanic preparations, no classification laps, and no need to go to pit to refuel... if you want. A lot of characteristics to use it... or not. As you wish.
You also can activate or deactivate computer helps to make the car brake, with no inputs from you, before any important curve.

SO, there is game for literally EVERYONE. Also, the game has a secret circuit totally made-up in Hawaii (with part of it inside a volcano) and two "cheat" drivers with insane velocities (one of them is so fast, it becomes literally non-useful, because is instantly crashing everywhere).

This is probably the best serious racing game in all that generation. And yes, i'm including Gran Turismos, because those games suck your soul and your life time demanding lots of licenses, making money... before getting a fucking interesting supercar. And they do not even have damages. You can literally make apart your wheels or your wings in this F-1 WGP, if you want. I LOVED THIS GAME. IT WAS AMAZING. You felt like in a real Grand Prix, also with the radio constantly informing you about the course ongoing: your times, the positions of your competitors, if anyone retired... 

Those 2  F1 WGP games were so good because their creators (only in N64 versions, the originals) were Paradigm, people specialized in SIMULATIONS, the same guys of another game I recommend now:


-Pilotwings 64: Much more known, this simulation game was the game everyone overlooked because everyone got SM64. This game is full of little details and surprises if you try to simply investigate the maps. Especially, the Mini-USA map. Loved. Was like flying or driving in GTAV... but many years before. One of the first games for N64, and a total masterpiece.

-Mace the Dark Age: You want a classic 1vs1 3D Fighting game? got that, only for N64 and arcades. It is hard as fuck, like a classic Mortal Kombat game. Characters use arms, so is not a Tekken, but a Soul Edge kind of a game.

-ISS64/98/2000: best soccer games ever, by Konami, very arcade, very fun.

-Hybrid Heaven: Do you like RPGs turn-based? Do you like fights? Well... here you have an RPG fighting game with some misterious story of alien invasion behind, also by Konami, and some action between.

-Castlevania 64/Legacy of Darkness. After more than 25 years of totally undeserved campaings of hate and ridiculizations, NOW, finally, seems some people started to defend these games, at last. Maybe because people matured and stopped to follow idiots full of crap and deception. MAYBE BECAUSE THIS IS THE MOST REAL CLASSIC CASTLEVANIA EXPERIENCE, IN 3D, EVER EXISTED. Or maybe both. Yes, it is difficult, and yes sometimes is frustrating, but FUCK, that is EXACTLY what 2D Castlevanias ALSO supposedly are!!! instead of boring non-stop fightings and stupid movie-like games, THOSE ARE 2D platforming Castlevanias... in 3D. You fall from the platform? FUCK YOU. Exactly as people expected by then. And they are good. Fuck Alucard mythomania and PSX-fanboy frightened jealousy, desguised as "purity, the Castlevania have to be 2D because sanctity" and other stupid ideas. Perhaps people now, after many years of Konami fucking up the 2D series, can finally give them a chance. 

-V-Rally 99: You don't like F1 but you like Rallies? well, here you have a bunch of european and japanese licensed REAL cars, running in a bunch of races inspired on the World Rally Championship. The most realistic rally car game for N64.

And of course, WaveRace 64, Forsaken, Star Wars Rogue Squadron, Turok series, Shadowman, Goemon 64, Smash Bros, Perfect Dark, Banjo Kazooie, Body Harvest, Star Fox 64, 1080º, Conker BFD, Star Wars Episode 1 Racer (which i see someone finally said it), World Driver Championship, Ogree Battle 64 (the 7th episode in the main saga, I think) ... and many many more.

Ohhh yes, how could I forget that one!? And F1 World Grand Prix, as someone mentioned here... It was truly an extraordinary racing catalogue (with others like Excitebike, V-Rally 99, Xtreme G or Star Wars Episode 1 Racer being also relevant at that time  =)  ).