Alot of people, I believe did complain, and were fusterated with Nintendo's total incompotence during the N64 generation. Nintendo had good momentum with the SNES being the best system ever, but totally squandered every single advantage they ever had, and because of their mistakes, is why I've (personally) never bought a Nintendo product since.
The N64 first party, and second party IPs were the best of the generation, IMO, even the PSX games weren't as good, on whole. There were indeed a few great PSX games, but again, I never felt they were as good.
Problem was, I got very fusterated with Nintendo's incompotence with getting the great 3rd party games on their system - Tomb Raider, Final Fantasy, Metal Gear, and all of the rest. Nintendo really botched that generation worse than any others combined.
As for sales, and the hope that Nintendo bringing big game X onto the scene would help it out - I really didn't care. All I was praying for was more than 1 decent RPG on the N64, and that never happened as long as I cared for the N64. About 3 years in, I dropped my support for the console entirely, in favor of the PC.