It's Ganon from Ocarina of Time, no questions asked.
No, he wasn't particularly hard, but there was something about him that terrified even as it thrilled. I think the biggest part of it is that you never get a very good look at him: the battlefield is damnably dark, and the only luminescent parts of him are his eyes, his sabers, and his tail, which you don't see for most of the fight. He's only fully illuminated by flashes of lightning, which throw him into stark relief so briefly that you never get much more than an impression of what he actually looks like.
It made him much more impressive - and terrifying - than he would have been in a well-lit arena. I wish they'd use that trick again. It worked beautifully, bringing to mind the terror of fighting Ganon in the dark in the original Legend of Zelda, or even in Link to the Past.
And yeah, I'll admit it. When Navi says, "He won't hold me back again! This time, we fight together!" I get goosebumps.