Yep. Devil May Cry 3 was way too hard when I was a child. I tried it hard, but it was beyond my reach at that time.

What comes to mind is quitting on Granstream Saga.

A long time ago though, getting to the final boss much too easy and carrying absolutely no items to heal myself or restore things (mana?) at a point of no return!!!
I would have had to use an older save that would make me play like half the game again.... screw that!

Ghost and Goblins for the NES can burn in a fire. Still can't beat that game.

Battletoads is another game that I never been able to beat due to the finite continues and brutal platforming/vehicle sections.

I remember grindin a lot on Grandia before the final boss but I still couldn't beat him and I was so bored of the battle.

Also, Chrono Cross, I was at the final dungeon, i remember I had to kill some color guardians but I couldn't so i put off the game for a couple of months, when I returned I discovered my save file was GONE!

And the more recent dificult game I rage quit was Resonance of Fate, In another stage of my life I would still play that game, but now I don't have much time, and really don't want to spend that little time losing and dying :/

One word "Ikaruga".

I felt folklore was pretty difficult at the end and one other boss battle. Whereas Star Ocean 4, I was just giving the OP some tips to take out the final boss; it took me over an hour of continuous gameplay to kill that boss, because of the stupid save system.

Well, it was more of a forced abandonment back in the days of video game rentals. The game was too hard before we had to bring it back. That game was Battletoads for the NES

1) Dark Souls - Didn't like it as much as Demon Souls so couldn't bring myself to infinite deaths
2) NES Games
3) Persona 1 (PSP)
4) Chrono Cross - Originally couldn't beat the seargent boss who was white colour as I didn't take time to set up colour attacks on ym characters. A few years ago I replayed and spanked him and beat the game
5) World at War: Vetern. Beat normal but gave up on 4 levels on Vetern as it was just too difficult.

6) Last Remnant: Awful Combat System so youtubed the game

PS2 Games: Radiata Stories, Deus Ex, Dark Cloud 1/2, Rogue Galaxy, Dragon Quest 8 = Couldn't do the grind now so I youtubed the games haha

Couldn't beat final bosses so youtubed ending:

1) Blue Dragon

2) Lost Odyssey

3) Infinite Undiscovery

yes on some old games back in the day,when there was no continues and saves and easy levels. I remember playing double dragon/battletoad game back in the game,and gave up because,once you ran out of continues you had to start all over. I could never beat it,so I gave up.

The only game in recent memory that I've considered quitting was Touhou: Embodiment of Scarlet Devil. Although it was my first foray into the bullet hell genre.