P3:FES > P4 > P2:IS > P2:EP > P1
It's a tough choice between Persona 3 and 4. Each has their strengths and weaknesses, but both are fantastic games.
IMO, P3 has a more interesting story, better atmosphere, better "monthly" boss battles, it's the one that redefined the series and has Mitsuru <3. Also, Aigis is possibly the best character in the series. However, the dungeon crawling was pretty bland especially since you were exploring the same dungeon throughout the game, and it did have serious pacing issues. It also had Fuuka and her annoying voice.
Persona 4 fixed mostly all the gameplay complaints from P3, had better characters, superior soundtrack, and more memorable & hilarious moments. With that said, the dungeon crawling was still bland (but was given different looking dungeons at least), had a copy-paste scenario for half the game's story, and the boss battles were just boring. There was no strategy involved; it was just attack them with your strongest skills.
In the end, I go with Persona 3. Maybe I'm biased since I played it before P4, and it was such a refreshing experience. It holds a special place in my heart.
