Keep in mind that there are two Persona 2 games. Innocent Sin and Eternal punishment. Innocent Sin wasn't translated into English until 2011 (12 years after it's original release) when it was remade for PSP.

Anyway, you can jump into Persona 5 as your first one if you want. Or whichever you like. They're all pretty stand alone like FF games, with the exception of the two Persona 2 games.

Yeah, on second thought I would also recommend starting with P3, as the former games were pretty different in that regard. Something that made P3 so popular was the inclusion of Social Links.

But I'd say start with Persona 3: FES.
Not Persona 3 or Persona 3 Portable.

Start with 3, the ones before it still didn't have the Social Links and even as they do resemble the ones after 3, from 3 onwards is where it took the clear direction it have nowdays. (there are some minor references you'll miss, but that's about it, you are good to start with P3 and going through P4 and P5, if you can't get enough of it you can go back to 1 and 2 to get some extra time with your Personas xP)

That's fine. But in case you feel that playing 1 or 2 somehow burns you out from playing the rest (which can happen even if you're enjoying the games), know that P3 and up are vastly different, and worth trying out in their own right.

As for FES, it's basically Persona 3, with extra content. New events, scenes, etc. As well as a bonus Epilogue game that takes place after the ending, which is roughly 20 hours long.
The issue with Portable is that it uses a top down view and a point and click system rather than actual 3D graphics where you get to control your character walking around town and school, etc. It only has 3D graphics while in the dungeons I believe.
On the other hand Portable has a female main protagonist. SO that can be worth checking out. But P3: FES is definitely the best version.

I do think a lot of how Persona 5 is going to be remembered in the history of gaming won't be known until retrospective reviews about a year down the line.

Retrospectively, the original Nier was horribly underrated, while Bioshock: Infinite and especially Undertale have seem to have lost some of their initial luster. Bloodborne and The Witcher 3 (while highly rated at the time) have both gotten significantly more praise in the two years since they were released. MGS: V has lost a lot of its initial enthusiasm, whereas MGS3 (which was originally one of the lesser reviewed MGS's) has significantly improved in how people perceive it.

Simply put, when judging the GOAT, you can't just judge the game based upon initial impressions. People may still ultimately see Persona 4 as a more daring game than Persona 5 and retroactively prefer Persona 4. We'll see.

Yeah, P3 is the best starting point for the current feel of Persona.  It's the template for the series, now, and a damn good game, even if it's not as polished mechanically as P4 (and, I assume, P5).  

On topic, I'm kind of torn.  Should I go ahead and buy P5 for the PS3?  Or wait until I get a PS4 (perhaps later this year).  Hm.