alright, im seeing a lot of "when it doesnt fit in your pocket" responses... ahem...
my sweatshirt has a front pocket that could my New 3DS XL, a 7.5-inch tablet, and my 5-inch phone with room to spare. so dont use pockets to determine appropriate phone sizes.
now for a serious answer, we need to remember that phones are just that, PHONES. meaning we should be able to hold them in one hand, and make calls using just one hard. if youre holding a phone in your hand, and your thumb cannot reach to the other side of the screen easily, you have gone too far