I dont know how the PS4 works but it streams flawlessly into the Vita aswell. Pretty good techs.
I experienced lag on the Vita. Played Tomb Raider in my livingroom (which admittedly is further than the Wii U can reach) and the voices were out of sync and it would give a connection notification. The screen would glitch, too. I don't know if my Vita was connected via the systems own wireless or the internet, though. Still was pretty amazing seeing those visuals on a handheld console!
Vita Remote Play was actually one of the big things that swayed me to the PS4 instead of the Xbox One last month.
Did you choose in the option to stream from your PS4, or were you using the wireless?
I did the same thing and i had no problems, but maybe your living room is farther away from your room than mine.
Can't remember which setting I used. I just select "PS4 Link" and then "Remote Play" and I go from there. But, in Vita's defense, I haven't played Remote Play in the same room with my PS4 yet (which is almost the only option for my Wii U--same room or one room/bathroom away) so the Vita connection will probably be flawless in the same room as the console.