I experienced lag on the Vita. Played Tomb Raider in my livingroom (which admittedly is further than the Wii U can reach) and the voices were out of sync and it would give a connection notification. The screen would glitch, too. I don't know if my Vita was connected via the systems own wireless or the internet, though. Still was pretty amazing seeing those visuals on a handheld console!
Vita Remote Play was actually one of the big things that swayed me to the PS4 instead of the Xbox One last month.
Did you choose in the option to stream from your PS4, or were you using the wireless?
I did the same thing and i had no problems, but maybe your living room is farther away from your room than mine.