Both will do well this holiday PSVita has one of best launch line up in videogame history so it will do well but 3DS will do better because of bigs systhemsellers.
PS4 - over 100 millions let's say 120m
Xbox One - 70m
Wii U - 25m
Vita - 15m if it will not get Final Fantasy Kingdoms Heart and Monster Hunter 20m otherwise
3DS - 80m