I predict that the next three home consoles will grow from their predecessors but that the 3DS and Vita will sell less than the DS and PSP (respectively).
The Wii U will benefit from its tablet controller, HD graphics, and a (hopefully) more comprehensive online system. It will appeal to the tablet gaming market and bring back some of the more hardcore gaming market that the Wii lost in its last couple years. I expect it to increase significantly in sales over the Wii like the PS2 did to the PS1.
The PS4 and Xbox 720 will both benefit from the current momentum of their predecessors. While the PS3 and 360 are selling well now, they both had relatively sluggish starts. The 360 had to built up its brand name strength practically from scratch, and the PS3 was hurt by a high price point and lack of early killer apps. However, both the PS4 and 720 will likely launch to reasonable price points, have established online networks, and a strong foundation of western third party support.
As for the 3DS and the Vita, I don't think they will sell less than their respective predecessors due to smartphone competition. It will be because Nintendo (and particularly Sony) will fail to release the mass market games necessary to expand their userbases. With the 3DS, Nintendo's early killer apps seem to be Mario titles. While this is fantastic for gamers, they won't bring the necessary expanded audience to build on the DS's success. Same for Uncharted and Call of Duty on the Vita. The DS was primarily successful because of titles like Nintendogs and Brain Training. The expanded audience gained from these games then poured over into traditional Nintendo titles like Super Mario Bros. and Mario Kart. However, if Nintendo relies on just these Mario series for the 3DS, they will carry over this expanded audience they have already earned but will fail to expand it further (as well as lose some gamers who have moved on to other systems or naturally won't be repeat customers).
The 3DS could still have very impressive sales and easily crack 100 million, but I don't see it selling more than the DS. I expect similar results for the Vita. To be fair though, I expect both systems to be very successful sales-wise and financially. Just that they won't beat their predecessors.
Overall, I expect the 8th generation to be larger than the 7th. But I expect most of the growth to come from home consoles.
