But will prior sales analysis help here. There has never been a out of the box hybrid game system before. What prior sales data do you compare it to?
Sales data always helps as long as it's interpreted correctly. The most important thing is to not limit a sales analysis to only the previous generation, but rather take the entirety of video game history into account and look for factors and circumstances that apply to an upcoming system. Another important thing is that not all console manufacturers are the same, so applying the PlayStation model for analyses to Nintendo is absolutely nonsensical.
Of course you are right that a simple 1:1 analysis with any previous system does not work with Switch because there is no previous console that fits the bill. Breaking down key points for success as well as failure in the past and looking how Switch matches up with them is the best you can do, but you still have an unknown variable because Switch is something new. Underserved and unaddressed markets add to the variables and at times you don't even know what kind of demand could be created; here the first question is if Switch can sell to people beyond Wii U and 3DS owners, and since Switch has positive traits that were either not present or only there in limited form on Wii U and 3DS, the answer for me is a clear yes. However, quantifying Switch's new ceiling for lifetime sales isn't anywhere close to as easy.
What's also in Switch's favor is that Nintendo is making serious efforts to increase their mindshare in the general public, so Nintendo's strategy goes beyond the console alone. An important question is where does Nintendo see the future? They obviously don't believe that a console should be limited to the TV anymore, and that's actually in line with human behavior where we see a shift away from the TV as the absolutely central place for entertainment consumption. Movies and TV shows can be watched on smart devices; smart devices can be carried around and enjoyed anywhere you want (although an internet connection is a necessity for many applications). Success is more likely for companies who adjust to human nature and behavior than those companies who want humans adjust to them.