Well, think about it like this. It's WATSON powerful? Well, it's powerful enough to win at jeopardy (and that's off-line.) So more powerful than that.
Also, in your original post you are forgetting about compression. You wouldn't need 5mb/s to send compressed game data - you could use much, much less than that. You could send the basic structure, and where all the light is bouncing around, then the local computer/X1 would do the rest with it's GPU.
Think about all the data that Netflix is streaming off Amazon servers.







