3gb vs 5.5gb (ram usable for games) is what's going to kill Switch's chances of getting games like Witcher 3 or the next Elderscrolls/Fallout, regardless of the GPU/CPU deficit.
There is a big difference in power, and when a big third party title launches on both Switch and PS4 the difference will be very apparent.
You pay a price for portability.