Fi never really annoyed me that much personally, she was just kinda there. Unnecessary half the time, but not bad enough to substantially reduce my enjoyment.
Skyward Sword itself is awesome, I feel no need to get the Switch version as I already have in on Wii, but it's one of the top ten games on the console for me thanks to its amazing dungeons and fun motion-powered swordplay. It definitely has its flaws, but its core design and warm heart outshine them.