Ghirahim's hand stops following you when he stands in front of you. Wait for him to shift his stance - when he plants his feet, that means his hand is slower, and you can hit him easily from the opposite side without him grabbing your sword.
I can sort of understand why some people don't get that, but Ghirahim doesn't actually attack you in that mode, so there's every reason to try to feel him out a bit before flailing at him.