Wii Remote Problem

[TWRoO]

Assuming you can do so... you need to properly sync the remote with the console (as ringo says)
If you have done that and it is still doing this then I don't think anyone here can help unless they had the same problem. But you should still be able to swap players as Pyro said (but the Wii has to be on already)