Yes, but compared to XB360 Wii added little more RAM and while the driver itself for motion is surely small, some memory will be consumed by the game code that actually uses it giving a meaning in the game to the motion detected... In the end Wii has little more RAM but it uses it very well for things that people notice more than graphics, once these have reached an acceptable level and even exceeded SD TV's capabilities, increasing graphics performance would require a lot more RAM, not for the resolution itself, but particularly if increasing effects, textures size, antialiasing level, etc (as those 32MB of the first GeForce, for example, were enough for 1024x768@32bit, that's more than SD, but with very basic antialiasing and relatively small textures).
My point, to cut it short, is that having little more RAM it's perfectly natural it doesn't exceed XBox graphics capabilities in a dramatic way, as graphics are quite heavy on RAM, but Nintendo never meant to do it.