I'd go with GH3, because
A) It's cheaper
B) It has DLC like Rock Band so it'll last a while
C) The brand doesn't really matter, so even though the original GH makers are making Rock Band...it's a simple game to be honest, and it's really the songs included that matter...and GH3 has an impressive list.
D) Yes Rock Band has more instruments, but that hasn't proven to make the genre more fun. Drums may be fun, but they might not be as fun as the guitar portions, and honestly, I think karoake games suck, so that's a wasted instrument for me. Even though I am a good singer, I don't want to sing in front of them on the couch, especially if I'm trying to outsing an original performer, who obviously can do it better than me.