On the surface the obvious answer - and it has been given a lot of times already - is that the GBA had no competition worth mentioning; Nokia's N-Gage fell into this era, but that device sucked ass.
Digging a bit deeper, the thread's question is more interesting if we assume that both systems launched into a similar competitive environment and ask which one would have been more successful. This strips the GC of the biggest excuse that is made for it while it hardly affects the GBA.
After all, the GBA had good battery life, had an SP revision that fixed the one glaring omission of the original GBA (the lack of a backlight), competent ports of noteworthy 8 and 16-bit games, solid original games and of course Pokémon which was still riding high. There's simply no reason to believe that the GBA would have not been successful if it had had to compete for real.
On the other hand, the GC was home to many installments of big Nintendo IPs that were either disappointing or left fans with mixed reactions. Remove the competition and it's still not looking good. It doesn't work the way that if there was no competition, the GC would have sold 100 million units due to the market having no choice, because the market always has the option not to buy anything. 3DS vs. Vita proves this to be true, because the Vita quickly vanished while the 3DS still could never get close to the sales pace of the GBA, let alone the DS. Of course the GC would have sold more than the 21m it did in the real world, but forget about 100m in the hypothetical scenario.
The bottom line is that the GBA was much closer in matching the market's expectations towards a Nintendo console than the GC could ever dream of. By now we've had enough console generations to safely conclude that any Nintendo console can become successful based on Nintendo's output alone and regardless of how strong its competition is.
Legend11 correctly predicted that GTA IV will outsell Super Smash Bros. Brawl. I was wrong.
