I know it sounds crazy, but there are plenty of people who didn't think Kirby Air Ride or Diddy Kong Racing would work, but they ended up being great games, the former being wildly unappreciated when it came out.
Now that we've seen Link make his way into Mario Kart, and the introduction of the Master Cycle Zero in Breath of The Wild, who says we can't have a Legend of Zelda racing spin-off? :)
It shouldn't just be another kart racer though. The game would have to change things up enough that it's own thing, instead of being just another Mario Kart clone. Having the Master Cycle be Link's vehicle for the game would be an absolute must, and since it's Zelda-related it would need a story and hub world like Diddy Kong Racing. Most likely the story would be something involving a plot by Ganon to AGAIN get all three Tri-Force segments, but story mode would let you change characters.
Maybe give the racing it's own combat system, like striking other characters with iconic Zelda weapons, and use Rupees to buy vehicle upgrades.
Well, those are my thoughts on how a Zelda racing game could work, what about you? :)