Khuutra, have you ever watched Around the Horn on ESPN? 4 people, who don't always agree with 1 moderator. You could have 2-3 regulars, with 1-2 people you cycle out. You get controlled controversy, as the host can "mute" a speaker if they are being ridiculous (or cutting someone off), and the host can direct questions to specific person if it is suited for them. Obviously we wouldn't have the point system, so you wouldn't really have to have every person answer every question either.
You also get familiarity with the host (you) and people are generally familiar with a few of their favorite "analysts", but you still get to include new guests.
edit: oh, also the host can interject his own thoughts, but he leaves the majority of the discussion to the others.