Isn't it kind of obvious that this will have no impact? Shocking news: the kind of people who like Trump, don't like Colbert. The kind of people who like Colbert, don't like Trump. Both sides are going to come out of this looking like a winner/loser, depending on who you ask.
I'm sure there are exceptions, but it's not like that means anything.