Stephen Colbert is ready to put some rumors to bed.
In a recent interview, the Late Show host responded to speculation that CBS execs are hoping to pull a late night swap -- giving James Corden's well-performing Late Late Show Colbert's 11:35 p.m. time slot -- and he's not too happy about it.
WATCH: Watch Stephen Colbert Crash the Republican National Convention and Diss Donald Trump
"The implication of that question is that the show isn't good enough in its present position," Colbert told The Hollywood Reporter. "So, of course, that makes you feel bad. But it doesn't jibe with what I know about our show, so you recover."
Rumors of the switcheroo have been perpetuated by the media in recent months, after Colbert has failed to maintain consistent ratings or a develop a strong digital presence, compared to Corden's success with Carpool Karaoke. Howard Stern in particular, fueled the fire, asking Corden of the rumors during an interview last month. While Corden was quite adamant that a flip-flop would not take place -- "Never gonna happen," he told the radio host -- speculation continued to circulate.
"That's a reasonable question for Howard to ask, I suppose, because Howard asks provocative questions. I mean, any question is a reasonable question if you think it's interesting, and that's a very interesting question," Colbert said. "But, I mean, what can I do about a question? All I can do is the best show I can."
RELATED: Jon Stewart Impersonates Donald Trump While Reuniting With Stephen Colbert on 'Late Show'
Despite Colbert's "uneven" ratings (as described by CBS Entertainment President Glenn Geller), CBS asserts they have no plans to change the current show lineup.
And recently, Colbert has seemed to have gotten his groove back -- thanks in part to his old Colbert Report persona, and appearances by pal and fellow Late Show EP, Jon Stewart. Let's not forget that Colbert thrives in the political arena, which is exactly what CBS execs are banking on, as the Late Show will air live during both political convention weeks.
"I have no obligation to be balanced in any way," Colbert told ET about his live shows. "I have an obligation to be funny, and to be honest, talk about things that I think that really interest me or are worth making jokes about. On one side, you've got an extremely cautious candidate who doesn't want to do anything out of the ordinary, and on the other side you have a giant orange manatee that is constantly screaming please harpoon me."
The Late Show host caused quite a stir after he crashed the Republican National Convention on Sunday, and Tuesday's night episode featured a much-buzzed about Melania Trump impression from Broadway star Laura Benanti.
EXCLUSIVE: Laura Benanti on Melania Trump's RNC Speech: An Apology 'Should Have Happened Yesterday'
See the hilarious performance in the video below.