During a press conference held on Wednesday to answer questions about the nuclear agreement the United States and other world powers reached with Iran this week, President Obama explained that there is no mechanism currently in place for revoking the honor, because it's never happened before.
Obama refrained from specifically commenting on the controversy surrounding Cosby, but he did state that this country -- and any civilized country -- should have no tolerance for rape.
"If you give a woman, or a man for that matter, a drug, then have sex with that person without consent -- that's rape," Obama said.
Cosby has not been charged with a crime and vehemently denies any wrongdoing.
The 78-year-old Golden Globe winner was awarded the U.S.'s highest civilian honor in 2002 by President George W. Bush. The medal is given to Americans who have contributed richly to the national life in some way. Past recipients include Lucille Ball, Walt Disney, Bob Hope, Maya Angelou and Stevie Wonder.
The call for rescinding Cosby's medal comes in the wake of more than a dozen women claiming they were sexually abused by the comedian, as well as the recent unsealing of 2005 court documents that revealed Cosby admitted to obtaining Quaaludes to give to "young women" he "wanted to have sex with."
The Medal of Freedom isn't the only honor that Cosby has managed to hold onto. Hollywood Chamber of Commerce president Leron Gubler said that Cosby's star on the Hollywood Walk of Fame would also stay put, as the stars are "considered a part of the historic fabric" of the Walk of Fame.