"He was pretty revealing about some emotional stuff and alcoholism," Pauley told ET. "He says at the age of 11 his dad gave him a sip of a Christmas cocktail or something, and Dave liked it, and that's when it started."
Letterman spoke to ET about his drinking back in 1996, explaining his decision to put down the bottle.
"Maybe part of it was the first time I was married I was dead drunk, so I don't drink anymore," Letterman said. "I haven't had a drink in like fifteen years, so maybe that's part of it. Maybe if I was continuing to drink, I would be lining them up left and right."
Letterman's first marriage to Michelle Cook crumbled after nine years, and the brutal breakup made him cautious about tying the knot again. However, in 2009, the legendary comedian wed his current wife, Regina Lasko, after 23 years of dating and having a child together, 11-year-old son Harry.
The same year that the two said "I do," Letterman found himself caught in the middle of a blackmail plot over a sex scandal. Pauley recalled Letterman's public confession that he had sex with women that worked for him at Late Show in a preemptive strike that put the kibosh on the blackmail plot.
"He came on the air and publicly apologized to his staff, to the viewers and the recognized that he might have jeopardized his career and might have jeopardized his family," Pauley said.
"I don't think I'll ever be back in this building again, honestly," Letterman says in a clip from the interview. "There have been a lot of great things to celebrate. There have been a lot of arguments, nearly fist fights and every emotion that people can share for over 20 years ... My family will be here. Many people have been with me for 33 years, 35 years, going back to the morning show. We'll go and have a little party and then the dark cloud of never-ending depression will set in."