Kids were a big reason Chip and Joanna Gaines decided to end Fixer Upper -- but not because Joanna is pregnant.
"There's so many [rumors], even leading up to it -- I had this skincare line, so I'm leaving. It made it look like I was doing my own thing and Chip didn't even know about it. That was just a big rumor," the mother of four told ET's Kevin Frazier in New York City on Tuesday. "That our marriage is on the rocks...I was pregnant. Its funny when you start hearing this stuff. I was like, 'I guess people can just make stuff up.'"
The HGTV couple announced last month that they would be ending their show after its upcoming fifth season. "Our plan is to take this time to shore up and strengthen the spots that are weak, rest the places that are tired and give lots of love and attention to both our family and our businesses," they said at the time.
"For me, I've always thought I'd rather be missed than somebody get tired of us. I would prefer to leave one year too early," explained Chip, who was celebrating his new book, Capital Gaines: Smart Things I Learned Doing Stupid Stuff.
"That's honestly what led us to the decision to leave the show. We just felt like now is the right time," he continued, noting Joanna's sweet Instagram tribute. "I certainly know how I felt before the fame."
"I just want to sit somewhere in my sweatpants!" he added.
The couple -- who said they're "loving every minute" of wrapping up season five, which finishes filming the first week of November -- are happy with their decision to take a break, but they're not ruling out a possible return to TV.
"We really don't know [if this is the end of us on TV]. I feel hopeful for whatever it is," said Joanna, who admitted that they still don't have a TV in their house. "We have a real-life business on the ground in Waco, Texas, and we are opening a restaurant in January and February. There are lot of things, we just got to get on the ground and be a really big part of that as these things are kind of still going."
Chip added that Capital Gaines will show a new side of his and Joanna's journey.
"[Readers will learn] that it wasn't easy. There was not a single, easy thing about it, lots of late nights. We put in tons and tons of hard work, even as we were bringing our family along on this journey," he shared. "I just wanted it to be authentic and sincere, and I know that this journey for us has been just that."
"There's a few things you'll see me chime in and say, 'Actually, this is the way I saw it,'" Joanna said of some of the book's lighter moments. "I don't think you left out much... There was a lot in there."
"The fact that we got to go through this and together come out stronger, I think is a real gift," she added.
Capital Gaines: Smart Things I Learned Doing Stupid Stuff is available online and in bookstores now.