"You look at your brother and you want to think the best about him. I always wanted to be like him. But one of the toughest things I ever had to tell my older brother was, 'I don't want to be like you anymore,'" John-David, 25, says, revealing that he feels it's now his duty to be the new role model as the second eldest Duggar brother.
"It's a big load. It's a big weight upon me now," he shares. "I've got a whole bunch of younger siblings looking up to me, and I've got to be an example to them. Because whatever I am, that's what they want to be. I know that's what it was for me and Josh."
John-David explains that just as "everything started to settle down" from Josh's first scandal -- in which the 27-year-old admitted in May to molesting five minor girls, including four of his sisters, when he was a teenager -- the family "got word of a second thing that we had no clue about."
"That was a little more of a shock probably than the first stuff coming out," he says about discovering Josh had been "unfaithful" to his wife, Anna, during the Ashley Madison hack. The father of four also admitted to a pornography addiction in August.
Joseph, 20, claims that the family had no idea Josh was "living such a secret life."
"Whenever somebody you respect the most is willing to get up and proclaim what we believe as Christians, about being true to your wife, you'd never expect that person to be the one that's involved in it," he says. "It broke my heart."
While the family dynamic has drastically changed over the years, Joseph says that growing up, he tried to get past his brother's wrongdoings.
"Even after the first thing that came out when he was younger, I never had quite as close of a relationship as I did whenever I was really young," Joseph reveals, holding back tears. "But over time, it's built back up. But then whenever this next situation came up, it blew my mind. I had no idea."
Watch the full clip below.
Earlier this week, a source told ET that despite their marital drama, Anna has been writing letters to her estranged husband, who entered a faith-based rehab facility in late August.
"Anna is planning on getting back together with Josh, that's just how it goes," the source said. "The family was taught to have a new level of forgiveness towards Josh."