"In terms of not physically being there -- yes he is, because he's making a choice," Kennedy said.
When Williams described the relationship to The Huffington Post, he said that he was "rebounding" at the time and referred to Kennedy as his "playmate."
Kennedy tells us she's temporarily staying at her mother's house. Meanwhile, Williams has relocated to Branson, Miss. to shoot a reality show about his new stage production in the small town and his struggle to drum up an audience. The series, A Very Barry Branson premiered on GAC last month.
"He told me in 2012 that with the money he had he could retire and we would be able to live comfortably for the rest of our lives," Kennedy told Brooke. "[The show] is not about money. That's about promoting himself -- his fame."
When ET reached out to Williams for comment, he replied, "I don't want to dignify her claims with a response."
Kennedy says that her wish is simple.
"I want him to grow up and do the right thing by his daughter," she said.
UPDATE: Williams’ lawyer responded to Kennedy’s claims in a statement Thursday:
"Mr. Williams is working and 'taking care of his own kid,'" the statement reads. "He has taken care of Samantha since her birth. Mr. Williams listed himself on the birth certificate as Samantha’s father when she was born. It is Elizabeth who prevents all contact between Mr. Williams and Samantha."
"Mr. Williams commenced his move to Branson, Missouri, and the start of his show there, before he even knew Elizabeth was pregnant," Williams’ lawyer continues. "He has not abandoned Samantha."