Stern slammed Cowell for allegedly turning AGT, specifically the judges' panel, into "the ultimate example of a boys’ club," according to The Hollywood Reporter.
"He sets it up that the men stay no matter how ugly [the men] are, no matter how old they are, no matter how fat they are, no matter how talentless they are," Stern said, referring to how Cowell casts the show's judges.
"What he manages to do on all his shows is he constantly replaces the hot chicks with hotter chicks and younger chicks. Which is so obvious,” said Stern, who himself served as a judge for four seasons, alongside Howie Mandel.
Cowell joined the show's panel of judges immediately following Stern's departure.
Stern is the latest celeb to weigh in on the controversy, which erupted into the public eye following an expose in Variety last week that claimed Union was cut from the show after urging producers to make a complaint to human resources about incidents that allegedly took place on set.
Among the alleged incidents that occurred during Union's time on the show was an act that she found racially insensitive during the audition rounds. Additionally, sources who agreed to speak to Variety on the condition of anonymity claimed that Union received specific critical network notes on her hairstyles, which were allegedly deemed "too black" for the AGT audience, and that she received these notes numerous times.
A network insider denied the volume of those notes to Variety, claiming that there were only notes about "hair continuity" for both her and Hough.
NBC and Fremantle, the production company behind AGT, responded to the reports and allegations on Tuesday, telling ET in a statement, "America’s Got Talent has a long history of inclusivity and diversity in both our talent and the acts championed by the show."
"The judging and host line-up has been regularly refreshed over the years and that is one of the reasons for AGT’s enduring popularity," the statement continued. "NBC and the producers take any issues on set seriously."
Another incident allegedly occurred when Union and Hough were shooting a segment with guest judge Jay Leno back in April when the former late night talk show host allegedly made a racially insensitive joke.
The incident reportedly involved the comedian making an off-the-cuff remark about a portrait of Simon Cowell alongside his dogs. Leno allegedly joked that the dogs looked like they would be "on the menu at a Korean restaurant."
The alleged joke reportedly offended a few staff members, since it was viewed as perpetuating stereotypes about Asian cultures eating dog meat.
Union urged producers of the show to report the incident to NBC's Human Resources department. She reportedly had a conversation with an NBC exec, but the incident was never brought to HR, according to Variety. Leno's comment was cut before the segment aired.
The comedian, however, had nothing but praise for Union as he was caught by paparazzi outside of the Comedy and Magic Club in Hermosa Beach, California, on Sunday night.
"I love Gabrielle Union. She's a great girl," Leno told paparazzi in a video obtained by TMZ. "I really enjoyed working with her."
"I don't know [if she was treated fairly]," he added. "She's a great girl."
Meanwhile, Stern also weighed in on Leno's alleged comments and Union's reaction to them.
"Let’s go on record as telling people that one million dogs are still eaten annually in South Korea, and if Gabrielle Union wants to effect positive change, maybe focus on the horrors of Korean dog farms and let the old irrelevant comic live in peace," he said. "She wants Jay beaten in the streets. I think maybe Jay makes a point."
In a statement to ET on Sunday, NBC, Fremantle and Syco said: "We remain committed to ensuring a respectful workplace for all employees and take very seriously any questions about workplace culture. We are working with Ms. Union through her representatives to hear more about her concerns, following which we will take whatever next steps may be appropriate."
However, Union's complaints have also sparked a SAG-AFTRA investigation into the working conditions on AGT.
In a statement on Monday, a spokesperson for SAG-AFTRA told ET: "We take issues of workplace health and safety very seriously. We immediately reached out to Ms. Union's representatives when these reports came to light. It is our practice to work closely with members who reach out to us and their representatives in instances like this, as that usually affords the best protection and best resolution for the affected member."
"For certain matters, our investigation and enforcement needs to happen independently and we are prepared to handle this issue accordingly, as warranted," the statement continued. "Our enforcement action is usually handled confidentially to protect the member involved, and we typically do not publicize these matters unless the member requests that we do so."
SAG-AFTRA added that, "while we have taken steps to investigate this matter, we have nothing to report now."
A source told ET that although Union is "done with AGT," it's her hope that she can "work with NBC to help improve the culture surrounding the show and studio."
"Gabrielle has always been an activist and wants to make positive change so nothing like this happens again," the source added.
For more on the ongoing drama, check out the video below.