Cannon served as a gust judge on the talk show earlier this week as one of the temporary hosts that have stepped up as Williams continues to be on her hiatus due to health concerns.
According to a report by the DailyMail, Williams' extended break has caused drama behind the scenes, as producers and other employees said they were left "in the dark" about her absence and now claim she took the time off to deal with her husband's infidelity.
"I mean it's probably time Wendy steps away from the show," one source told the outlet. "She's not authentic anymore. How can she discuss hot topics when the biggest topic involves her husband publicly cheating on her for years?"
"It's really sad as you'd think that Wendy would have more respect for herself and she would stand up, dump her cheating husband, take control of her business and empower women who are going through the same thing," they added.
The source continued saying that employees would be happy to see Cannon take over as full-time host of the show, and if he said no Jerry O'Connell would be a good second option.
Cheating rumors have revolved around Williams' husband Kevin Hunter having a decade-long affair with masseuse Sharina Hudson, with the outlet previously reporting she has been spotted with a diamond ring on her right finder.
Williams resurfaced last week in Florida, sparking speculation that she has been spending time with family as she figures out whether or not to divorce Hunter.
Along with Cannon, KeKe Palmer, Michael Rapaport and Jason Biggs have been tapped to guest host in the series throughout February episodes. A return date for Williams has not been announced.
The report comes as Page Six reported that Williams was scared she could lose the series altogether due to her health hiatus.
"She's super scared and anxious," an insider told the outlet. "Watching [fill-in host] Nick Cannon being a huge hit with her audience is tough for her. They're already starting to mess with the format. It's like watching them screw around with her baby."
Williams last appeared on the show back in December. Since then her return date had been postponed until the show began to announce guest hosts to keep production of new episodes going in her absence.