Friend: Matt Lauer ‘Weak and Broken and Ashamed’ a Week After Firing

One week after Matt Lauer‘s surprising termination from NBC, the veteran broadcaster is still shaken up.

“He is weak and broken and ashamed by his own admission,” one of Lauer’s former Today colleagues told PEOPLE.

“He is my friend,” the colleague adds. “He is human, and he is flawed. But is he going to use his power to be a predator on women? Until I see solid evidence of that, I just can’t believe it. We don’t know the details.”

Lauer, a face of NBC’s morning news broadcast for more than two decades, was ousted from the network following a complaint of “inappropriate sexual behavior in the workplace” toward a colleague. After his firing was announced on Wednesday, multiple women came forward alleging that Lauer behaved inappropriately toward them in a variety of ways.

Detailed reports by Variety and the New York Times published accounts by women who said Lauer exposed himself, gifted sex toys or assaulted them in his office.

His colleague who spoke to PEOPLE isn’t convinced all these allegations are true.

“All of us are career women, and not one of us would have put up with an environment that’s as distasteful as has been described,” the colleague insists. “There was never anything I was offended by on any level. I wouldn’t have put up with it. Neither would any of us.”

Another Today staffer said most employees inside 30 Rock are “heartbroken” over Lauer’s demise as they are torn between the person they knew and the person being accused of these lewd acts.

“The reason people are so upset and people are crying on our staff, which is like 80 percent women, is because he was really loved and treated people really well,” the employee told PEOPLE. “And it’s shocking to think that there could be this side.”

Despite his friends’ support, Lauer admitted in a statement Thursday that “there’s enough truth in these stories” to make him feel embarrassed and ashamed by his actions.

“Repairing the damage will take a lot of time and soul searching and I’m committed to beginning that effort. It is now my full time job,” Lauer said in part with a statement to Today.