The 33-year-old Kardashian is finally leaving Cleveland in the summer to return to Los Angeles, and Thompson is coming with her.
"Tristan will be in L.A. as well. Khloé's family is still very unhappy with Tristan," a source told PEOPLE Magazine on Friday. "It will be interesting to see how they will treat him. Although Khloé defends him and her reasons to stay with him, her family feels she deserves much better."
An insider said Thompson is "nervous" about seeing the Kardashian-Jenner family.
"He knows they're mad at him and he's definitely going to work to get back on their good side," the insider told PEOPLE.
The family might not like Thompson, but they are trying to focus on Kardashian's happiness. They want to have a "big celebration together" once Kardashian gets home to Calabasas, California.
"The plan for Khloé was always to travel with the baby back and forth between Cleveland and LA. This is why she decorated two nurseries. The nursery in L.A. is ready for her return," the source told PEOPLE. "Khloé can't wait to get back to L.A. She will spend a lot of time in L.A. this summer. She is very excited. She misses her family and the every day activities they usually do together. She wants to be around her cousins as well."
Kardashian welcomed daughter True Thompson on April 12. Days before, photos and videos showing Thompson with other women while Kardashian was pregnant surfaced, but the couple have remained together. The scandal has given Thompson a distraction while the Cleveland Cavaliers try to beat the Golden State Warriors in the NBA Finals, though.
"Tristan is completely emotionally and physically exhausted from the cheating allegations and of course the playoffs," a source told Us Weekly earlier this week. "Khloé has been harping Tristan about lining up a trainer in Los Angeles for the off season. He is focused on the winning a championship for Cleveland."
Another insider also told Us Weekly last week that Kardashian and Thompson are constantly fighting.
"She doesn't know how she's going to rebuild trust in him," the insider said on May 25. "She knows her family and friends aren't supporting him or her right now."
Kardashian has stayed in Cleveland, but her mother, Kris Jenner, recently said she will be moving back to Los Angeles for good.
"She'll be home soon, so it's really great," Jenner told Us Weekly at the premiere for the Paramount Network's American Woman on Thursday.
"Khloé is great! True is too," Jenner added. "I've been FaceTiming with them for the past couple of weeks."