He is famous for his marriage with Kris Kardashian. He is the father of Kourtney Kardashian, Kim Kardashian, Khloe Kardashian and Rob Kardashian. He got married to Kris in 1978 and got divorced in 1991. He then married Jan Ashley in 1998 and divorced her, the next year. Ellen Person was his last wife. He married her in 2003 and was with her till his death in the same year. After his divorce from Kim, he was engaged to Shakarian Halicki whose husband was Late H B Halicki. But, he broke the engagement and got married to Jan. He got married to Ellen six weeks before he died and was diagnosed with esophageal cancer, eight weeks before his death.

The reason for the first separation was that Robert caught Kris having an affair with Todd Waterman. He accused that Kris was spending a lot and was trying to rekindle relationship with Todd and Bruce. She then married Bruce. Robert said that she was having an intimate relationship with her boyfriend, even in front of his children. He said that he knew that Kris was cheating on him, many years ago, but did not want to spoil his family name. But, due to his anger problems, the court gave the custody of his children to Kris.

Kris stated that Robert did not give her enough funds to carry on with her luxurious lifestyle. She said that she eats only in fancy restaurants, buys for high end retailers and spends tens of thousands for parties and complained that Robert did not give her money for the luxurious needs. She said that to meet her needs, she had to take a job. In 1991, she said that she needed more than 37 thousand dollars to spend for her expenses and Robert was earning only $2000. It was said that the firm he worked for was underpaying him and she told him to work enough to bring her more money. Robert said that Kris spent only $800 for the clothing expenses of all four children, but used $2000 for her clothes. She asked Robert to pay all the debts on the store cards which was $21,000 at the time of separation. She said that Robert abused her in front of Kim when she was 9 years old and that caused the separation. She said that Robert emotionally abuses her whenever he found a chance. Just after a month of separation, Kris married Bruce.

He and Jan got married with the condition that he would not have any more children. But, she changed her mind later and wanted to have children and this difference led to the separation. In 2012, almost a decade after his death, Jan and Ellen announced that Khole was no this daughter. It was said that Kris was having an extramarital relationship when she got conceived with Khole and Robert has only mentioned this to Jan and Ellen and no one else.