Ray J and his legal team might be cooking up something for Kanye West following the release of his “Famous” video.

West’s controversial visual shows famous entertainers’ fake bodies laying naked together in a bed, and Ray J just so happened to be one of them. West’s inspiration for the video stemmed from Vincent Desiderio’s painting “Sleep,” but Ray J isn’t too pleased with the way he decided to re-enact the painter’s artistry.

West also name dropped the singer in his song “Highlights,” rapping, “I bet me and Ray J would be friends/ If we ain’t love the same b****,” and a few years ago, following the release of “I Hit It First” by the singer, he performed “Bound 2″ on Jimmy Fallon adding new lyrics, “Brandy little sister lame and he know it now/ when a real brother hold you down you supposed to drown.”

The “One Wish” singer’s business manager, David Weintraub, spoke with Heat Magazine about West’s constant digs towards his client, and it seems like enough is enough now.

“We’re talking about potentially doing something,” said Weintraub. “There’s too many times Kanye has done this. You can’t really continue to use people’s likenesses without either compensation or clearance. It seems to be becoming an issue—this is like the fourth time in six months he’s done this. We haven’t really worked anything out. There were kind of discussions but it’s getting old, so our legal team is working out what we want to do with everything.”

Ray J has been trying to turn his life around for the better and the way his image is portrayed by West is against his likeness.

“I’m engaged,” he told Entertainment Tonight. “My marriage is all messed up ’cause of craziness. I’m not part of nothing crazy. I walk down a good path. My fiancée tripping. It’s all bad in my world, and that ain’t cool. Don’t put me part of nothing that’s weird, that don’t make no sense. Can I grow up?”