For me it's because he was pushed to the moon by Vince just because of his size and look, despite being one of the worst workers ever.
He was also a complete nightmare to work with, who thought he was above 'wrestling' as a business. A total arsehole to his coworkers who he had no interest in protecting in the ring, or making look even decent because it was all about him and his brand.
He was a narcissistic prick basically, and even though he mellowed slightly before his death he'll still be remembered by many as a toxic presence.