Why is boxing history so kind to illegal substance users?

[knockemout]

Here's some fighters that have either tested positive for illegal substances or admitted to using them, in the past:

James Toney
Evander Holyfield
Roy Jones Jr.
Shane Mosley
Tommy Morrison
Vitali Klitschko
Fernando Vargas

Why is it that nothing is taken away from the legacy of these fighters? Is it suddenly a good thing that all these fighters used illegal substances, at some point?