People often blame the fighters for ducking good opponents, but it's the promoters and managers that hold them back.
Denis Boytsov was willing to fight Wladimir Klitschko last fall, but his promoter said "no".
Alexander Povetkin actually never ducked Wladimir, his promoter, trainer and manager advised him to avoid the fight in 2010.
Deontay Wilder would fight anybody, it's his manager who is putting him in with low-level clubfighters.
One exception however: Floyd Mayweather, who decided on his own to "retire" in 2007 when there were several very good prime welterweights. But most other fighters do not make the decisions themselves.