You could see it starting to show up toward the end of the 70s, especially when he moved to 147. He could have stayed at 140 for a bit, won a title or whatever, but mainly he looked soft at 147 apart from the Leonard fight. As it was, he could have stayed at 147 if not for his laziness, even after the Leonard fight.
He could have moved up to 154 or even 160, but stayed in brilliant shape and won all his fights instead of losing and winning some.
But, as it stands he did enough. Could certainly have stayed more in shape and shown up much fitter for his latter career big fights. Imagine a 154/160 pound Duran with the intensity and stamina of his earlier career? He lost so many of his fights because he had no intensity, motivation or stamina and of course all the weight affecting his speed, footwork etc.
You could say this type of stuff about so many fighters though.