Frank Warren, co-promoter for WBC heavyweight champion Tyson Fury, says the unification with Anthony Joshua would have to take place on August 14.
On Tuesday, Joshua's promoter Eddie Hearn had mentioned two tentative dates, August 7 and August 14 - with event being staged in Saudi Arabia. Hearn said the 14th was the preferred date.
Warren indicates that his side of the fence will not even consider the date of August 7, to avoid any potential clash with the Tokyo Olympics.
Besides the Olympic, Joshua's head trainer Rob McCracken is also the performance director of the British boxing team. The team will be competing in the Tokyo games between July 23 and August 8.
"It definitely won't be on August 7 because that's when the Olympic games are on and that's going to be a problem for TV. Besides that, you've got a UFC PPV in the States and UK," Warren told Talk Sport.
"So if it goes on, it'll be on August 14. I don't think the date is the problem at all, I think everyone's agreed that can be moved to August 14."
At the moment both sides are working out the details of the site free agreement.
Fury is refusing to sign that agreement, because the unbeaten boxer and his handlers are requesting a documented guarantee to confirm the funds being offered to stage the card.
"What we're all trying to do is get this over the line, but the only way we'll get it over the line is by the fighter, Tyson, getting certain assurances. If he gets them, he'll put pen to paper, but as it is now, he's waiting for the other side's lawyers to confirm what he wants," Warren said.
"We want the fight, everybody wants the fight, for once Eddie Hearn and I agree that we have a common aim here. But it hasn't been signed, I don't know why he's saying it has been signed. There have to be assurances given and confirmation given that Tyson is happy with.
"I really do feel there's a way forward on this, I do feel having spoken with one of the principles there that he desperately wants to do this deal. But as I say there's just a couple of things that need to be addressed and I'm quite sure we'll get them addressed and get it over the line and we can get it on."