According to the reports, the 83-year-old Mori, a former Japanese prime minister, sparked a global outcry with sexist comments that women talk too much, which he made during an Olympic committee meeting.
Mori has apologised for his comments but has so far refused to resign, despite growing calls for him to step down.
His resignation less than six months before the Summer Olympics are scheduled to begin would raise new doubts over the viability of holding the postponed Games this year.
Games officials are already struggling with how to hold a safe Olympics, with tens of thousands of athletes and possibly spectators, during the coronavirus pandemic.
