Matt Hancock is facing calls to quit as an MP and threats of deselection from his own local party following the revelations that saw him leave government.
The former health secretary is facing a revolt from senior Conservatives in his Suffolk constituency that could see him ousted from parliament altogether.
Ian Houlder, a Tory councillor on West Suffolk council, said he had written to his local Conservative association chair about Mr Hancock’s conduct, which he found “really contemptible”.
“Think of people who haven’t been able to bury their mothers or fathers,” the councillor told the Telegraph newspaper.
“There he is, just groping away, hands everywhere, tongues everywhere, out of his bubble.”
Mr Houlder added: “I think there should probably be a by-election. I don’t think he should cling on, hoping that people’s memories fade.”
Meanwhile Terry Clements, another Tory councillor in the area, told ITV News Mr Hancock would “find it impossible to carry on” as a backbench MP.
Mr Clements, who has served on the local council for 38 years, suggested Mr Hancock could soon have discussions with local Conservative members who would “advise” him about his position.
“I’ve been in some difficult positions and decisions have to be made. He will come to the right decision, I’m sure of that,” Mr Clements told the broadcaster.
There is thought to be a range of opinion in Mr Hancock’s local party about whether he should stay on, with allies saying the issue was still being processed.
The health secretary stepped down on Saturday following revelations about his breaking social distancing while having an extramarital affair with an aide.
The aide had also been appointed a non-executive director at Mr Hancock’s department and was meant to scrutinise him.
There are also separate allegations swirling around whether Mr Hancock used a personal email address for official work – thought to be problematic for reasons of both security and scrutiny.