It was the moment that America’s State of the Union address, once a staid affair punctured only by applause, turned into a verbal brawl more akin to Britain’s House of Commons.

Joe Biden accused some Republicans of wanting to “take the economy hostage” and slash social welfare entitlements. “Booo!”, “No!” and “Liar!” came the response. US presidents typically ignore hecklers but Biden chose to take them on.

“So, folks, as we all apparently agree, social security and Medicare is off the books now, right?” he sparred. “They’re not to be touched? All right. All right. We got unanimity!” He gave Republicans an offer they could not refuse: to rise from their seats and stand in support of the elderly.

At a stroke, the combative Biden had bested his opponents and at least partially assuaged doubts that, at the age of 80, he has the fight and fortitude for a gruelling re-election campaign next year. It was an important victory at a moment when opinion polls show that even most fellow Democrats hunger for a new generation of leaders.

John Zogby, an author and pollster, said: “Like Muhammad Ali, he floated like a butterfly and stung like a bee.”

Even so, one speech will not be enough to solve the continuing political puzzle of the two Joe Bidens. One is the Biden visibly energised by Republican jeers who found a way to squash them without smugness; the Biden who rallied the west to support Ukraine and helped Democrats defy history in the midterm elections; the Biden who reeled off the most consequential list of legislative accomplishments since President Lyndon Johnson more than half a century ago.

But the other Biden has not gone away. He is the one who began his lengthy State of the Union address – which drew the second smallest audience TV audience in at least 30 years – somewhat lethargically, describing Chuck Schumer as Senate minority leader when he should have said majority and saying relatively little about abortion rights. This is the Biden who presided over soaring food and petrol prices, bungled America’s withdrawal from Afghanistan and left classified documents in his garage.

A survey in late January by the Associated Press-NORC Center for Public Affairs Research found that just 37% of Democrats say they want Biden to seek a second term, down from 52% in the weeks before last year’s midterm elections. Overall, 41% approve of how Biden is handling his job as president and only 22% say he should run again.

Among Democrats aged 45 and over, 49% say Biden should run for re-election, nearly as many as the 58% who said that in October. But among those under age 45, just 23% now say he should run for re-election, after 45% said that before the midterms.

Interviews with poll respondents suggest that many voters believe the president’s age is a liability, with people focused on his coughing, gait and gaffes and the possibility that the world’s most stressful job would be better suited for someone younger.

A separate Washington Post-ABC Newspoll showed that 62% of Americans think Biden has accomplished “not very much” or “little or nothing” during his presidency, while 36% say he has accomplished “a great deal” or “a good amount”. Some 60% say he has not made progress creating more good jobs in their community, even though he has overseen the fastest pace of job growth in US history and unemployment sits at its lowest level since 1969.

The disconnect might feel like a kick in the teeth for Biden after notching four big legislative victories with coronavirus relief, a bipartisan infrastructure law, legislation boosting domestic production of computer chips and tax and spending measures that help to address the climate crisis and improve the government’s ability to enforce the tax code.

The gap between perception and reality is hard to explain. The chaos of the Donald Trump years, a pandemic that killed more than a million Americans and an ongoing reckoning over racial justice have inevitably left the nation disoriented. But some critics argue that the White House is failing to communicate its achievements.

Michael Steele, a former chairman of the Republican National Committee, said: “The messaging by the administration has been lacklustre. It has not been well coordinated. It has not been well reinforced by agency heads and cabinet members who can take that work that they’re doing out to the country.”

Steele pointed to the $1.2tn infrastructure bill, signed by Biden in late 2021, as an example. “Everybody’s jumping up and down but what they did not explain to the country was, now we’ve got to go put in place the regulations that would correspond to the allocation of those dollars … That part of the conversation never happened so voters are sitting there going, ‘Well, I don’t see any impact from this. They’re not doing anything in my community.”

The misstep cost Democrats control of the House of Representatives, Steele added. “I’m now watching commercials of the president delivering on his promises and I would say, yeah, that’s probably about four or five months late.”

There have been frustrations for Biden over police reform and votings rights, which could potentially cause disillusionment among Black voters. In his address, he continued to urge reform but did not explicitly call for the George Floyd Justice in Policing Act to pass.

But activists argue that he should call out Republicans and make clear that they are the ones standing in the way.

Rashad Robinson, president of Color of Change, an advocacy group, said: “There’s a reason why we don’t have those pieces of legislation and it’s not President Biden. But if he doesn’t tell a story for people about why we didn’t win those things, who stood in the way of those things, who is profiting from preventing those things, he will be blamed by people.”

Biden’s defenders argue that there has been a concerted effort to sell his agenda and accomplishments. A day after his State of the Union address he travelled to Wisconsin, and a day after that he went to Florida, while other top officials are crisscrossing the country to spread the message. It can be effective at a local level but struggles to compete with eye-catching national headlines like the flight of a Chinese spy balloon. Some argue there is no substitute for concrete results that affect people’s everyday lives.

Elaine Kamarck, a former official in the Bill Clinton administration, said: “You’ve got to see things happening. You have to see the bridges being built. You have to see the tunnels being fixed. You have to see the airports. That’s the reality. The challenge is to make this real. This is a problem of political timing, which in the term of a president is very short but often getting big things done takes a long time.”

Kamarck, now a senior fellow in governance studies at the Brookings Institution thinktank in Washington, added: “He’s got to get something on the books. He’s got to get something going. He can say it till the cows come home but if there’s no reality on the ground it won’t matter.”

Biden’s biggest first-term legislative accomplishments are almost certainly behind him. He must now work with an aggressive Republican majority in the House that wants to cut spending in return for lifting the government’s legal borrowing authority, as well as launching myriad investigations into the pandemic response, Afghanistan withdrawal and business dealings of the president’s son, Hunter Biden.

He also faces nagging doubts within his own party. Having first been elected to the Senate from Delaware in 1972, he has been on the national political stage for more than half a century and is the oldest US president in history. His verbal stumbles – he recently called Congressman Don Beyer by the name “Doug” four times – receive more scrutiny than ever.

Biden could face another election against Trump, a twice impeached former president who instigated a violent coup attempt on 6 January 2021. Yet in a hypothetical rematch, 48% of registered voters said they would favor Trump compared with 45% who prefer Biden, according to the Washington Post-ABC News poll.

Julián Castro, a former housing secretary under President Barack Obama, noted that this finding undermines the general consensus that Democrats are content with Biden taking on Trump. “Two years is forever and it’s just one poll, but if he’s faring this poorly after a string of wins, that should be worrisome,” he tweeted.

But Biden, who has not yet officially announced he is running, benefits from a lack of obvious successor. His vice-president, Kamala Harris, has endured similarly low approval ratings and is yet to distinguish herself as the automatic choice. The transportation secretary, Pete Buttigieg, is still only 41 and a promising generation of Democratic governors are widely perceived as not yet ready.

Steele, who served as lieutenant governor of Maryland from 2003 to 2007, said: “The question I ask those Democrats who like to wax poetically about Biden not being their nominee is this: tell me which Democratic governor or former senator or current elected official is going to challenge an incumbent United States president.

“Every time we’ve seen that happen – which is in our lifetime has only been twice – it has not ended spectacularly well for the challenger, so I don’t know what the hell they’re thinking. This is the horse that got you through the storm. This is the horse you’re going to need to ride into the sunset and that’s just how it is.”

Many Republicans acknowledge that Biden had a good night at the State of the Union and quelled doubts about his age. But they believe that he could be vulnerable in the 2024 election if he faces a candidate promising generational change and who does not carry Trump’s political and legal baggage. Ron DeSantis, the governor of Florida, and Nikki Haley, the ex-governor of South Carolina, are among potential contenders.

Ed Rogers, a political consultant who worked in the administration of Ronald Reagan, the oldest man to serve as president until Trump, said: “There would be all this talk about Reagan’s losing it, Reagan’s out of it, Reagan’s not mentally hitting on all cylinders – and then Reagan would do something and people would observe for themselves and it would clear the bar and calm that talk for a while.

“Biden certainly did that. The speech was well delivered … It’s not like people see their lives improving because Biden says it is or people are not fearful of crime, the future, the state of the schools because Biden says they’re OK. That’s part of what’s not good about the Biden administration. The speech was a net plus. It wasn’t transformative for the Biden political condition.”

Rogers reckons Biden will win the Democratic nomination for 2024 and is well placed in the general election – but nothing is guaranteed. “We re-elect 75% of our presidents. If you had to bet today, you’d bet on the incumbent. But if it’s not Trump, if somebody showed up as an energetic change candidate, Biden could be beat.”