Joe Biden’s journey from idealist to realist continues. But it is not quite complete.

After 36 years in the Senate, he stepped into the US presidency in 2021 as an apostle of bipartisanship, convinced that his authoritarian-minded predecessor Donald Trump would fade away and the Republican party would rediscover its bearings.

By the start of this year he had come to understand that Trump’s malign influence still runs deep. “We must be absolutely clear about what is true and what is a lie,” Biden said on 6 January, the first anniversary of the attack on the US Capitol, stopping short of using Trump’s name.

Then, a month ago, a group of historians reportedly gathered in the White House map room for two hours to give the president a dire warning about the threats facing American democracy. They included Jon Meacham, author of The Soul of America.

Polls show a country still unravelling. More than two-fifths of Americans believe civil war is at least somewhat likely in the next decade, a share that increases to more than half among self-identified “strong Republicans”, according to research by YouGov and the Economist.

Given that context, Biden used a primetime “soul of the nation” speech on Thursday night to deliver the starkest warning of his long career about the danger of Trump – whom he did name this time, – extremist “Maga” (Make America great again) Republicans and political violence.

“This is a nation that rejects violence as a political tool,” he said. “We are still, at our core, a democracy. Yet history tells us that blind loyalty to a single leader and the willingness to engage in political violence is fatal in a democracy.”

It was pugnacious talk that many progressives, worried that Biden is trapped in a rose-tinted past, have been urging for a long time. It was also sure to enrage Trump and his allies although Biden, who recently called the Maga philosophy “semi-fascism”, did not repeat that f-word.

But the speech also, perhaps understandably, tried to err on the side of optimism by drawing a distinction between Maga Republicans and mainstream Republicans. Biden cast the former as a weird rebel sect that opposes the rule of law, seeks to overturn elections and revels in violence. Maga has, he implied, imposed minority rule on a party of otherwise reasonable people.

“Donald Trump and the Maga Republicans represent an extremism that threatens the very foundations of our republic. Now, I want to be very clear up front: not every Republican, not even a majority of Republicans, are Maga Republicans. Not every Republican embraces their extreme ideology. I know, because I’ve been able to work with these mainstream Republicans.”

He continued: “But there’s no question that the Republican party today is dominated, driven and intimidated by Donald Trump and the Maga Republicans and that is a threat to this country.”

Perhaps. The point is at least debatable after the past seven years. Numerous mainstream Republicans have retired or been purged, most recently congresswoman Liz Cheney. Polls show a majority of Republicans believe the 2020 presidential election was stolen. Trump and Florida governor Ron DeSantis, both extremists, are the two leading contenders for the party nomination in 2024. There is no moderate alternative with any reasonable chance.

In what may be a triumph of hope over experience, Biden still wants to believe that Maga is being enforced from the top down. But there is a case to be made that it comes from the bottom up, with millions of grassroots Republicans willing to buy into false conspiracy theories and vote for extremist midterm candidates. As Trump may discover to his cost, the base has taken on a life of its own.

If Biden, with elaborate stagecraft at Independence Hall in Philadelphia, where the nation’s founding documents were written, was striving to isolate Maga and bring Democrats and democrats together to reject it, he may have more work to do. The right was bound to compare his critique to Hillary Clinton’s infamous 2016 “basket of deplorables” comment anyway.

Rupert Murdoch’s Fox News network did not show the full speech and, when it did, questioned why Biden spoke against a dramatically lit “blood red” backdrop. Far-right host Tucker Carlson, the most watched prime time figure on cable news, sneered and deployed chyrons such as: “This is by far Biden’s most shameful moment.”

The network’s 9pm bulletin added, “Biden vilifies millions of Americans”, “Biden uses primetime address to fuel more division” and “Clueless Biden spews hate in dark, dismal speech”.

Biden has evolved a long way from the man who suggested that he could turn back the clock to a golden age of Democrats and Republicans debating together, dining together and respectfully agreeing to disagree. But in November’s midterm elections, or in 2024’s presidential poll, his harshest lesson may yet be to come.