The deeply partisan US Congress is rarely a conducive place for national introspection and Tuesday’s Senate hearing on the Afghanistan withdrawal did not provide an exception.

In the midst of the point-scoring and blame-shifting on display in the senators’ questions to the nation’s military leadership, it was clear that it was a contest to apportion shares in failure.

And behind the debacle of the pullout that left tens of thousands of vulnerable Afghans behind, there were fleeting references to the far deeper failure of the preceding two decades – a reckoning that has only just begun for the Pentagon and the US foreign policy establishment.

For obvious reasons, the Republicans on the armed services committee sought to keep the focus on the past eight months, accusing Joe Biden of surrendering to terrorists, throwing in occasional demands for the immediate resignation of the three military leaders sitting before them: the chairman of the joint chiefs of staff, Gen Mark Milley, the head of Central Command, Gen Kenneth McKenzie and the defence secretary, Lloyd Austin.

The three witnesses, nudged from time to time by the Democrats, would occasionally point out that the surrender in question truly began in February 2020, with the Trump administration’s agreement with the Taliban in Doha.

In return for a US withdrawal by 1 May this year, the Taliban agreed to seven conditions, but as Gen Milley pointed out, they only stuck to one of them: not to attack the departing Americans. Every other undertaking, such as entering into good-faith negotiations with Kabul and breaking ties with al-Qaida, the Taliban had reneged on. But Trump nonetheless sought to accelerate the withdrawal, at one point demanding it be completed by January.

The Republicans barely mentioned the Doha agreement, preferring to frame the withdrawal as Biden’s original idea. It was hardly surprising. Over the past four years, the party has become accustomed to cognitive dissonance. The one backhanded Republican reference to Doha came from Thom Tillis who asked why Biden had stuck to Trump’s bankrupt deal.

“I don’t buy the idea that this president was bound by a decision made by a prior president,” Senator Tillis said. “This was not a treaty. And it was clearly an agreement where the Taliban were not living up to it. President Biden could have come in, reasserted conditions and completely changed the timeline.”

The answer the White House has given is that the Doha agreement gave the Taliban considerable momentum, boosting their morale, destroying Afghan government self-confidence, and freeing 5,000 Taliban fighters. Reneging on the deal by a new administration would have meant a return to war with far higher stakes, and probably tens of thousands of soldiers.

Milley, McKenzie and Austin, however, took a different route in their testimony. They confirmed that in the policy of review in February, March and April, they had advocated retaining a small force of about 2,500.

They were not questioned on how such a remnant could have withstood a resumption of direct attacks from an invigorated Taliban. Instead the focus of many of the exchanges centred around Biden’s baffling insistence in an ABC News interview on 19 August that he had received no such advice from his military advisers.

The three witnesses tried to get around this awkwardness by saying that while they had believed a small force should have been retained, they could not possibly disclose the nature of their advice to the president.

Asked by Senator Tom Cotton if Biden had been speaking the truth, Austin could only squirm and insist the president was “an honest and forthright man”.

In his opening statement Austin had sought forlornly to widen the focus from the last few weeks of the US presence in Afghanistan to the preceding two decades and warned: “We need to consider some uncomfortable truths.”

“Did we have the right strategy? Did we have too many strategies?” he asked. So much faith had been put in nation-building that the sudden collapse of the army and the government, and the panicked flight of President Ashraf Ghani “took us all by surprise”. “It would be dishonest to claim otherwise,” Austin said.

Milley said the US military would be trying to learn the lessons of the Afghan failure for a long time to come, but he outlined some of his initial thoughts.

One of his lessons, the general said, was not to give the enemy a fixed date for your departure as the Doha deal had done, but base it on a set of conditions. Another of Milley’s conclusions was “don’t Americanise the war” and don’t try to build another country’s army as the mirror image of US forces.

But as Milley himself conceded, these were lessons that were supposed to have been learned in Vietnam. The Biden White House had been anxious to avoid parting scenes reminiscent of Saigon in 1975 but that is exactly what happened, in part because those lessons had been forgotten.

The deeper question – never likely to be answered, or even asked, in the bearpit of Congress – is why the US has suffered from such recurrent amnesia each time it has gone off to war.