Some of America’s most experienced diplomats, politicians and former generals have been saturating the airwaves in the aftermath of the Taliban’s capture of Kabul issuing dire warnings of what may come next. Many are advocating, even in this final hour, to abandon the withdrawal and return to combat against the Taliban. Such alarmist rhetoric from our senior-most figures does much to explain why America has failed so spectacularly in this war: some of the most influential voices over the past 20 years possess an unhealthy and irrational lust to use war as the first option to solve every foreign problem.

Before 2001, Americans generally thought they were immune from foreign attack. Those beliefs were shattered with the shocking images of the twin towers falling and the Pentagon smoldering on September 11th. In the aftermath, fear and anger descended over the US population. Political leaders and senior military commanders sought to look and sound tough to calm a jittery people.

On 20 September 2001 President Bush sought to calm the American people with a speech before a joint session of Congress. “Tonight, we are a country awakened to danger and called to defend freedom. Our grief has turned to anger and anger to resolution,” he said. That resolution, Bush went on to explain, was for the Taliban to turn over “every terrorist” in Afghanistan “or share in their fate”.

It’s what he said next, however, that set the stage for the forever war that would consume 20 years and the bodies of hundreds of thousands of people in the United States and around the world.

“Our war on terror begins with al-Qaida,” the president confidently declared, “but it does not end there. It will not end until every terrorist group of global reach has been found, stopped and defeated.” The congressional chamber erupted into thundering applause in response. No one knew it in that euphoric moment, but those words were to commit the United States to prosecuting a war that couldn’t be won.

Those objectives were flatly impossible to accomplish.

There are tens of thousands of those the United States considers a “terrorist” around the world on any given day. They are scattered throughout every corner of the globe. They are shadowy figures, hidden from obvious view, often not known by even their own family members. To set a standard of finding and defeating “every terrorist group” is a physical impossibility. Yet that was the initial goal Bush articulated and what led to the first blow of what would become known as the “war on terror”. The initial success American troops had in Afghanistan perversely sucked our troops even deeper into an unwinnable war.

There was near-euphoria in the United States when US troops, joining forces with the Taliban’s arch rival the Northern Alliance, wiped out the Taliban by the end of December 2001 and gashed al-Qaida so badly that they have never regained their former strength. That was the first opportunity to take the win, end the war, and to genuinely have improved our national security. But Bush appears to have gotten greedy and wanted more.

In January 2002, Bush made his infamous “axis of evil” speech in which he effectively declared Iraq, Iran and North Korea as enemies of America. One year later he went to war with Iraq over spurious claims they posed an existential threat to America with non-existent weapons of mass destruction and opened the floodgates of opposition to Iran and North Korea which has resulted in a near-permanent state of hostility between our countries.

These decisions gave pre-eminence to a foreign policy school of thought which prioritized choosing military power as the tool of choice to solve any international challenge. This view remains so deeply ingrained in the Washington establishment and US media that almost any consideration of engaging diplomatically with countries we don’t like is reflexively rejected or condemned as “appeasement”. Any discussion that does not start and end with the military option is rarely given serious consideration. That is now on embarrassing display as we near the final hour of the Afghan war.

As the Washington Post’s Craig Whitlock revealed in the Afghan Papers – and John Sopko detailed every year for a decade through the Special Inspector General for Afghan Reconstruction reports – American leaders have known, for well over a decade, that the Afghan government was so corrupt it likely wouldn’t survive our departure and the Afghan military was incapable of fighting without the US and Nato. Yet publicly virtually every president and senior general hid these truths and continued the war.

The reality on the ground finally – as it always eventually does – tore through decades of lies and bad policy to impose itself on the world through the simultaneous collapse of the Afghan government and the final assault by the Taliban military in mid-August.

With the rapid collapse of the Afghan government constitution and existential indictment of Washington’s nation-building project, one might think those who have strongly supported perpetual war in Afghanistan would now be shamed into silence and relegated to the dustbin of history. That may yet happen, but reality has done nothing to dampen enthusiasm for military power by some in Washington.

Former secretary of defense Leon Panetta said, “our work is not done” in Afghanistan, and added, “[w]e’re probably going to have to go back in when al-Qaida resurrects itself, as they will, with this Taliban.” US senator Lindsey Graham threatened to seek impeachment of Biden if he ended the war by 31 August, and argued Biden should use the “full force” of the US military to remain beyond that date. But most illustrative of the problem in Washington is the former national security adviser HR McMaster.

Bob Woodward’s 2018 book Fear, said that upon assuming office, Trump’s “only goal was to get out” of Afghanistan. The new president, Woodward continued, “was now saying that Afghanistan was Vietnam, a quagmire with no clear national security purpose, the latest example of the incoherence of American policy”. As Trump’s national security adviser, McMaster’s job was to execute the president’s directives. But McMaster had other ideas.

Instead of subordinating his own preferences for faithfully executing the president’s policy, McMaster brought significant pressure to bear on Trump to persuade him, not only to avoid leaving, but to escalate. According to the Washington Post, in 2017 McMaster, floated the idea of “possibly sending tens of thousands of additional troops”.

Even Trump couldn’t be persuaded to ramp up that many troops, but with the secretary of defense, James Mattis, McMaster’s pressure eventually succeeded, and Trump ordered an increase of 3,000 in August 2017. “My original instinct was to pull out,” Trump said of his reversal. “But all my life, I’ve heard that decisions are much different when you sit behind the desk in the Oval Office.”

It is no surprise that McMaster has been perhaps the most outspoken critic of Biden’s withdrawal, saying last week that “it’s time to reverse course,” on the Afghan withdrawal and continue the fight. Given a chance by Chuck Todd on Sunday’s Meet the Press to admit any fault of his over the years in Afghanistan, McMaster punted, choosing instead to blame others for leaving. “We had a sustainable effort, in place,” the former general said, “that if we had sustained it we could have prevented what’s happening now.”

Ignoring the fact that the Afghan government and its armed forces disintegrated after 20 years of investment – exposing their fatal corruption – McMaster, like so many of the military-first advocates, still believes continuing a failed war was the right answer. Fortunately, the current president is ignoring the former national security adviser’s advice. Had Trump also resisted the pressure from McMaster for more war, the war could have ended in 2017 when almost all the advantages were on America’s side.

The Taliban controlled merely 6% of the country at that time; the government nearly all the rest. Then there would have been no Taliban onslaught, no immediate collapse of the Afghan government and military, and the US and coalition forces would have been able to conduct an orderly and professional withdrawal.

Men like McMaster, Panetta, and Graham – and many more like them – are, bluntly stated, addicted to war. They have become virtually incapable of seeing any solution to any foreign problem that does not include the barrel of a gun. Such views are dangerous because they not only deprive us of the ability to solve problems with reality-based diplomacy, it gets us into wars that should never be fought – and keeps us in wars that unnecessarily drain our country of blood and treasure.

The Afghan war is the first war America has militarily lost since perhaps our failed attempt to attack Canada in 1813. That is a stain on our reputation that will take generations to remove. If we acknowledge why we lost and make changes – starting with elevating diplomacy above the military-first approach – we can at least salvage something of value for America.

• Daniel L Davis is a senior fellow for defense priorities and a former lieutenant colonel in the US army who deployed into combat zones four times. He is the author of The Eleventh Hour in 2020 America