The fall of Kabul was inevitable. It marks the end of a post-imperial western fantasy. Yet the west’s reaction beggars belief. Call it a catastrophe, a humiliation, a calamitous mistake, if it sounds good. All retreats from empire are messy. This one took 20 years, but the end was at least swift.

The US had no need to invade Afghanistan. The country was never a “terrorist state” like Libya or Iran. It was not at war with the US; indeed the US had aided its rise to power against the Russians in 1996. The Taliban had hosted Osama bin Laden in his mountain lair through his friendship with the Taliban leader, Mullah Omar. At an immediate post-9/11 “loya jirga” in the southern city of Kandahar, younger leaders pressed the mullah to expel Bin Laden. Pakistan would probably have forced his surrender sooner or later. After the 2001 invasion the US defence secretary Donald Rumsfeld demanded that George Bush “punish and get out”.

Yet neither Bush nor Tony Blair listened. Instead they experienced a rush of blood to the head. They commandeered Nato, which had no dog in the fight, and began “nation building”, as if nations were made of Lego. It would be an age, said the political scientist Joseph Nye, of the “velvet hegemon”. For reasons never fully explained, Blair declared a “doctrine of international community” and pleaded for Britain to be in the first bombing run over Kabul. He then sent Clare Short as the minister for international development to stop the Afghans growing poppies. Afghan poppy production soared to an all-time high, spreading from six to 28 provinces, probably Britain’s most successful farm product of all time. Opium floated the Taliban back to power.

When I visited Kabul in 2006, I had heard nothing but bombast about what already seemed a doomed venture. A British army of 3,400 volunteered to suppress resurgent Taliban rebels in Helmand. The defence secretary John Reid promised that only “remnants” of the Taliban remained and that “not a shot needed firing”. His general, David Richards, said it would be “just another Malaya”. Seven year later, British troops left defeated and the Americans took over before also being defeated. The local Pashtun are masters at humiliating outside powers.

From then on, retreat was only a matter of time. What is happening now is ghastly. Twenty years of dependency on lavish western taxpayers means that soldiers, interpreters, journalists, academics and aid workers are seeing friends threatened and killed. Years of assistance and training is at risk. A reputed trillion dollars of American money has been wasted. Britain alone has wasted £37bn.

How many times must it be drummed into British heads that the British empire is over? It is dead, finished, outdated, not to be repeated. Yet Boris Johnson has just sent an aircraft carrier to the South China Sea. Britain has no need, let alone right, to rule other countries, to “make the world a better place”. No soldier need die for it, let alone 454 British soldiers and civilians in Afghanistan. The best Britain can now do is establish good relations with a new regime in Afghanistan – in liaison with Kabul’s neighbours Pakistan and Iran – to protect at least some of the good it has attempted to do this past 20 years. The world is not threatening Britain. Terrorism does not need state sponsors, nor will it be ended by state conquest.

• Simon Jenkins is a Guardian columnist