Much uncertainty still surrounds China’s balloon spying program and the other mysterious objects that have now been identified over North American airspace. One thing seems certain, however: incidents like these could too easily spiral out of control in the future, ending in disaster. Unless something is done, dangerous waters lie ahead. Unfortunately, both sides are reluctant to do what’s needed.

Let’s make no bones about it. It would be better if China didn’t spy on the United States. Any US leader would be hard pressed not to shoot down a Chinese object that the American public has seen flying over US sovereign airspace – as President Biden decided to do on 4 February. But the reality is that events like this will be more and more likely in the next decade as the United States and China bump up against each other globally. The risk of clashes in the Taiwan Strait and South China Sea have preoccupied military experts for years, but the balloon incident shows they could happen almost anywhere.

Unlike the cloak and dagger spying of an earlier age, modern intelligence collection is multi-domain, persistent and global – at least when it comes to the great powers – and everyone is racing to build up their capabilities in new areas like cyber and space, even as they deploy human spies to go after each other’s most precious national secrets. China has been growing not only its military capabilities, but also its intelligence collection platforms. Meanwhile, the US continues to grow its own capabilities. Take just one data point: US reconnaissance flights along Chinese borders are now flying multiple times a day.

This makes for a much more crowded military and intelligence operating environment, an increasingly dense jungle where the United States and China are likely to run into each other again and again in the future. When they do, there will be a serious risk of an accident or miscalculation that leads to inadvertent escalation – or even outright war.

Inadvertent escalation happens when one nation’s mistake accidentally frightens another into an attack. A pilot could go off course, a government’s internal bureaucratic process might break down, satellites or aircraft could collide, sensors or other equipment could malfunction – to name just a few possibilities. It can also happen if our leaders misjudge how important some piece of military or intelligence equipment is to the other side, target it and mistakenly unleash all holy hell.

This time, the damage seems to have been contained. This was in large part because the United States judged the threat to be low early on. But the story might have been different if US leaders had thought otherwise or if they had not been able to figure out whether or not the balloon was a threat in the first place. Containing crises in the future will be much harder if lives are lost or sensitive military and intelligence systems are damaged.

During the Cold War, the United States and the Soviet Union developed ways of reducing the risk of inadvertent escalation. Through experience, diplomacy and communication, Moscow and Washington came to understand each other’s boundaries – where, and where not, to step in the jungle. Rules of the road for certain military and intelligence activities emerged. Both sides may have broken the rules sometimes, but they were at least aware of them and this reduced the risk they would go to war by mistake. Washington and Moscow also set up military to military hotlines so that when accidents happened, they could communicate quickly and stop an escalating crisis from spiraling out of control.

Some analogous arrangements between China and the United States do exist today to prevent potential clashes at sea or in the air from inadvertent escalation. But these arrangements haven’t been working well – a problem recognized among many experts, including some in the White House. A significant part of the blame here falls on Chinese shoulders. Beijing often doesn’t follow the agreed rules and has been reluctant to use the military-military hotlines designed for emergency communication in a crisis. Last week, they refused to use the established hotline on the grounds that this was a civilian incident, even though the balloon was controlled by the Chinese military.

But it doesn’t help the situation that the US Congress has been priming for a fight. Condemning China seems to be a rare bipartisan sport these days in Washington. Beijing has brought much of this hostility on itself through its bombastic nationalist turn under President Xi, but Washington’s pugnacious mood makes it all the more likely that US leaders will overreact when the next crisis hits – especially if they misunderstand what’s really going on in Beijing because diplomacy has broken down.

Washington’s fighting mood also makes it harder to communicate US red lines and strategic boundaries ahead of time. But this is exactly what’s needed in order to establish the rules of the road that will help avoid future collisions. Getting China to engage constructively on these issues may require a more flexible US approach, including a willingness to talk not just about the security problems that worry the United States, but also the issues that most worry China.

A serious diplomatic effort to build these ties could eventually go well beyond crisis management to encompass a broader transparency agenda, information exchanges, verification, safety, arms control and other measures. In the coming years, both sides are certain to seek advantages through intelligence collection, military posturing and other moves, but each side also has a vital interest in preventing an unintended spiral of escalation that could end in catastrophe. Hopefully, this episode can serve as a reminder of the work that lies ahead and how much we stand to lose if we fail to do it.

• Christopher S Chivvis is the director of the American Statecraft Program at the Carnegie Endowment for International Peace