In the months after the 11 September 2001 terrorist attacks on New York and Washington, Americans were shaken by “a sudden sense of vulnerability”, Evan Osnos writes. There was an eagerness, in those early days, to avoid the divisiveness of the Vietnam era. So much for that. In 2003, President George W Bush invaded Iraq and polarised the homeland. The novelist Norman Mailer warned of “a pre-fascistic atmosphere in America” and suggested that democracy was “a condition we will be called upon to defend in the coming years”.

Twenty years after 9/11, and eight years after his return to the US from reporting tours in the Middle East and China, Osnos, a staff writer for the New Yorker magazine, grapples with Mailer’s prophecy in his new book Wildland: The Making of America’s Fury. The title is explained in the prologue, which quotes from Chinese revolutionary Mao Zedong: “A single spark can start a prairie fire.” The image of a landscape primed to burn haunted Osnos, living and working in Donald Trump’s Washington. He came to understand it as a parable for a time in American history “when the land and the people seemed to be mirroring the rage of the other”.

The author’s years away gave him a fresh gaze on the US, and his antenna is finely tuned to the way America is perceived abroad. Speaking from a friend’s basement via Zoom, he recalls a conversation with a neighbour in Beijing. “This retired factory worker had never been out of the country but watched the news every night on Chinese state television. I told her: ‘We’re moving back to the US,’ and she said, ‘Ah, the US. Be careful because it is a very prosperous country but everybody has a gun.’”

As a reporter working in various authoritarian countries, it was not Osnos’s job to be a flag-waving salesman for American principles – democracy, progress, the rule of law, trust in empirical facts – but his mere presence conveyed the message implicitly. Then, he says, “I came back to the US and really found those principles under threat, even before Donald Trump was in office.”

On Osnos’s first day back at the New Yorker in 2013, the US government shut down, the latest political stunt by hardline Republicans determined to thwart President Barack Obama at all costs. “I remember calling the White House and getting voicemail, and if there is any more distinct metaphor for a country that is sleepwalking through a period of profound global competition, that was it. It really was that first day back at work where I began to say what has gone wrong here?”

So Osnos went back to three places from his past – Greenwich, Connecticut, where he grew up from the age of 10; Clarksburg, West Virginia, where he moved at 22 to work as a newspaper photographer and Chicago, Illinois, where he interned at the Chicago Tribune – in an effort to trace the roots of the present malaise.

Greenwich was a privileged town intimately associated with moderate Republican politics, personified by Prescott Bush, the father and grandfather of presidents, who believed in ideas such as raising taxes to pay for science education and research. So when Trump won the Republican primary there in 2016, Osnos had to reckon with how members of the party in his home town had put their faith in the brash, vulgar populist.

“I found that to be one of the areas that was most in need of elaboration as an American and as a political observer,” Osnos says. “Because the casual rendering of Trumpism around the world was, these are desperate people who have effectively pulled the fire alarm of American politics. What had really happened was that some of America’s most powerful people had made a choice to advance the candidacy of somebody that many of them would say privately was totally unfit to hold the presidency. But they made these calculations for their own personal or professional or business interests that put somebody in office who then wreaked the havoc that we recognise.”

For Osnos, it all came down to personal ambition. “We’ve created instruments on Wall Street that allow greed to take its full fluorescent form, and we’ve created systems in politics that allow somebody like [Republican senators] Josh Hawley or Ted Cruz to be able to pull together the full instruments of personal political enrichment and advance themselves at the expense of their party, their purported values and so on. One of the things that runs through this is the honing of the tools. Hedge funds are just fundamentally different from banks in the 18th century.”

This perfecting of tools is manifest in Greenwich where, despite one of the lowest crime rates in the country, walls around private houses grew from 2ft or 3ft to 6ft (“Fuck you” walls, as one local official calls them) – a product of one of the most extraordinary expansions of wealth in American history. The rich had found ways to merge businesses, cut expenses and grow stock markets. The middle and working classes were left behind.

“One of the themes that runs through Greenwich and West Virginia, and also Chicago, is the segregating power of race and class,” Osnos says. “Because it is possible if you live in pockets of American bounty to shield yourself from the full encounter with American distress. In fact, you can live your life almost perfectly insulated from it.” He cites the statistic that life expectancy for adult men in McDowell County, West Virginia, is 18 years lower than for those in Fairfax County in the neighbouring state of Virginia. “Are we then surprised that our politics are coming apart when people are living, on the most elemental biological level, more or less non-intersecting lives?”

The myth of progress in the US has been shown to be just that, Osnos says. “One of the details that became vivid to me was that, by measures of intergenerational mobility in the US, now it is harder for a child to out-earn their parents than it is for a child in China to out-earn their parents. That is in some ways so contrary to the myth that we tell ourselves in this country that it should be a five alarm fire.”

At the opposite extreme from Greenwich is Clarksburg, in the green highlands of northern West Virginia, where Osnos moved in January 1999 to intern in the photo department of the Exponent Telegram newspaper for $230 a week. When he returned two decades later, he found the town of 16,400 people ravaged by unemployment, poverty and the opioid epidemic. A one-time Democratic stronghold – the Kennedys campaigned here – had shifted decisively to Trump. “I was looking in my old apartment and the window was broken and there was a sheet over the window and it was flapping in the breeze. I just had the sense that something had gone deeply wrong in the economic groundwater.”

This is coal country. West Virginia is richly endowed, but in recent decades the profits had been going out of state, again thanks to the most innovative tools of financial engineering. Hedge funds, swooping in like vultures, were able to extract the final, juiciest morsels from a dying industry. Trump offered hope to people who felt they had nothing to lose. “Coal miners would say: I didn’t love Donald Trump, he didn’t really seem like my kind of guy, but he came here, he spoke to us, he said he was going to save this industry and what else did you expect us to do?”

West Virginia is also a prime example of the “news deserts” left behind when local newspapers are killed off by the internet. Many residents turn instead to TV cable networks such as Fox News, or rightwing groups and conspiracy theorists online. “You’ve had the decline of local news in the same period of time that you’ve had the growth of this nationalised news discourse, which in some quarters, particularly on Fox, is clearly designed to agitate, to generate fear, to promote a sense of hatred of the other,” Osnos says. As local communities become more fractured, “people go into their homes. All of a sudden they’re connected to people far away. They’ve created new identities. They no longer think of themselves as, ‘I’m from this county and I should look out for my neighbour.’ They think of themselves as: ‘I’m united in this grand project with people who are very far away.’”

Trump was defeated last year by Joe Biden, who promised to heal divisions and unify the nation – essential, in his view, for America to prove that democracy can still deliver better than autocracies such as China. That partly explains why Biden was eager to pivot away from Afghanistan before the 20th anniversary of the 9/11 attacks. But it did not prove so simple.

“I’m struck by how swiftly the Taliban swept away so much that American politicians had described as permanent exports – democracy, human rights, a standing Afghan army,” says Osnos, whose previous books include a well received biography of Biden. “Americans were never asked to engage fully in the war in Afghanistan and, in the end, the roots we put down there barely extended beyond the topsoil.”

In Wildland, two tumultuous decades in American history are bookended by 9/11 and 1/6, the latter referring to the deadly insurrection by Trump supporters at the US Capitol on 6 January 2021 that briefly disrupted Congress’s certification of Biden’s election victory. What no one could be sure of that day is whether it was the end of that era or the beginning of something new and even darker.

It depends, Osnos says now. “We are contending with a political system that became so sclerotic and out of touch with the public that one of the two major parties is participating in a delusion that 6 January did not happen or did not matter. To imagine that we’ve put the trauma of Trump behind us is a very dangerous fantasy, not only because he could be back, but because the underlying conditions which produced him are still raging.”

However, the future does not look entirely bleak. “I do find reason for some hope in this unmistakable reality of the racial disparities of the Covid epidemic and then of course the [Black Lives Matter] protests last summer,” he says. “Awareness has political potential too, and we may actually begin to look back on this as the period when we’ve just begun to recognise the full scale of the problem and to address it. But the idea that we’ve extinguished the fire is probably the most dangerous thing we could tell ourselves.”