Shortly before midnight on 8 November 2016, Hillary Rodham Clinton dozed off in the bedroom of her Manhattan hotel. The presidential candidate was exhausted from the rigours of a bruising election campaign and rattled by a flurry of early results that suggested the verdict was far from a foregone conclusion. She closed her eyes as the slight favourite to become the US’s first female president. She opened them to a nightmare and a world turned upside down.

Clinton’s shock loss to Donald Trump – the embodiment of chaos to her bastion of control – left so much destruction in its wake (a victory party mothballed, shell-shocked staffers out of work, 66m futile votes) that it was easy to overlook the small matter of 2,000 hours of behind-the-scenes campaign footage, shot in a spirit of cautious optimism but now left to languish. Clinton’s office suggested cobbling it together as an official record, an insider’s account of what went wrong. But film-maker Nanette Burstein recoiled when the idea was put to her. “Too soon,” she says now. “Too raw. Too disturbing for the public. I know I wouldn’t want to watch that myself, never mind make it.”

Instead, Burstein received clearance to use the footage for a bolder, more wide-ranging project. Hillary, the result, is a four-part biography that frames Clinton’s life against the arc of the women’s movement and recasts her career as a series of giant leaps and bounds. Significantly, it ends not in despair but in hope. Clinton, insists longtime adviser Cheryl Mills, is so much more than another stumbling pretender to the throne. She’s a mould-breaker and a risk-taker. “The tip of the spear – the person who blazes the trail.”

If this makes it sound as if Hillary comes to praise Clinton not bury her – well, that’s broadly the case (“largely flattering”, remarked the Hollywood Reporter). Yes, the documentary is at pains to cover its subject’s myriad missteps and controversies, be it the lucrative Wall Street speeches, the Whitewater real estate investment affair or the overcooked saga of her private email server. But the talking heads are effectively defence witnesses, solid representatives of blue state USA. We hear nothing from Newt Gingrich, Sean Hannity, Lindsey Graham, or indeed any of her tormentors from the other side.

This, Burstein says, was not for want of trying. “But Hillary is so toxic in that community that any association is seen as bad. I made it clear that it was an objective film and that I wanted their point of view. But they also knew that she was participating, which meant that it wasn’t going to be some rightwing take-down of her – and anything short of that was unappealing. There was no upside, it was just a risk. All of them right away said no.”

In the event, the director spent a week at the Clintons’ home in Chappaqua, a hamlet 30 miles north of New York City. She sat with Hillary for 35 hours of interviews, going back over her various – and often conflicting – incarnations as lawyer, first lady, senator and globe-trotting secretary of state. Perhaps most compellingly, she sat with Bill, who brokenly recalls his cataclysmic affair with Monica Lewinsky, likening himself to a weary boxer who grabs for something, anything, that might momentarily distract him from the fight. “I’m a totally different person now than I was 20 years ago,” he insists – and the physical evidence would appear to bear him out. The former president, arguably the consummate politician of his age, now looks hollowed out, diminished and somehow older than his years.

As for Hillary, who can say for certain? “She’s not a confider,” one friend needlessly points out. Instead, she sits ramrod straight in her book-lined sitting room, sipping coffee as make-up artists periodically dip in to attend to her hair. She’s candid, impressive and radiates a certain blunt warmth. But on a personal level, she’s as guarded as Fort Knox. Every question is a potential hazard. It needs to be quickly checked for tripwires and trapdoors.

There is, of course, good reason for this. With the possible exception of Trump, no US politician inspires such fear and loathing. She’s like a bespoke trigger for the red state white male. There they are – the MAGA army, the deplorables – burning her effigy at a rally in Kentucky and hollering “Iron my shirt!” inside the town hall meet-and-greet. “Smile!” orders a shaven-head in camouflage pants as the candidate walks by – and, ever the pro, Clinton duly obliges.

But it’s small wonder that her smile is a firewall and her laugh flak. She’s hemmed in on both sides, damned either way, regarded as an establishment neocon by the left and a radical feminist harpie by the right. Depressingly, these prejudices sometimes hop the fence: Clinton’s decision to remain in her marriage only proved how weak she was, right up until the moment it proved that she was Lady Macbeth, a cold-blooded careerist. “I’m the most investigated innocent person in America,” she snorts at the end of one interview session, when she perhaps believes herself to be off-mic.

Burstein previously co-directed The Kid Stays in the Picture, a spry documentary about the Hollywood producer Robert Evans. She freely admits to being a Clinton supporter and feels that most of her problems were caused by forces beyond her control, a reflexive sexism in the land at large. But Burstein does believe that, occasionally, Clinton could have handled herself better. Maybe relaxed her defensive lawyerly crouch, or eased back on the well-paid corporate speaking gigs – for the sake of her public image, if nothing else. 

“If she has a flaw – and she has a few – it’s the optics. Maybe she’s not in the pocket of Wall Street, but to give speeches at a time when the banks are being bailed out, all the while knowing she’s going to run for president – it looks bad. And she has this self-righteous quality that is her blind spot. She thinks, ‘Well, I’m not doing anything wrong, no one should think I’m doing anything wrong, so who cares what it looks like?’ And that’s what gets her into trouble again and again.”

Bill, no doubt about it, was more adept at rolling with the punches, charming his way out of trouble, while Barack Obama’s special power was rising nobly above the fray. But it’s a fools’ game to use past giants as a kind of roadmap to the White House. Both men (each pioneers in their way) still had to contend with a different set of hurdles to those faced by Hillary. And both, finally, could afford to be looser, less apologetic and more at ease with who they are.

“Greatness of spirit is the most important quality that a politician can have,” says Joe Klein, one-time Time magazine columnist and bestselling author of Primary Colors, a loosely fictionalised novel about Bill’s 1992 presidential campaign. “The successful Democrats – Franklin Roosevelt, Harry Truman, John F Kennedy, Bill Clinton, Barack Obama – have all had it. To have greatness of spirit, you have to be optimistic and you have to at least give the appearance of openness.” Hillary, he concludes, “was never going to be that”.

Burstein, thank heavens, spies the obvious problem with this. “It’s a very male thing to say. I mean, OK, maybe there’s some truth to the idea that it’s hard to pick a woman who has the kind of charm or the greatness of spirit that you see in a man. But is that because it doesn’t exist? Or is it because it comes in a different package that we don’t know how to recognise?”

Undeniably, Hillary offered a different package to the one presented by Donald Trump, the bait-and-switch conman who now sits in the Oval Office, presiding over a 100,000 Covid-19 death toll and an economy in the toilet. For all that, Burstein chooses to end her series not in the ash and rubble of November 2016, but amid the green shoots that have sprung up since. She spotlights the 5 million-strong Women’s March that followed the inauguration, the 2018 midterms that turned the lower house blue and the vibrant attempts of Elizabeth Warren and Kamala Harris to run for president. Obviously, it would have been heartening if one had secured the Democratic nomination. But can’t narrow defeats and near misses be markers of progress, too?

“Well, yeah,” Burstein says. “Because change happens incrementally. The fact that all those women ran for president in and of itself is a triumph. What happened in the midterm elections – that’s a triumph as well. Because this is how it works. The more you see it happening, the more you get used to it. And the more you get used to seeing women in positions of power, the more likely you are to one day vote for it.”

Clinton forged the path. Many others now walk it. Two steps forward, one step back.

• Hillary is on Sky Documentaries and Now TV from 11 June.