President Barack Obama and President-elect Donald Trump once sat together in the Oval Office. “I was immediately struck by Trump’s body language,” wrote journalist Jon Karl in his memoir Front Row at The Trump Show. “I was seeing a side of him I had never seen. He seemed, believe it or not, humbled.”

It was November 2016 and, just for once, Trump was not in charge of the room, Karl recalls. Obama was still president, directing the action and setting the tone. His successor “seemed a little dazed” and “a little freaked out”. What the two men discussed in their meeting that day, only they know.

But what became clear in the next three and half years is that Obama remains something of an obsession for Trump; the subject of a political and personal inferiority complex.

Observers point to a mix of anti-intellectualism, racism, vengeance and primitive envy over everything from Obama’s Nobel peace prize to the scale of his inauguration crowd and social media following.

Ben Rhodes, a former Obama national security aide, tweeted this week: “Trump’s fact-free fixation on Obama dating back to birtherism is so absurd and stupid that it would be comic if it wasn’t so tragic.”

“Birtherism” was a conspiracy theory that Trump started pushing in 2011 (“He doesn’t have a birth certificate. He may have one but there is something on that birth certificate – maybe religion, maybe it says he’s a Muslim, I don’t know.”) . Nine years later, he has come full circle with “Obamagate”, which accuses his predecessor of working in league with the “deep state” to frame Trump for colluding with Russia to win the 2016 election.

There is zero evidence for this claim. Indeed, a case could be made that the supposed “deep state” did more to help Trump than hurt him when the FBI reopened an investigation into his opponent, Hillary Clinton, just before election day. When questioned by reporters, Trump himself has struggled to articulate what “Obamagate” means. Ned Price, a former CIA analyst, dubbed it “a hashtag in search of a scandal”.

But his allies in the Republican party and conservative media are stepping up to build a parallel universe where this is the big story and Obama is at the center of it. Sean Hannity, a host on Fox News, demanded: “What did Barack Obama know and when did he know it?” Over the past week, the channel’s primetime shows have devoted more coverage to the bogus crimes of “Barack Hussein Obama” than to the coronavirus pandemic – and Trump’s mishandling of it.

Tara Setmayer, a former Republican communications director on Capitol Hill, said: “Donald Trump always need a foil. This riles up his base because they cling to anything that diverges responsibility for anything from Donald Trump over to someone else. And in this case Barack Obama is the boogeyman of the month.”

Beyond political expediency, there is a more profound antipathy at work. From the Iran nuclear deal to the Trans Pacific Partnership, from environmental regulations to the Affordable Care Act, Trump has always seemed to be on a mission to erase his predecessor’s legacy. With few deep convictions of his own, Obama became a negative reference point for Trump. Between 22 November 2010 and 14 May 2020, he tweeted about Obama 2,933 times, according to the Trump Twitter Archive.

There are a few reasons, argues Setmayer, host of the Honestly Speaking podcast. “First off, Donald Trump has a problem where I think he’s just jealous of the fact that President Obama is still so admired. Number two, I think he has a problem with people of color who are in authority that don’t do the kind of song and dance that he wants them to do.

“Barack Obama is not a ‘shuck and jive’ person of color, and those are the kinds of people that Donald Trump seems to be attracted to if you look at who he surrounds himself with as far as minorities are concerned.”

Third, Setmayer points to the 2011 White House Correspondents’ Association dinner, where Trump sat stony-faced and humiliated as Obama lampooned the Celebrity Apprentice host’s nascent political ambitions. Obama even pointed to a photoshopped image of a Trump White House with hotel, casino, golf course and gold columns.

“A lot of people think that this is where this all started,” Setmayer continued. “President Trump does not have a sense of humor, he’s not self-deprecating, and the White House correspondents’ dinner is a fun event where people make fun of each other, especially in politics.”

Rashad Robinson, president of Color of Change, a civil rights advocacy group, said: “This obsession, of course, is absolutely rooted in racism. Some of the accusations have been deeply racialized, from the questioning of Obama’s intelligence to talking about how much basketball he plays to questioning his birthplace and citizenship.”

Trump has shredded many norms, including that of presidents maintaining a respectful contact with their predecessors. He has dismissed the idea of seeking Obama’s input during the coronavirus pandemic. For his part, Obama has carefully chosen his moments to condemn certain decisions or policies without mentioning Trump by name.

But tensions flared last week when a tape leaked of Obama on a private conference call with about 3,000 alumni of his administration, describing Trump’s leadership in the pandemic as “an absolute chaotic disaster”. He also warned a justice department move to drop charges against Trump’s former national security adviser Michael Flynn, who admitted lying to the FBI about his conversations with the Russian ambassador during the presidential transition, indicates that “the rule of law is at risk”.

Trump has described Flynn as a wronged “hero” and argued that Obama and his vice-president, Joe Biden, the presumptive Democratic nominee for November’s election, should “pay a big price” for supposedly derailing the retired general’s career. Critics suggest that the president is seeking to weaponise the justice department for electoral gain.

Matthew Miller, a former director of the office of public affairs at the department, said: “In terms of any real action against Barack Obama, he obviously doesn’t have anything to worry about. But when you look at what’s happened at the justice department with the complete politicisation of that department, I think it’s quite possible that they’re going to be coming after people from the Obama administration, using the criminal justice process any way they can.”

It would be one of the gravest consequences of Trump’s Obama obsession. Miller added: “There’s some racism there but, most of all, it’s driven by the fact that Obama has the thing that Trump has always craved but never achieved, and that’s respect. I’ve always thought that the respect that Barack Obama gets from people in this country and around the world is something that just eats Trump alive inside.”

Obama issued a tweet on Thursday that contained one word: “Vote.” He is expected to campaign vigorously for Biden, wooing voters who crave a return to what they saw as the dignity and stability of his era. But his presence is also likely to be inverted by Trump to rally his base with dark warnings that, like Clinton before him, Biden would effectively represent a third term of Obama. The 2016 rally chants of “Lock her up!” might be replaced by “Lock him up!”

The 2020 election could yet turn into a final showdown between Obama and Trump, even if only one of their names is on the ballot.

It will be a clash of opposites: one a mixed-race cerebral lawyer who has been married to the same woman for nearly three decades and publishes annual lists of his favorite books; the other a white billionaire and reality TV star who wed three times and measures success in TV ratings. Where one is renowned for elegant turns of phrase and shedding tears after mass shootings, the other serves up jumbled word salads and schoolboy spelling errors and has struggled to show empathy for the coronavirus dead.

Michael D’Antonio, a political commentator and author of The Truth About Trump, said: “There’s so much that separates them, it’s hard to imagine two presidents more different. It’s very obvious Trump is continually comparing himself with Obama in his own mind. Obama’s over his head, over his shoulder, always looming as the guy who could speak in paragraphs and juggle more than one thing at once and deal with them effectively.”