Common sense suggests that America ought to reform its ancient constitution. The country, after all, is vastly different from what it was when founded in the 1780s and 1790s. The electoral college may have made sense at the dawn of the democratic age, but now it is an embarrassment, violating the core principle that every vote in presidential contests ought to count the same as any other.

Having had no experience with the mass democracy they called into being, the framers of the constitution gave little thought as to how best to keep monied interests from corrupting electoral outcomes. And they had no clue about how questions of sex and sexuality would one day convulse their republic. Constitutional amendments passed today could abolish the electoral college, curtail the influence of private (and especially dark) money on politics, and establish a right to an abortion or a broader right to privacy in matters sexual and otherwise.

When we ask, however, whether any of these amendments have a reasonable chance of becoming law, the answer is no. The explanation is as mind-boggling as it is straightforward: For all intents and purposes, the constitution cannot be changed. The framers set an impossibly high bar for revision: two-thirds approval for a proposed amendment from each House of Congress, followed by majority approval from three-quarters of the state legislatures. Imagine a vote for Brexit crossing that double threshold. It never would.

The US constitution has been amended a mere 27 times across its 230-year history. The meaningful total is actually far less. The first 10 “Bill of Rights” amendments should not be regarded as amendments, since they were part of the original debate and ratification of the constitution in the years from 1789 to 1792. The three civil war amendments (1865-1870) were passed in unique circumstances of internal war, secession, and reconstruction. Two Prohibition amendments that canceled each other out (the first authorized a ban on alcohol and the second repealed it 14 years later) inflate the official count. A few other amendments addressed matters too minor to discuss. The total number of significant amendments passed in non-civil war circumstances, then, rapidly shrinks to single digits: about one every 25 to 30 years. Only during the Progressive era (1900-1920) did Americans find a way to make amendments a useful tool of politics: the direct election of senators, women’s suffrage and Congress’s right to levy income taxes were all written into the constitution at this time. No prior or subsequent generation has figured out how to duplicate the Progressives’ success. Even Antonin Scalia, the great believer in the genius of the constitution as it was originally written, admitted that a constitution written in stone was not serving anyone well.

The unchangeability of the constitution is not a new problem, of course. Liberal and conservative jurists across the generations have creatively refashioned the constitution into new shapes to address new realities. Consider Louis Brandeis, who insisted that the constitution be treated as a living document whose principles needed to address matters “of which our fathers could not have dreamed”. Twentieth-century judges, Brandeis believed, were obligated to adapt 18th-century principles to novel circumstances and, occasionally, to discern in those principles as yet unenumerated rights. To think otherwise, Brandeis declared, would be to turn the constitution into a series of “impotent and lifeless formulas”.

If the supreme court sometimes sought and achieved moments of Brandeis-style brilliance, it also suffered through periods of hubris or brittleness when justices, in pursuit of a political agenda or a misguided sense of principle, forgot where the ultimate source of their authority lay: not with the statutes themselves, or with framers of the constitution, but with the American people.

Between 1789 and 1791, large assemblies of citizens in nine of the 13 states voted both to ratify and modify the document that the framers had handed them. This ratification process gave meaning to the critical preamble to the constitution: “We the people of the United States … do ordain and establish this constitution for the United States of America.” The supreme court must sometimes rule against majority opinion, which can be ill-considered, even tyrannical. But if the court repeatedly ignores or, worse, displays contempt for deep-seated and enduring popular convictions, it risks not just its own authority but that of the entire governing system of which it is part.

Two historical examples illustrate this point. The first was the notorious Dred Scott decision of 1857, when Chief Justice Roger Taney and a large majority of justices declared on specious grounds that African Americans, enslaved or free, were not and would never be entitled to US citizenship and thus to constitutional rights and privileges. The outrage generated in the north by this decision hastened America’s descent into civil war.

The second moment occurred in the 1930s, when four conservative justices were preparing opinions to strike down two pillars of Roosevelt’s New Deal, the Social Security Act and the National Labor Relations Act. These “Four Horsemen”, as they were known, were opposed by a progressive bloc consisting of Brandeis and two other justices wishing to uphold the New Deal. In the middle sat two moderates, Chief Justice Charles Evans Hughes and Associate Justice Owen Roberts. Had one or both joined the horsemen, they might have plunged America into a second civil war, this one between capital and labor.

The scenario of war was not far-fetched. Americans had declared their support for the New Deal by giving Roosevelt a resounding election victory in the 1936; they would not have tolerated the supreme court frustrating the will of the people by striking down the New Deal.

To save his legislative program, Roosevelt was threatening to push through Congress a law that would allow him to “pack” the court with his own appointees. Meanwhile, members of the United Auto Workers had occupied several General Motors factories in Michigan, forcing one of the world’s most powerful corporations to shut down production. Staying for six weeks, the “sit-down” strikers dared mayors, a governor, judges, and a president to call in the police, national guard, or US military to evict them.

At this moment of industrial confrontation and looming political crisis, both Hughes and Roberts signed on to two critical decisions that secured FDR’s New Deal. Roberts insisted in subsequent years that jurisprudential evolution, not political pressure, had shaped his decision. Hughes struck a different pose. He seemed to understand that the judiciary, though independent, was part of a political system established to make the people sovereign. And that at certain crucial moments, the will of the people had to be honored. If this could not be done by constitutional amendment, it would have to occur through some other means.

The supreme court today faces another critical test of its legitimacy, as it prepares to deliver pivotal rulings this year on abortion, gun rights, and government funding for religious schools. It is likely that important “right to vote” cases will soon come before the court as well. The court must render its rulings in circumstances that have already seriously damaged its reputation. I am referring, of course, to the true steal in American politics: not the presidential election of 2020 but Mitch McConnell’s hijacking of two supreme court appointments to achieve the GOP’s 40-year quest for an impregnable conservative majority. The beneficiaries of that steal – associate justices Neil Gorsuch and Amy Coney Barrett – have given conservatives their largest majority on the court in 90 years.

Will this court, and its swollen Republican majority, succumb to the Taney temptation in Dred Scott, and attempt to settle divisive matters once and for all in ways that suit the wishes of their most fervent supporters? Or will the court follow the Hughes path and recognize that this is a moment when considerations of the American people’s “general welfare” must enter judicial deliberations?

Chief Justice John Roberts has shown himself to be a Hughes man, able to put country before party (as he did in his critical vote upholding the Affordable Care Act). But McConnell’s machinations have removed control of the court from Roberts’s hands. Clarence Thomas, Samuel Alito, and Neil Gorsuch seem implacable in their conservatism. The progressive caucus of Stephen Breyer, Sonia Sotomayor, and Elena Kagan is too small to accomplish anything on its own, even with Roberts as a sometime ally. That leaves the future of this court in the hands of Barrett and Trump’s third appointee, Brett Kavanaugh. Does either have the integrity or vision to move the court and the country to a better place? We shall see.

• Gary Gerstle is Mellon Professor of American History at Cambridge. His new book, The Rise and Fall of the Neoliberal Order, will be published in April. He is a Guardian US columnist