Socialism used to be a dirty word. Is America now ready to embrace it?

In his 1998 film, Bulworth, Warren Beatty played a Democratic senator who goes rogue, embarking on a truth-telling binge in which he spouts radical truths no Democrat serious about election would ever utter. At one stage, Bulworth even mentions socialism.

“In America that’s like saying cocksucker,” Beatty once told me with a chuckle over dinner at his home in Los Angeles shortly before the film came out in Britain a year later. “We have this so-called thriving economy which has missed most people and while the disparity between rich and poor increases we have just one party – the money party, made up of Republicans and Democrats.”

When I suggested to him he sounded like a socialist he swatted the term away.

“I’m interested in a government that looks out for people who need to be looked out for,” Beatty said. “Ideology seems to be so unfashionable, so why not take advantage of it and not name oneself with a term that has become particularly problematic?”

Polls vary on how Americans feel about socialism now. In May, Gallup found that 43% thought some form of socialism would be good for the country, putting socialism at a statistical tie with Trump, who’s approval ratings were 42%. The term was particularly popular among the non-white and the young. A Harris poll released a couple of weeks earlier found that only 24% said they would vote for a socialist. An NBC poll indicated “socialist” was the least attractive trait voters were looking for in a president, significantly lagging “someone over the age of 75” and “a Muslim”. A Harris poll from March suggests half of those under the age of 40 would “prefer to live in a socialist country”. Three-quarters of Democrats believe the country would be “better off” if it were more socialist.

But quite what people mean by “socialist” is an open question.

“The way I translated it to people was: you shouldn’t have to choose between paying for prescription and paying for groceries,” explains Sarah Innamorata, the socialist Pennsylvania state representative I met in Pittsburgh, who defeated a five-term incumbent who had been elected unopposed the last three times. “If you work for 40 hours a week you deserve to be able to support yourself and your family. And when you go outside you should be able to breathe clean air and turn on the faucets and get clean water. And really none of that is going to change unless we change who represents us and we change the way our government works.”

To some extent, the views on socialism say as much about how people are feeling about capitalism. This is not a new trend. When President Barack Obama was planning his run for a second term his pollsters noticed that the time-honored rhetorical appeals to a life of relentless progress and upward mobility weren’t really working.

“The language around the American dream wasn’t carrying the same resonance,” Joel Benenson, one of Obama’s key pollsters, told the Washington Post. “Some of the symbols of achieving the American dream were becoming burdens: owning that house with the big mortgage was expensive, owning two cars and more debts, having your kid go to college. The cost and burden of taking out those loans was making a lot of Americans ambivalent. They weren’t sure a college education was worth it.”

Over the last 40 years wages have stagnated in real terms while the price of college has risen eight times as fast and the price of health insurance has also outpaced earnings.

At a local level it is also a response to a sclerotic democracy riddled with entitlement, nepotism and corruption. In south-west Pennsylvania, Innamorata beat Dom Costa. His extended family had dominated local politics for several years. One of his cousins, Paul, was defeated in his race for state representative by another socialist candidate, Summer Lee, an African American woman. The previous year another of Dom’s cousins, Ronald Costa, was beaten by Mikhail Pappas, another socialist candidate, for district judge.

In Chicago, where six socialist-backed candidates won seats to the 50-seat city council earlier this year, the same pattern of taking down a corrupt establishment was clear.

The city’s 33rd ward had been in alderwoman Deborah Mell’s family for the past 44 years. Her father Dick Mell, the father-in-law of the former Illinois governor Rod Blagojevich (currently in jail for corruption and extortion), had held the seat since 1975. When Dick retired in 2013, Mayor Rahm Emmanuel appointed his daughter as his successor. She lost the seat to a socialist candidate, Rosana Rodriguez-Sanchez. Another socialist, Andre Vasquez, beat Patrick O’Connor, who has been on the city council for 36 years and was part of the overwhelmingly white council group that fought hard to block the agenda of the Chicago’s first African American mayor, Harold Washington. All three of the aldermen who preceded Jeanette Taylor, also backed by the Democratic Socialists of America, were indicted for fraud.

It is potentially, in these local spaces that the shift to the left and the shift to the grassroots might meet. At the first ward meeting for Rodriguez-Sanchez, more than 60 people crowded into a small room, where their new representative said a few words before encouraging people to approach her or her staff with issues they would like raised. Rodriguez-Sanchez beat Deborah Mell by just 13 votes. Every last piece of organisation helped.

The Democratic Socialists played a role, explains local activist Sandy Gutstein, but so did lots of groups, including immigrants’ rights organizations and anti-racist campaigners. “A lot of people were coming into the office who have not been involved in politics before,” she said. “I think people were attracted to the campaign because of that.”

Across town, that same night, Jeanette Taylor had to choose a ward superintendent, a paid position to monitor whether local people are getting the services to which they are entitled. Usually they are appointed. Taylor held an election. “I let the community decide,” she said. “Anything that’s going to get built in this community, it will be up for the community to decide whether it gets built or not. Three people applied. We heard a presentation from all three folks. They were asked to leave. We had some discussion and then they voted.”

According to Gallup, Americans are more likely to associate socialism with “equality” than “government ownership or control”, as they did in the 40s. “I don’t think it’s about class antagonism,” explains Jesse Sharkey, the leader of the Chicago Teachers’ Union. “It’s about humane capitalism – having social control over the harshest features of capitalism like healthcare and pensions.”

The point is it’s a question that is now regularly put in polls and a label that is popularly claimed. It remains problematic for many.

But it’s no longer like saying cocksucker.

Carlos Ramirez-Rosa has been an alderman in Chicago since 2015. When he ran four years ago he “flirted with the idea of running as an open democratic socialist” but decided against it. “There were a number of people in the community who wanted to see me run but said ‘Carlos, don’t run with this group of socialists’. Because you’re going to go out there and wave this red flag and these folks who are pushing you to do that are going to stay on the sidelines.”

Ramirez-Rosa credits the shift to Bernie Sanders, the democratic socialist candidate who mounted a surprisingly effective challenge against Hillary Clinton for the Democratic nomination in 2016 and is running again. “Seeing him so openly talking about being a democratic socialist. Not running away from the label; leaning in to it.” Earlier this year he ran as a democratic socialist and won again.

“Having been involved in the work I’m not surprised at all that we have six socialists on city council. But when I take a step back and put myself in the place I was in 2015 I think, ‘Wow, that’s tremendous.’”

Rick Perlstein, a Chicago-based author who has chronicled the rise of the American right since the civil rights era, believes some of socialism’s appeal comes from Republicans using it as an epithet against Barack Obama in the midst of an economic crisis. “I think a lot of people felt, well if that’s what socialism is then how bad can it be?”

I met Perlstein at a packed Martyrs’ nightclub in Chicago where the young dance while older folk clap and stomp as Daniel Kahn and the Painted Bird sing about gentrification, toxic masculinity and economic inequality. Flitting between Yiddish and English Kahn draws on pre-second world war leftwing Jewish traditions, adapting songs written by socialist Bundist leaders of yore.

The event, organised by Chicago Yivo, which aims to promote the cultural history of Jewish life in eastern Europe, Germany and Russia in the city. “It’s an alternative to a Zionist identity,” says Karen Underhill who organized the event. “It’s not a total rejection of it but it tries to move beyond the two pillars of Jewish identity – Zionism and the Holocaust – by embracing left Jewish culture pre-world war two.”

Shtumer Alef, the yiddish punk support band for Kahn, warmed the crowd up by getting them to sing along in Yiddish to: “Remember to always punch Nazis.”

“Trump has politicised everything,” says Lorrie, who sings in the band, as she pops out for a cigarette. “People talk about politics all the time. But the Democratic establishment keeps failing upwards. That’s why they are so scared of someone like Alexandria Ocasio-Cortez. She’s a powerful symbol of a different way of doing things.”

• This article was amended on 10 September 2019 because an earlier version misnamed Karen Underhill as Kate Underhill. This has been corrected.