A last-ditch effort to force through legislation that would weaken environmental protections and fast-track energy projects has failed.

Joe Manchin, the fossil fuel-friendly senator from West Virginia, had attempted to latch the controversial deregulation and permitting reforms to a must-pass defense bill – after failing to get his so-called “dirty deal” passed earlier this year.

The proposal to attach his bill to the National Defense Authorization Act (NDAA), an annual appropriations bill that will be voted on later this week, was reportedly supported by Joe Biden and House leader Nancy Pelosi.

But progressive lawmakers and hundreds of climate, public health and youth groups opposed the move to pass such consequential reforms without proper scrutiny. Manchin’s legislation would weaken environmental safeguards and expedite permits to construct pipelines and other fossil fuel infrastructure while restricting public input and legal challenges.

On Tuesday, more than 750 organizations sent a letter to the House speaker, Nancy Pelosi, and congressional leadership opposing what they call a “cruel and direct attack on environmental justice communities”. Attaching the “dirty deal” to the NDAA, which would have been one of Pelosi’s final acts as speaker, threatened her legacy and the party’s climate credibility, the groups said.

The deal was ditched – for now at least – amid mounting criticism aimed at the Democratic leadership.

Environmental groups welcomed the news, but warned the fossil fuel industry would not give up.

Ariel Moger, government and political affairs director at Friends of the Earth, said: “Manchin’s efforts to tie his dirty deal to any must-pass legislation he can get his hands on are undemocratic and potentially devastating for the planet. With momentum on the side of frontline communities, the fight will continue until the bill dies at the end of this Congress.”

Jeff Ordower, 350.org’s North America director, said: “Senator Manchin cannot get away with last-ditch efforts to push forward his fossil fuel fast tracking bill. The industry will keep trying these secretive, last minute efforts to push forward dirty deals, so we will continue to be alert and we won’t let up the fight.”

Manchin, who receives more campaign donations from the fossil fuel industry than any other lawmaker, warned of dire consequences for America’s energy security. He said: “The American people will pay the steepest price for Washington once again failing to put common sense policy ahead of toxic tribal politics. This is why the American people hate politics in Washington.”

Manchin’s bill, described by environmentalists as a “fossil fuel wishlist”, was first attached as a side deal to Biden’s historic climate bill, the Inflation Reduction Act, but was eventually thwarted after widespread opposition from progressive Democrats and civil society groups. It included limits on legal challenges to new energy projects including the 303-mile Mountain Valley gas pipeline across the Appalachian mountains that has been stalled by concerned communities and environmental groups in West Virginia and Virginia.

He and other proponents have said that fast-track permitting is needed for a rapid transition to renewables and in order to modernize the country’s outdated power transmission systems.

But Jeff Merkley, the Democratic senator representing Oregon, said Manchin’s deal was a dirty one, and had nothing to do with renewables. “This [bill] will give a whole lot more impetus to fossil fuels and run over the top of ordinary people raising concerns, that’s why it’s a dirty deal. This is a real travesty in terms of legislative deliberation, and in terms of environmental justice.”

On Tuesday, Rashida Tlaib, the Democratic congresswoman from Michigan, had called on her colleagues to stand up against the fossil fuel industry and the undemocratic manner in which leadership was trying to push through the bill without scrutiny. She said: “It’s outrageous enough that Congress wants to spend another $847bn on our military-industrial complex, the largest annual military budget in history; we cannot allow them to then ram through Manchin’s dirty deal in the process.”

The NDAA is considered a must-pass bill because it authorizes pay increases and compensation for harmed troops, as well as establishing the following year’s personnel, arms purchasing and geopolitical policies.

Environmental and climate justice groups warned Democrats that frontline communities would not forget and would hold them accountable in 2024 if the deregulation bill was pushed through.

“To think that this is happening at the hands of Democrats, and their very last action of power is going to be to hurt our communities and strip our voice is really hurtful. I feel betrayed,” said Maria Lopez Nunez, deputy director of the New Jersey-based Ironbound Community Corporation and member of the White House environmental justice advisory council.

“For any Democrat that’s listening, if you’re playing along to this charade, our community will call you out and we will hold you accountable.”

On Wednesday, Manchin launched yet another bid to garner Republican support for his bill in the Senate, in hope of getting it through as an appendage to the NDAA. It seems likely to fail.

“Nobody wants Manchin’s filthy lump of coal, no matter how many ways he tries to polish it,” said Jean Su, energy justice program director at the Center for Biological Diversity. “Speaker Pelosi and Leader Schumer need to see this dirty deal for what it is and leave it as a failed footnote in the 2022 history books.”