Still, he acknowledged some troops were doing good work fighting drug crime and said he plans to allow 100 of the roughly 360 state troops now deployed to keep working with the federal government.
"I'm trying to acknowledge there are some legitimate concerns but I'm not going to play into the hype and the politics," he told reporters before signing an executive order changing the troops' mission.
Former Gov. Jerry Brown agreed in April to deploy up to 400 troops to the border in response to a request from the Trump administration made to four border states. Brown made it clear then that California troops couldn't aid in immigration enforcement, but Newsom said there's been a "gray area" in their duties.
Maj. Gen. David Baldwin of the California National Guard said the troops have not participated in immigration detention but some are conducting camera surveillance that could inadvertently aid in immigration enforcement.
Newsom's rebuke of Republican President Donald Trump's administration came on the eve of Newsom's first state of the state address as governor of the nation's most populous state and frequent foil to Washington. In announcing his decision to withdraw troops, the governor ratcheted up his rhetoric against the president.
"This whole thing is the theater of the absurd and California has had enough," he said.
The Trump administration hasn't commented. Trump and Vice President Mike Pence held a meeting Monday on border security with sheriffs from around the country. Sheriff Donny Youngblood of California's Kern County was in attendance, according to the White House.
The drawdown of California's troops will begin immediately but may not be completed until March 31, when the state's current agreement with the federal government is set to end.
Newsom has reassigned about 110 of the troops to beef up California's fire preparation efforts ahead of the next wildfire season and to expand the guard's counterdrug task force program. The expansion requires approval from the U.S. Department of Defense.
Newsom made clear during his campaign that he did not support the use of California Guard troops at the border. He took action about a month into his governorship because he wanted to responsibly review the issue, he said.
He initially wanted to pull all of California's troops back but said he was convinced by Guard officials that good work is being done related to combating drug trafficking.
If the Trump administration does not agree to Newsom's new terms "we'll bring the rest back," he said.
Newsom's move came on the heels of a decision by New Mexico Gov. Michelle Lujan Grisham, also a Democrat, to pull back her state's troops from the border.
Newsom's Monday order said the increase in Central American migrants crossing the border is the result of a desire to escape violence and repression fueled in part by the activities of transnational crime organizations. The California Guard's resources are best spent tackling those activities, he argued.
Texas and Arizona still have troops on the border.
The 360 troops are a fraction of the roughly 14,000 that have been deployed throughout California for various operations since 2016, according to Newsom's office.
Copyright 2019 The Associated Press. All rights reserved. This material may not be published, broadcast, rewritten or redistributed.