California report highlights benefits and risks of generative AI use by state agencies
OAKLAND, Calif. - Two months after Gov. Gavin Newsom issued an executive order requiring state agencies study generative artificial intelligence, a new report released Tuesday focused on the technology’s benefits and risks.
The 34-page report is the first of several expected by California’s AI task force aimed to proactively prepare for GenAI’s expansion and increased use.
GenAI is rapidly evolving and the state is trying to keep up by looking for ways agencies can improve government services and the lives of Californians.
"We’re taking a nuanced, measured approach," Newsom said in a statement. "Understanding the risks this transformative technology poses while examining how to leverage its benefits."
The report outlines opportunities but also potential harms, including widening existing disparities and amplifying biases.
"A lot of that bias gets backed into the technology," said Prof. Roxana Daneshjou, an AI researcher with the Stanford School of Medicine. "It’s built off of large volumes of human created data. We need to think about how things might impact vulnerable populations."
That data, the report suggests, could influence decision-making within state systems. Chatbots can spit out incorrect information or provide inconsistencies and encourage bad actors to spread misinformation or interfere with privacy.
"We can’t assume the technology is going to make things better without actually testing it," said Daneshjou. "I don’t feel these models are ready for any kind of standalone decision-making. There are real world consequences for getting this wrong."
The state study shows GenAI does pose safety and security risks requiring security controls, monitoring and validation techniques to guard against any attacks.
Despite that, the report said "this technology offers possibilities to improve the lives of Californians."
Jon Gacek, general manager of AI company, Veritone said regulation is necessary along with transparency, audit standards and working to make models free from bias or human influence.
"We’ve been helping solve some of the problems that are described in the report for a while," Gacek said. "GenAI makes humans more efficient, more thoughtful and in a way, gives you more superpowers to do things."
Some benefits to the state embracing GenAI include summarizing policies in plain language, converting and translating documents into other languages, and communicating with obsolete systems and computer code, according to the report.
The state also said the technology is capable of providing interactive tax assistance or unemployment support.
However, the report stresses personal information should never be plugged into programs like OpenAI’s ChatGPT and text, audio and video from AI systems should never be used verbatim.
"Putting some guardrails around it is great, but once everybody gets in and starts using it, I think it will move faster and people will understand it better," said Gacek. "It’s not scary. It’s actually quite fun."
Under Newsom’s executive order, state agencies are getting input and working to develop guidelines for the state’s use of GenAI. That work is expected to be completed by January 2024.
Brooks Jarosz is an investigative reporter for KTVU. Email him at brooks.jarosz@fox.com and follow him on Facebook and X @BrooksKTVU