Legislative Report: July 1, 2022-July 1, 2023

Overview

The Office of Data and Innovation (ODI) was established on July 1, 2019, as an office within the Government Operations Agency (GovOps) and became a standalone department on July 1, 2023. This year’s legislative report showcases the progress made during the 2022-2023 fiscal year. It includes key work, resulting from two Executive Orders issued in the first quarter of the current fiscal year, which will have significant impacts on the future of Californians and our state’s workforce.

As part of our current strategic plan, ODI adopted a new mission and vision statement to reflect our establishment as a stand-alone department: 

• MISSION: “We drive innovation across California state government, using data, technology, and human centered design to achieve exceptional, equitable outcomes for all Californians. We collaborate with leaders and communities throughout California and the world. We partner to build empathy for the people we serve and create a government that is easy to interact with and solves big problems.”

• VISION: “A state government that is modern, effective, and provides exceptional service to all Californians.” 

ODI continues to build strong relationships across state government and leverage external partnerships to improve services to Californians. ODI launched projects and processes to allow for transformational work through human centered design, service innovation, and data analysis. Subject matter experts working within ODI are serving the state by leveraging multi-disciplinary skills in engineering, user research, analytics, content design, training, operations, talent, engagement and communications. 

ODI is positioned to lead statewide initiatives in partnership with the Governor’s Office, as well as collaborate on progressive solutions within departments to improve delivery of vital state services. ODI exists as a resource within the state to rethink processes and policies in consultation with the appropriate control agencies, rebuild systems, solve challenges and deliver on its commitment to empower the state’s workforce. 

To fulfill our mission, ODI is taking the following strategic approach:

• Research user needs for statewide program/policy areas;
• Develop online and in-person modern service standards;
• Build and foster a statewide culture of service innovation based on user needs;
• Streamline internal and external data access and analytics;
• Train department staff on the use of platforms and analysis best practices, including the development of training modules for leave-behinds and future training programs; and
• Create standards that allow for broader adoption, equity, and sustainability.

It is our privilege to share the results of ODI’s work in this report.

Activities of the Office 

The activities of our office fall into two main categories: statewide strategic initiatives and projects with partner departments. 

Our statewide strategic initiatives are cross-departmental efforts that require collaboration between multiple state entities, including GovOps and the Governor’s Office. This work is ODI’s opportunity to drive government transformation at scale by embedding our human-centered approach in statewide standards, policy recommendations, and statewide technology systems. 

We also partner directly with departments to perform problem definition, discovery, user research, prototyping, and piloting of innovative solutions to address pain points in their service delivery process. These projects are sourced through a solicitation process and are funded by the Data and Innovation Services Revolving Fund (DISRF), known as the Data and Innovation Fund (DIF).

Statewide Strategic Initiatives

Generative Artificial Intelligence (GenAI) Executive Order

While ODI’s work on Generative Artificial Intelligence (GenAI) began in fiscal year 2023-2024, given the significance of this initiative, ODI is sharing the following details for your awareness. In September 2023, Governor Gavin Newsom issued Executive Order (EO) N-12-23 requesting multiple departments across the state to author a first-of-its-kind benefits and risks analysis on GenAI use in state government. 

ODI was a contributor to the final report alongside GovOps, the California Department of Technology (CDT), the Department of General Services (DGS), and the Governor’s Office of Business and Economic Development (GO-Biz). This team of departments researched and reported on the beneficial use cases and potential risks of GenAI. ODI’s subject matter experts were instrumental in both the community engagement efforts, as well as the research and drafting of this historic report. The EO and its directives showcase California’s leadership in technological advances, positions the state as pioneers for equity in innovation, and establishes the guardrails and engagement needed before any consideration or use of GenAI.

Equity Standards Executive Order

Because of ODI’s expertise in human-centered design and data, our office was specifically tasked in Governor Newsom’s EO N-16-22 with developing statewide standards for collecting race, ethnicity, and other demographic data and for developing a set of service delivery standards that support equity.  

Delivering these data standards and service delivery recommendations for the EO will have lasting impacts in advancing equity initiatives across the state through better tracking and analysis of disparities, and ultimately, to better inform policies and practices. Adopting these equity best practices enables improved access for Californians to state services, promotes consistent data practices across state agencies and departments, and allows Californians to self-identify in the ways that best represent themselves.

• Demographic Data Standards
  • Race and Ethnicity Data Standard: In response to EO N-16-22, ODI is developing recommendations for a data standard to establish consistent practices and guidance around race and ethnicity data collection and analysis. This data standard would ensure that state departments have a strong foundation on which they can develop equity metrics and programs. Due to the federal government’s parallel efforts to update their race and ethnicity data standard (anticipated for release in 2024), ODI’s recommendations are contingent on the finalized federal data standard in order to ensure alignment between the two frameworks. To solicit feedback for these race and ethnicity standard recommendations, ODI partnered with GovOps in hosting community roundtables, state department consultations, and interviews with academic experts. ODI’s work in this critical area will continue in FY 23-24, focusing on alignment with the finalized federal data standard, user research, community engagement, and user testing efforts to refine the draft race and ethnicity standard to better serve California’s diverse population.
  • Demographic Data Standards: As part of the EO, ODI is developing a preliminary assessment of standards for sexual orientation and gender identity and disability data collection and analysis. Working in partnership with the Possibility Lab, ODI will develop data standard recommendations for these additional areas later this year. This effort will involve focus group interviews with subject matter experts, community engagement and testing, and literature reviews across academia, civil society, and the federal government to leverage the latest best practices in these areas.

• Plain Language Recommendation
  • Through our experience designing government services, and through our subject matter expertise in research, ODI recommends use of plain language as the foundation for equitable service delivery. 
  • Plain language increases access and usability for all: it works better with screen readers, respects the mental and emotional state of those seeking services during a time of crisis, and allows for efficiencies of scale. 
  • While both California and the Federal government each have legislation requiring the use of plain language, neither specify a target reading level, which we have identified as a critical component for achieving our plain language goals. ODI has recommended that all state entities issue documentation, both in print and digital, written at an 8th grade reading level or below. 
  • To support broader state adoption of this plain language standard, ODI has created an Innovation Hub, located on the ODI website. It includes resources on human centered design as well as guidance for state entities to implement plain language in their Californian-facing language.
  • ODI’s training arm, CalAcademy, was developed in June 2023 and began its initial planning to launch a Plain Language course in Q1 2024. The course is designed to be offered free of charge to all state staff, and is being developed to provide hands-on practice to apply the principles of ODI’s plain language standard. 

Infrastructure Website

The Federal Government has provided a once-in-a-generation funding stream to the State of California allowing the state to expedite infrastructure projects that will transform communities throughout the state. Following a $1.2 trillion financial allocation to the State of California through the Infrastructure Investment and Jobs Act (IIJA) and $391 billion over 10 years for climate-related investments in the Inflation Reduction Act (IRA), Governor Newsom issued the Infrastructure EO N-8-23. The EO creates a Strike Team to identify where climate, infrastructure, transportation, technology, and energy needs  intersect.

ODI did not receive any federal infrastructure funding. However, as part of the EO, ODI served as a key member of the Strike Team and ensured the site met the State’s plain language standard. The build.ca.gov website launched in early 2024. The new site outlines how state and federal infrastructure investments are tackling some of California’s toughest challenges, including climate change, equity and job creation.

CalAcademy

The CalAcademy program began its implementation in June 2023. The charter, currently in progress, will design professional development opportunities around human-centered design, data best practices, and other relevant disciplines within ODI to foster a culture of continuous learning and innovation across state departments. 

CalAcademy will serve as an internal and external training and resource hub, to share high-quality training content allowing ODI to test, refine and scale future learning and development opportunities. It allows for opportunities in partnership with the California Department of Human Resources (CalHR) as well as other departments through DIF engagements.

ODI’s DIF projects, by design, are largely pilots. ODI works with our partners on identifying problems, solution discovery and design, and building a solution through piloting that can be scaled and maintained by partner departments. To scale our solutions and their impact, we involve and train our state partners staff directly, so that they can build the necessary skills and capacity to drive their own initiatives in the future. This is the foundation for ODI’s planned CalAcademy training programs. In 2024, ODI will also pilot three courses on CalHR’s CalLearns Learning Management System to reach a wide group of state employees.

Data and Innovation Fund (DIF) Partner Projects

ODI is legislatively mandated to lead innovation and statewide transformation. The DIF was funded to foster collaboration between ODI and state agencies, allowing for innovative solutions that improve services to Californians. These projects empower state staff to perform their jobs more effectively, efficiently, and with improved outcomes. DIF-funded engagements are unique because of their people, process, innovation, and data-focused problem statements.

In addition to its DIF-funded engagements, ODI works in partnership with CDT, to improve state technology infrastructure through CDT’s Technology Modernization Fund (TMF) and Technology Stabilization Service. 

ODI builds pilots, tools, and resources on underlying information technology (IT) infrastructure that is managed by CDT, including any shared digital services managed by CDT’s Office of Digital Services (e.g., digital ID, web content management system, data infrastructure). ODI complements CDT by focusing on service innovation and data. Together, these three funds ensure California state departments innovate by applying human-centered design, data, and IT investments to yield quick and meaningful results.

ODI launched the first DIF project solicitation in June 2022. The first cohort of DIF included six projects. Since then, ODI has partnered with 24 state departments on 30 standalone and multi-departmental collaborative projects. These projects have improved Californians’ experience with their government,¹ increased social safety net program uptake, and increased the democratic engagement of Californians who have historically not participated in local government initiatives.²

How the Data and Innovation Fund Works

DIF is ODI’s funding mechanism for its portfolio. Applications to DIF are offered on a cohort basis and require a competitive application, partnership and leadership interviews, Advisory Committee review, and final selection by the DIF Selection Committee. 

The DIF Selection Committee prioritizes projects with the greatest potential return on investment, and ideally with the potential to scale across departments to accelerate innovation in the state. 

Through the DIF process, any California state department or agency can apply to work with ODI. The process encourages collaboration and co-creation. It begins with an application designed to elicit a high-level understanding of the problem, the impact that problem is creating, and the impact that solving it would have. If applicable, data access and integrity is also reviewed. 

Once the DIF Selection Committee chooses a project, ODI works with the department to further define their need through a structured problem definition process. During that process, ODI and the department develop a project scope, goals, desired outcomes, timeline, and potential deliverables. In order for this process to be a success, there must be a willingness to work in an agile approach where the work of discovery, research, experimentation, testing, and iterating determines outcomes and deliverables. 

Services Offered through the Data and Innovation Fund

ODI’s program staff are experts in many disciplines. Through a series of pre-application webinars and office hours, ODI teams educate potential DIF applicants about the services ODI offers and which may be best suited to their unique problem(s). ODI services include:

• The Data Science Accelerator (DSA) helps departments apply traditional artificial intelligence (e.g., machine learning, predictive algorithms) to resolve problems proactively, quickly, or at a reduced cost.

• The Modern Data Stack Accelerator (MDSA) helps departments rapidly adopt modern cloud-based data tools while working on a data challenge they want to solve.
• The CalInnovate Research and Innovation teams bring a tailored approach to each project. Their disciplines include user research, journey mapping, root cause analysis, problem mapping and definition, product management, content design, human-centered design, and user experience. Their work begins by defining the root problem through research, then co-creating human-centered solutions to improve and streamline the experience of both state staff and the Californians they serve.
  

Completed Projects³

California Victim Compensation Board (CalVCB)

• What we did: Provided analytics support to CalVCB to uncover opportunities to connect crime survivors with state resources.
  • Findings included recommendations for use of plain language, automated messaging and a new tool allowing staff to monitor outreach and analyze data.
  • CalVCB has translated application forms into multiple languages. 
  • The average grade reading level on the CalVCB website prior to ODI’s work was Grade 9, with several pages rated at Grades 10-14; the average grade level is now Grade 6. 
  • This project directly supported the Newsom Administration’s EO N-16-22 regarding the Embedding Equity goal to analyze demographic and geographic gaps in outcomes and access to funding and services, and reviewing community engagement strategies with a focus on Californians who reside in communities that have historically been underserved and/or marginalized.
  • Case study published to ODI website.

Department of Toxic Substances Control (DTSC)

• What we did: Used content design and plain language to improve public notices, and piloted geographically-targeted and multilingual social media ads to increase public participation in making decisions about toxic clean-up projects, with a special focus on engaging communities who have historically not been part of decision-making.
  • Pilots resulted in a 633% increase in public comments, 68% of which were in Spanish, despite most DTSC projects receiving few or no Spanish language comments on previous projects. The pilots had a reach of over 28,000 Californians. 
  • This project directly supported the Newsom Administration’s EO N-16-22 regarding the Embedding Equity goal to continue to work with agencies and departments to improve the delivery of language access and accessibility, and reviewing community engagement strategies with a focus on Californians who reside in communities that have historically been underserved and marginalized.
  • Case study published to ODI website.

California Department of Social Services – CalWORKs (CDSS)

• What we did: Conducted user research to identify ways to connect more eligible families to critical safety net programs through California Work Opportunity and Responsibility to Kids (CalWORKs). ODI research led to a series of findings, including:
  • Identifying burdens in the application process to improve enrollment and retention; 
  • Identifying opportunities to improve policy guidance to reduce similar burdens on county staff;
  • Opportunities to streamline communication to applicants and enrollees (timely messaging and plain language).
  • This project directly supported the Newsom Administration’s EO N-16-22 regarding the Embedding Equity goal to analyze demographic and geographic gaps in outcomes and access to funding and services, and reviewing community engagement strategies with a focus on Californians who reside in communities that have historically been underserved and marginalized.
  • Case study published to ODI website.

California Department of Emergency Services (Cal OES)

• What we did: Brought human-centered design approach to Cal OES in service of more user-friendly tools and services, sustainable IT systems, and organizational agility.
  • Through a collaborative partnership, the teams relied on research, human-centered design, and a mapping of existing processes to write an outcome-oriented procurement solution. 
  • It allows Cal OES IT to center its emergency system procurement not just on technical requirements, but on user needs, and allows for scalability. 
  • Ultimately, the research results put the human element of supporting business processes and objectives first. 
• This project received the Best IT Collaboration Award from Government Technology at the 2023 California Government Innovation Summit.

Ongoing Projects⁴

Benefits Recommender⁵

• What we are doing: This is a unique collaboration across state departments that is designed to close the gap between Californians eligible for benefits programs and those actually receiving benefits. California can lead the nation in social benefit program updates and best-in-class experience.
  • The Recommender widget is an ODI-hosted, open-source, low-code Application Programming Interface (API) placed on the confirmation web pages for a select group of benefit programs. It will drive successful benefit applicants to additional resources and benefits within the state that they may be eligible for, with direct links to the relevant other online applications. 
  • ODI’s main partner in this project is the Employment Development Department (EDD). ODI, in collaboration with EDD, has conducted stakeholder outreach with various other departments administering California benefit programs. The initial rollout was in late 2023 with an aim to scale the Recommender and release additional functionality in 2024. 
  • The pilot phase resulted in 2.1 million Recommender views, with an average click-through rate of 8.5%, resulting in over 3,200 submitted applications. 
  • For the CalFresh benefits program, the Recommender resulted in a conversion rate of 17.6%.

CalHR Modernization

• What we are doing: This project is part of a multi-phased, ongoing collaboration with CalHR in support of its Modernization project, using research and human-centered design to understand and improve the candidate journey and staff experience by enabling user experience (UX) improvements to CalCareers job filter and search, job postings, content design and technical support for enabling social media communications. Additionally, ODI has supported the research, development, procurement, and implementation of a new tool to help meet the requirement for an anonymous hiring system in the Examination and Certification Online System (ECOS) for job applicants, per EO N-16-22.
  • ODI anticipates that these efforts will materially contribute to a further improved candidate experience with a goal to increase the size of the candidate pool, the quality of candidates, and make it easier for candidates to find and apply for state jobs. 
  • This ODI engagement with CalHR was in support of Governor Newsom’s EO N-16-22 on Embedding Equity.
    

California Department of Emergency Services (Cal OES)

• What we are doing: ODI is continuing its work with Cal OES to develop human-centered design capability and capacity by focusing on business process analysis and improvements including making essential information collection and reporting more effective and efficient.

Data Science Accelerator Projects

State Water Resources Control Board – Division of Drinking Water

• What we’ll do: Forecast water shortages for local water systems impacted by drought through a data model that provides water forecasts, empowering staff to take prevention and intervention measures.

Department of Housing and Community Development

• What we’ll do: Flag errors of jurisdictions on affordable housing progress reports and lay the groundwork for an automated system capable of tracking affordability trends, building project timelines, and trends by jurisdiction.

Modern Data Stack Accelerator Project

CalHR

• What we’ll do: Migrate, update, and optimize data flows and queries to allow CalHR to readily track and measure hiring and job control processes to improve hiring and retention at the state. 

CalInnovate Research and Innovation Projects

California State Library

• What we’ll do: Conduct research in partnership with the State Library’s Research Bureau to define need, use cases and develop prototypes used for research summaries that support data-driven policy making. The State Library helps policymakers and their staff draft summaries in support of data-driven policy. The goal is to expedite the turnaround time by improving the process using database systems.

California Department of Technology – Digital ID

• What we’ll do: Support CDT’s Digital ID gateway team through user research to assist in research-informed opportunities and pilot testing of UX design.

California Department of Technology (CDT) – Project Approval Lifecycle

• What we’ll do: Conduct user research in support of CDT to track the Project Approval Lifecycle (PAL) and provide return on investment data as part of CDT’s broader Project Services Modernization (PSM) effort.

DIF Expenditures

The Digital Innovation Services Revolving Fund, known as DIF, was established by the legislature on July 1, 2020, pursuant to Government Code Section 12815(o)(1).  The chart below includes descriptions of all encumbrances and expenditures made from the DIF, as well as all revenues received by the fund, for fiscal year 2022-23 in compliance with Government Code Section 12815(o)(2).

DIF Fiscal Year 2022-23 Encumbrances

To accomplish current and anticipated DIF projects, ODI entered into the following contracts using DIF funds. The following encumbrances reflect outstanding commitments of DIF funds by ODI in FYI FY 2022-23:

DIF Fiscal Year 2022-23 Expenditures

The following expenditures were made from the DIF in FY 2022-23:

DIF Fiscal Year 2022-23 Revenues

The following are FY 2022-23 DIF revenues:

Appendix A: ODI Partnerships by State Department

Pursuant to California Government Code section 12815(o)(2)(A), below is, “A list of past and current engagements organized by state entity.”

• California Business, Consumer Services, and Housing Agency
  • Homelessness data story
  • Identifying core dataset needs
• California Department of Public Health
  • Abortion.ca.gov
  • Building human-centered design capacity across the state
  • Cannabis.ca.gov
  • Covid19.ca.gov
  • Human-centered procurement consulting
• California Department of Social Services
  • Benefits Recommender identifier 
  • CalWORKs discovery sprint
• California Department of Technology
  • Abortion.ca.gov
  • Ethical AI Framework and Toolkit
  • California Design System
  • Cannabis.ca.gov
  • Content design training
  • Master state entity list
  • Open data
  • Shared dataset development
  • ODI publishing system
  • Technology, Digital and Data Consulting Master Services Agreement
• California Department of Transportation
  • Credit card acceptance and fee reduction
• California Environmental Protection Agency
  • Drought.ca.gov
  • Identifying core dataset needs
• California Health and Human Services Agency
  • Data career fair and teams microsite
  • Interagency Data Exchange Agreement
  • Benefits Recommender (CalWORKs, WIC referrals)
• California Public Utilities Commission
  • Emergency Broadband Benefit user research
  • Human-centered procurement consulting
  • Lifeline website redesign
• California State Transportation Agency
  • Identifying core dataset needs
• California Victims Compensation Board
  • Reaching and helping victims of crime
  • Website analytics training
• Department of Cannabis Control
  • Building human-centered design capacity across the state
  • cannabis.ca.gov
  • Website analytics training
• Department of Finance
  • Master state entity list
• Department of Fish and Wildlife
  • Shared dataset development
• Department of Food and Agriculture
  • cannabis.ca.gov
• Department of General Services (DGS)
  • Master state entity list
  • Small business procurement user research
  • Technology, Digital and Data Consulting Master Services Agreement
  • DGS Office of Public School Construction – evaluation research and training
• Department of Human Resources
  • Master state entity list
  • Modernizing data jobs classifications
  • PowerBI training
  • State human resources modernization initiative
• Department of Parks and Recreation
  • Shared dataset development
• Department of Toxic Substances Control
  • Content design training
  • Public comment pilot project
• Department of Water Resources
  • Drought.ca.gov
• Employment Development Department
  • Benefits recommender
• Government Operations Agency
  • Credit card acceptance and fee reduction
  • Executive order on equity
  • Recruitment innovations
  • Small business procurement user research
  • Technology, Digital and Data Consulting Master Services Agreement
• Governor’s Office of Business and Economic Development
  • Credit card acceptance and fee reduction
• Governor’s Office of Emergency Services
  • Human-centered design
  • State website COVID-19 survey
  • Vaccine response research, data, and analytics
• Labor & Workforce Development Agency
  • Evaluation research and training
• Office of the Governor
  • Abortion.ca.gov
  • Benefits Recommender
  • Covid19.ca.gov
  • Drought.ca.gov
  • Homelessness data story
  • State human resources modernization initiative
• Secretary of State
  • Master state entity list
• State Controller’s Office
  • Master state entity list
• State Personnel Board
  • State human resources modernization initiative
• State Water Resources Control Board
  • Ethical AI Framework and Toolkit
  • Shared dataset development

1. https://innovation.ca.gov/our-work/projects/getting-compensation-to-more-california-crime-victims/ ↩︎
2. https://innovation.ca.gov/our-work/projects/lifting-californians-voices-in-toxic-substance-cleanup-project-decisions/ ↩︎
3. Funded through ODI General Fund Appropriation, and the Data and Innovation Fund. ↩︎
4. With the exception of the Benefits Recommender (see footnote 5), all ongoing projects are funded through both ODI General Fund Appropriation, and the Data and Innovation Fund. Any project work beyond 7/1/2024 is subject to the appropriation and availability of funds. ↩︎
5. Funded through ODI General Fund Appropriation. ↩︎