December 21, 2023
Amy Tong, Secretary
California Government Operations Agency
915 Capitol Mall, Suite 200
Sacramento, CA 95814
Dear Secretary Amy Tong,
In accordance with the State Leadership Accountability Act (Leadership Accountability), the Office of Data and Innovation submits this report on the review of our internal control and monitoring systems for the biennial period ending December 31, 2023.
Should you have any questions please contact Sergio Gutierrez, Chief Deputy Director, at sergio.gutierrez@innovation.ca.gov.
Governance
Mission and Strategic Plan
As a recently established office, the California Office of Data and Innovation (ODI) leadership and staff are dedicated to expeditiously improving state government innovation. During the ongoing development of internal processes and procedures, our foremost commitment is to promptly identify and address risks and implement robust controls to safeguard our organization and uphold its mission.
Mission Statement
ODI works to drive innovation across California state government, using data, technology, and human-centered design to achieve exceptional, equitable outcomes for all Californians. We collaborate with leaders and communities throughout California and the world. We partner to build empathy for the people we serve and create a government that is easy to interact with and solves big problems.
Strategic Goals
To accomplish our mission, ODI has established the following strategic goals:
- Establish ODI as a people-first innovation office that brings the value of human-centered design principles, tools, and methods to state teams.
- Encourage a data and analytics-informed culture where insights are central to service improvement and evaluation.
- Support development and implementation of sustainable innovation and culture change throughout state government.
Objectives
ODI’s objectives guide our day-to-day work.
- Engage directly with Californians on an ongoing basis to better understand their needs, and concerns in relation to services and products.
- Engage directly with California’s approximately 150 state entities on an ongoing basis to address their needs and concerns, and identify opportunities for continuous improvement in service delivery.
- Leverage research, analytics, and insights to design and prototype reusable or scalable human-centered solutions for service improvement.
- Develop a first-party data and insights infrastructure that uses new data and existing data to bring the attitudes, needs, and concerns of Californians into service design and evaluation.
- Provide direct support to departments to identify and help solve process, people, or policy issues, including consulting to provide analysis, insights, and recommendations.
- Work directly with state agencies and departments, as service owners, to build capacity to support the long-term sustainability of service improvements, including training to provide practical how-to skills, new approaches, and new tools.
Control Environment
The executive monitoring sponsor(s) responsibilities include facilitating and verifying ODI’s internal control and monitoring practices are implemented and functioning as intended. The executive monitoring sponsor(s) are listed below:
- Sergio Gutierrez, Chief Deputy Director
- Becki Gibson, Chief Counsel
These responsibilities are further shared with ODI’s seven (7) Deputy Directors and their respective teams.
ODI’s leadership exemplifies and reinforces a commitment to integrity and ethical values by embodying these principles in attitudes, behaviors, and through the establishment and communication of verbal and written standards of conduct through various documents. ODI’s directorate team sets an inclusive and positive tone for the organization, emphasizing the importance of ethical conduct, transparency, and a commitment to human-centric decision-making.
The organizational structure of ODI adheres to state personnel guidelines, ensuring the establishment of appropriate levels of responsibility and authority across the department. This structure is pivotal in fostering an environment where roles and responsibilities are well-defined and aligned with organizational objectives.
To ensure that operations run efficiently and effectively, ODI maintains documentation of its control system. Through reports and other forms of documentation, the directorate, executives and senior management team are able to detail information about the organization’s operations and its compliance with applicable laws and regulations.
ODI is dedicated to cultivating and retaining a competent workforce by adopting a multifaceted approach. This includes the strategic hiring of skilled professionals, providing ongoing professional development and training opportunities to enhance staff skill levels, and the creation of a positive work environment that motivates high performance and fosters employee retention. Additionally, ODI invests in executive coaching for all senior management staff members to enhance executive leadership skills.
Accountability is a core principle at ODI, extending from the top levels of management down to entity personnel. The enforcement of accountability mechanisms involves routine oversight, annual performance appraisals and adhering to human resource policies. This comprehensive approach ensures that all personnel are held responsible for fulfilling their assigned internal control responsibilities and accountable for the results, reinforcing a culture of responsibility and performance throughout the organization.
Information and Communication
ODI proactively communicates essential information to promote informed decision-making in support of its mission, employing a multifaceted communication approach that encompasses both internal and external channels.
Internally, information is shared through comprehensive avenues such as all-staff meetings, Friday huddle sessions, executive leadership meetings, senior management team meetings, operations and program meetings and team office hours. Beyond meetings, ODI disseminates critical information through email and other collaboration platforms. ODI develops and circulates policies and formal announcements to staff and is improving the centralization of information to its intranet, creating a hub for employees to share documents, policies, news, and updates.
Externally, ODI’s Leadership Team plays a pivotal role in fostering communication with diverse stakeholders. ODI actively participates in events and conferences, communities of practice, and offers valuable insights, direction and best practices to various committees for government departments. This strategic engagement aligns with ODI’s mission to drive innovation within the California state government.
ODI engages with private sector technology counterparts and academia to strengthen partnerships at the forefront of technological innovation. These relationships play a pivotal role in enabling ODI to gain insights into cutting-edge technologies that can enhance services and optimize departments serving the people of California.
Monitoring
The information included here discusses the entity-wide, continuous process to ensure internal control systems are working as intended. The role of the executive monitoring sponsor includes facilitating and verifying that the Office of Data and Innovation monitoring practices are implemented and functioning. The responsibilities as the executive monitoring sponsor(s) have been given to: Sergio Gutierrez, Chief Deputy Director; and Becki Gibson, Chief Counsel.
ODI ensures the effectiveness of its internal control systems through executive and senior management oversight, internal reviews, and external reviews. Oversight activities encompass routine meetings at the directorate, executive, and senior management levels, facilitating a comprehensive review and discussion of pertinent reports, alongside other diligent monitoring activities.
On a weekly basis, the directorate and executive team meets to address critical priorities, project status updates, and address issues requiring remediation or escalation. These sessions serve as a proactive platform for strategic decision-making and risk mitigation. Additionally, ODI’s directorate conducts multiple recurring meetings with executives and senior management to discuss workforce planning, workload priorities, and the progress of assigned initiatives, reinforcing a structured approach to organizational governance.
ODI proactively delegates the responsibility for monitoring and addressing vulnerabilities to the respective executives overseeing areas where issues are identified. The progress in mitigating these vulnerabilities is systematically assessed through regular evaluations conducted by the responsible executives.
ODI is in the process of finalizing formal policies and procedures, including a Risk Management and Monitoring Plan. These policies outline a structured framework for identifying and monitoring risks and controls. ODI leverages tools and worksheets to assess risks within their respective domains on a quarterly basis. These comprehensive quarterly reports will undergo review by the Executive Monitoring Sponsors. In a collaborative effort, the Executive Monitoring Sponsors will work with executives to promptly and effectively address identified risks by formulating and executing robust controls. Continuous monitoring by the Executive Monitoring Sponsors will ensure the ongoing effectiveness of these controls. This systematic approach positions ODI to swiftly identify and address risks, fostering a proactive and efficient risk management process.
Risk Assessment Process
The following personnel were involved in the Office of Data and Innovation risk assessment process: executive management, middle management, front line management, and staff.
The following methods were used to identify risks: brainstorming meetings, employee engagement surveys, ongoing monitoring activities, audit/review results, other/prior risk assessments, consideration of potential fraud, and other.
The following criteria were used to rank risks: likelihood of occurrence, potential impact to mission/goals/objectives, timing of potential event, potential impact of remediation efforts, tolerance level for the type of risk, and other.
ODI fosters a culture of open communication, enabling staff at all levels to identify risks that may pose obstacles to ODI’s mission. Upon the identification of risks, an escalation process is initiated through appropriate channels and the risks are comprehensively assessed by relevant leadership and stakeholders. The development of controls in response to identified risks are facilitated through active collaboration across ODI.
As a newly established department, ODI is working towards the development and improvement of state government innovation. This encompasses the implementation of a systematic risk evaluation process designed to empower ODI staff at all levels to promptly address potential risks. This process will facilitate collective assessments within each team to identify and evaluate risks in their respective area and to develop effective controls to safeguard ODI’s mission.ODI fosters a culture of open communication, enabling staff at all levels to identify risks that may pose obstacles to ODI’s mission. Upon the identification of risks, an escalation process is initiated through appropriate channels and the risks are comprehensively assessed by relevant leadership and stakeholders. The development of controls in response to identified risks are facilitated through active collaboration across ODI. As a newly established department, ODI is working towards the development and improvement of state government innovation. This encompasses the implementation of a systematic risk evaluation process designed to empower ODI staff at all levels to promptly address potential risks. This process will facilitate collective assessments within each team to identify and evaluate risks in their respective area and to develop effective controls to safeguard ODI’s mission.
Risks and Controls
Risk: Risk 1: Workforce Planning – Recruitment and Retention
ODI, similar to many state departments, continues to experience challenges filling positions and maintaining an adequate workforce. While ODI has seen significant growth in our staffing and made significant improvements to our talent acquisition process, we recognize that risk still remains due to vacant positions and retention of staff.
ODI’s mission, to improve services to Californians through the use of data, human-centered design, and technology, requires we create a workforce with specialized classifications. ODI’s work heavily relies on individuals with significant knowledge and expertise in key areas such as data management, advanced data analytics, machine learning, user research, human-centered design, and content design. Hiring for these specialized needs is challenging due to limited civil service classifications that support emerging technologies and the roles at ODI. ODI continues to work with the California Department of Human Resources (CalHR) and other departments to identify specific classification needs. Additionally, hiring challenges are further exacerbated by pay disparities between public and private salaries.
Without adequate controls planned to mitigate these risks, ODI may experience workflow delays, diversion of resources from critical program areas, and retention challenges. ODI’s previous and ongoing efforts to mitigate this risk include maintaining a fully staffed talent team. This highly skilled team utilizes modern hiring technology tools such as an applicant tracking system, online recruitment tools, and industry-related job boards to improve hiring and retention.
Control: Control A
Finalize and implement ODI’s Workforce Development Plan to improve ODI’s recruitment, development, and retention of a diverse, experienced, and impactful workforce.
Control: Control B
Fill existing vacancies by identifying and prioritizing hiring needs. The Head of Talent meets weekly with the executive leadership team to provide an update on ODI’s time -to-fill report and prioritize vacancies. The talent team will continue to update and monitor ODI’s time-to-fill report to provide accurate hiring timelines for vacant positions.
Control: Control C
Continue to develop hiring and onboarding process improvements to minimize hiring of vacant positions and improve retention. ODI will continue to utilize hiring tools to ensure a pipeline of incoming talent.
Risk: Risk 2 – Technology Modernization
ODI recognizes that the need for modern information technology (IT) tools and their solutions to meet these needs continues to develop rapidly. ODI understands the importance of the security of our information systems and the data within our systems. As a leader in innovation, ODI strives to model innovation and modernization with the technology solutions deployed within our workforce, clients, and partners. However, with the rate of change and the continued development of new technologies, such as Generative Artificial Intelligence (GenAI), ODI is aware that the risks associated with these technologies present new and complex challenges that could impact the security of ODI data, information, and assets.
Control: Control A
Maintain and continue development of a highly skilled IT operations team that is knowledgeable in modern IT tools and services to ensure the highest level of security for ODI data, information and assets.
- Over the past year ODI has hired a full time Chief Information Officer (CIO), Information Security Officer (ISO), and Information Technology Supervisor III (ITSIII). ODI now has a fully staffed IT Team. ODI is committed to maintaining and developing this team to support ODI’s ongoing IT needs and to protect ODI assets.
- ODI is committed to the ongoing training and development of all of ODI’s IT professionals. Identity Management Services training is planned in 2024 for our system administrators, which will improve user identification and access controls across all ODI applications and devices.
Control: Control B
Invest in technology tools that are fully compliant with the Statewide Information Management Manual (SIMM) security standards, enabling ODI to meet our current environment’s security challenges, and anticipate and plan for future challenges.
- ODI has implemented Multi-Factor Authentication (MFA) for all ODI staff. ODI has also implemented advanced email spoofing protection.
- ODI’s CIO and ISO continue to be closely involved in establishing IT standards and assessing IT products and services.
Control: Control C
Development, implementation and compliance monitoring of IT policies to ensure that all staff who have access to ODI assets are aware of risks and their responsibilities.
- ODI’s IT team continues to develop critical policies that meet state and federal standards. These standards are in compliance with the National Institute of Standards and Technology (NIST) and enable ODI to meet its mission. Policies are developed in collaboration with, IT executive leadership, CalHR, labor relations, and legal.
- ODI requires all staff to complete annual IT security training and is currently building a new information security and privacy awareness training platform for annual training and security policy acknowledgment.
Risk: Risk 3 – Sustained Funding
ODI relies on appropriations from the General Fund to fund its Data and Innovation Services Revolving Fund (known as the Data and Innovation Fund [DIF]). The DIF enables ODI to conduct projects with partner departments and to pursue projects to support special initiatives. ODI’s authority to expend funding from the DIF expires per Government Code section 12815 on June 30, 2024. ODI has committed most of the remaining DIF funds to current and planned projects to be completed by the end of this Fiscal Year (FY). If the DIF is not replenished for FY 24-25, and ODI is not given further spending authority, ODI’s future project portfolio would need to be significantly reduced. ODI’s ability to provide services for critical projects such as the Governor’s executive orders on GenAI and Equity would also be impacted if future DIF funding is unavailable.
Control: Control A
Implement new trailer bill language to extend the continuous appropriation of DIF beyond July 1, 2024.
Control: Control B
Workforce capacity planning to address project priorities based on urgency, statute, and Executive Orders.
Conclusion
The Office of Data and Innovation strives to reduce the risks inherent in our work and accepts the responsibility to continuously improve by addressing newly recognized risks and revising risk mitigation strategies as appropriate. I certify our internal control and monitoring systems are adequate to identify and address current and potential risks facing the organization.
Jared Johnson, Director
CC: California Legislature [Senate, Assembly]
California State Auditor
California State Library
California State Controller
Director of California Department of Finance
Secretary of California Government Operations Agency