AI-driven insights for transforming risk management and loss control

Artificial intelligence is already changing how public entities work. Employees are using generative AI to research information, analyze data, draft communications and streamline administrative work. Educators and students are incorporating it into the classroom, while government agencies are exploring applications that can improve efficiency and help employees make sense of increasingly complex information.

For public-sector risk leaders, widespread adoption brings a growing set of considerations. AI use can introduce exposures involving privacy, cybersecurity, employment practices, records retention, misinformation, reputational risk and public safety, and in many organizations, employees may already be using AI before formal policies, training programs and governance structures have fully caught up. 

Established risk management and loss control practices can provide a durable framework for managing these exposures, even as the technology continues to change.

Start with governance

One of the most useful questions a risk leader can ask today is: Do we know how AI is being used across our organization?

Without an organization-wide approach, individual employees or departments may make their own decisions about which tools to use, what information can be entered into them and when AI-generated output is appropriate to incorporate into their work. Those decisions can create exposures well beyond the employee who initially used the technology.

A formal AI governance policy can establish common expectations around acceptable and prohibited uses, data privacy and security, human review, employee training, vendor controls, records retention and incident reporting. 

For school districts, governance may also include student and parent AI-use agreements and clear expectations for employees handling student information. Written policies can provide a foundation for training, accountability and consistent decision-making, while also demonstrating that the organization has taken proactive steps to manage emerging risks. 

Because AI capabilities are increasingly embedded in phones, software platforms, search tools and other technologies employees use every day, public entities should establish clear parameters for how those tools can be used, what information they can access and where human oversight is required.

Follow the data

Public entities routinely handle information that should not be casually entered into an unapproved AI environment, including employee records, constituent information, protected student data and other confidential or sensitive material.

Risk leaders should work with IT, information security, legal and operational teams to understand where information goes when an AI application is used, whether it is retained, who may have access to it and whether the organization can retrieve it when necessary.

Public records requirements add another consideration. If employees use AI to create or modify content related to public business, organizations may need to determine whether that content is subject to applicable retention or disclosure requirements and whether it can actually be located and produced. 

A useful loss-control exercise is to follow the data from beginning to end: What information is entering the tool? Where does it go? What happens to it there? Who can access it? How would we retrieve it if required?

When those questions cannot be answered confidently, the use case warrants additional review.

Keep people accountable for the outcome

AI can produce information with remarkable speed and apparent confidence, including information that is incomplete, biased or incorrect. This becomes particularly important when AI influences decisions involving hiring, promotion, accommodation, licensing, claims, public communications or other consequential activities.

Effective human review requires someone with enough knowledge and authority to question the output, validate the underlying information and reject a recommendation when appropriate.

Organizations should also consider who may be affected when AI gets something wrong. Students may face concerns involving privacy, accessibility and safety; employees may be affected by decisions involving hiring, accommodations or advancement; and citizens may encounter questions involving equal access, transparency and due process. 

The potential consequences should help determine the appropriate level of oversight. AI applications that can significantly affect an individual may require additional testing, human review and clearly defined escalation procedures.

Experience is another consideration. A highly experienced professional may quickly recognize when an AI-generated recommendation does not make sense, while someone with less experience may be more inclined to accept the output at face value. Organizations should consider where additional support or review may be needed across different experience levels and roles. 

AI training should include guidance on recognizing questionable outputs, validating information and knowing when additional review is warranted.

Plan for when something goes wrong

Traditional loss-control principles remain highly relevant in an AI-enabled environment.

Critical information should be appropriately backed up and recoverable. Vendor contracts should clearly address responsibility, data protection and other appropriate safeguards, including indemnification provisions and confirmation of appropriate insurance coverage for AI-related exposures. As insurers introduce exclusions related to generative AI, organizations should work with their insurance brokers to carefully review policy language and identify potential coverage gaps. These exclusions may affect liability arising from AI-generated content, recommendations or decisions, including certain bodily injury, property damage and intellectual property claims. Organizations should also understand how vendors are using AI within the services they provide, since an entity’s exposure can extend beyond the technology it directly purchases or deploys.

Higher-risk applications may require additional safeguards. For example, if technology is expected to identify language associated with self-harm, threats of harm to others, bullying or another serious safety concern, risk leaders should understand whether automated monitoring and alerts are in place, what triggers those alerts and whether they reach the appropriate personnel for timely intervention. For school districts in particular, these safeguards can be important when technology plays a role in identifying potential threats to student well-being.

Organizations should also consider how they would regain control if an AI-enabled process began behaving unexpectedly. Can the technology be immediately isolated or disabled through an emergency shutoff mechanism, sometimes called a kill switch? Can critical information be recovered? Who has the authority to intervene?

These are familiar risk-management questions applied to a new generation of technology.

Make AI governance a shared responsibility

Effective AI governance requires participation across the organization. Boards and executive leaders may establish risk appetite and policy; HR can address workforce training and acceptable use; IT and security can manage technology and data protection; legal and compliance teams can address regulatory and privacy requirements; department leaders can oversee operational applications; and managers and employees remain accountable for how AI is used in everyday work. 

Risk leaders can play an important role in connecting these functions, identifying where exposures intersect and helping ensure there are clear lines of responsibility as new AI applications are introduced.

Seven questions for public-sector risk leaders

Public entities can begin strengthening their controls by asking seven practical questions:

1. Where are employees already using AI, including tools that have not been formally approved?
2. What information is being entered, and could any of it be confidential, protected or subject to records requirements?
3. Which AI-assisted activities could materially affect an employee, student, citizen or constituent?
4. Where is human review required, and are reviewers prepared to question and validate the AI’s output?
5. What situations should trigger an immediate escalation to a person?
6. Could we stop an AI-enabled process and recover critical information if something went wrong?
7. When did we last review our policies, training, vendor contracts and insurance coverage against the AI technology actually being used today?

These questions should be revisited regularly as new regulations, emerging technology, privacy and security risks, operational experience and changing employee adoption alter an organization’s risk profile.

Building resilience as AI evolves

AI is developing faster than many traditional public-sector policy and approval processes can move. In some organizations, policies must go through boards and other formal review processes, which can make it difficult for governance to keep pace with rapid changes in the technology. 

Established risk management practices provide a useful foundation: clear expectations, thoughtful controls, protection of sensitive information, meaningful human oversight, training, escalation procedures and regular reassessment. Those practices can also give public entities greater confidence when evaluating new AI applications and deciding where additional safeguards may be appropriate.

The organizations that establish clear expectations, strong oversight and practical safeguards today will be better prepared to adapt as AI continues to change.