Explore five emerging AI governance risks, from shadow AI and cyber security to HR, third parties and EU AI Act compliance.
Introduction: The Growing AI Governance Gap Many organizations now have AI policies and approved tools, but surveys reveal a significant gap between documented governance and the actual use of AI across teams. AI risks manifest in various ways, from sophisticated phishing emails to unapproved chatbots handling sensitive data. This fragmentation poses a broader AI governance challenge, with five key areas currently pressuring existing controls. The risk of shadow AI Employees often use unapproved AI tools for daily tasks, inadvertently uploading company or personal data without understanding the data's subsequent journey. This 'shadow AI' activity lacks organizational visibility, making it difficult to track tool usage and information sharing. While AI policies set expectations, effective control requires robust governance, oversight, and continuous AI literacy to ensure employees understand and mitigate risks proactively. AI in cyber security AI is transforming cyber security threats by making them more potent and elusive. Phishing and social engineering attacks are increasingly sophisticated and scalable due to AI, blurring the lines between genuine and malicious communications. Furthermore, integrating AI tools into organizational systems and data creates new pathways for access to sensitive information. The critical challenge lies in ensuring that existing security measures and testing protocols evolve to match the dynamic nature of AI-driven threats and the expanding organizational exposure. AI in HR and employment The use of AI is impacting human resources in two key ways: employees are leveraging tools like ChatGPT to draft grievances or employment claims, sometimes with inaccurate AI-generated content. Concurrently, HR teams are utilizing AI to streamline documentation review, summarize complex cases, and manage heavy workloads involving sensitive employee data. These applications introduce risks related to identifying fabricated information and ensuring data protection and human oversight in AI processes. Additionally, involving third-party suppliers for AI services adds complexity regarding data usage, protection, and accountability. The issue of assurance and third parties Organizations increasingly require proof that their suppliers adhere to appropriate AI governance, and must also be prepared to offer similar assurances to their own customers. Surveys indicate that formal standards like ISO/IEC 42001 (36%) and NIST AI RMF (33%) are already influencing responsible AI decision-making. The challenge is to move beyond mere compliance checks to establish meaningful assurance that demonstrates the controls around an AI system are proportional to its inherent risks. The EU AI Act Most provisions of the EU AI Act are now in effect, with high-risk obligations commencing in December 2027. This timeline provides organizations with preparation time, but AI Act compliance should not be viewed in isolation. Existing data protection obligations, particularly concerning personal data, must be integrated into comprehensive governance and compliance strategies. Frameworks like NIST AI RMF can offer a structured approach to assessing AI risks and enhancing existing governance processes, providing a pathway to meet the Act's requirements. Bringing everything together The diverse nature of AI risk highlights how it varies based on its application within an organization. The fundamental question across all five areas is whether organizations possess sufficient visibility and control to adapt to this evolving landscape. This will be the central theme of the DPO Centre's 'AI Reality Check' webinar day, part of their Global Privacy Puzzle Webinar Series. The event, on October 20th, will feature five expert-led sessions addressing different AI risk areas, offering attendees the chance to engage with specialists and understand how to manage these emerging challenges.
Introduction: The Growing AI Governance Gap
Many organizations now have AI policies and approved tools, but surveys reveal a significant gap between documented governance and the actual use of AI across teams. AI risks manifest in various ways, from sophisticated phishing emails to unapproved chatbots handling sensitive data. This fragmentation poses a broader AI governance challenge, with five key areas currently pressuring existing controls.
The risk of shadow AI
Employees often use unapproved AI tools for daily tasks, inadvertently uploading company or personal data without understanding the data's subsequent journey. This 'shadow AI' activity lacks organizational visibility, making it difficult to track tool usage and information sharing. While AI policies set expectations, effective control requires robust governance, oversight, and continuous AI literacy to ensure employees understand and mitigate risks proactively.
AI in cyber security
AI is transforming cyber security threats by making them more potent and elusive. Phishing and social engineering attacks are increasingly sophisticated and scalable due to AI, blurring the lines between genuine and malicious communications. Furthermore, integrating AI tools into organizational systems and data creates new pathways for access to sensitive information. The critical challenge lies in ensuring that existing security measures and testing protocols evolve to match the dynamic nature of AI-driven threats and the expanding organizational exposure.
AI in HR and employment
The use of AI is impacting human resources in two key ways: employees are leveraging tools like ChatGPT to draft grievances or employment claims, sometimes with inaccurate AI-generated content. Concurrently, HR teams are utilizing AI to streamline documentation review, summarize complex cases, and manage heavy workloads involving sensitive employee data. These applications introduce risks related to identifying fabricated information and ensuring data protection and human oversight in AI processes. Additionally, involving third-party suppliers for AI services adds complexity regarding data usage, protection, and accountability.
The issue of assurance and third parties
Organizations increasingly require proof that their suppliers adhere to appropriate AI governance, and must also be prepared to offer similar assurances to their own customers. Surveys indicate that formal standards like ISO/IEC 42001 (36%) and NIST AI RMF (33%) are already influencing responsible AI decision-making. The challenge is to move beyond mere compliance checks to establish meaningful assurance that demonstrates the controls around an AI system are proportional to its inherent risks.
The EU AI Act
Most provisions of the EU AI Act are now in effect, with high-risk obligations commencing in December 2027. This timeline provides organizations with preparation time, but AI Act compliance should not be viewed in isolation. Existing data protection obligations, particularly concerning personal data, must be integrated into comprehensive governance and compliance strategies. Frameworks like NIST AI RMF can offer a structured approach to assessing AI risks and enhancing existing governance processes, providing a pathway to meet the Act's requirements.
Bringing everything together
The diverse nature of AI risk highlights how it varies based on its application within an organization. The fundamental question across all five areas is whether organizations possess sufficient visibility and control to adapt to this evolving landscape. This will be the central theme of the DPO Centre's 'AI Reality Check' webinar day, part of their Global Privacy Puzzle Webinar Series. The event, on October 20th, will feature five expert-led sessions addressing different AI risk areas, offering attendees the chance to engage with specialists and understand how to manage these emerging challenges.
