Agentic AI Demands a New Governance Playbook
For years, global policymakers, industry leaders, and technologists have built governance models around the assumption that AI systems assist and do not act. But the rise of agentic AI challenges that pattern and demonstrates the need for a new governance playbook.
Most traditional AI governance frameworks—including broad, risk-based models, such as the NIST AI Risk Management Framework—were constructed for systems that were static, user-led, and consistently under human control. They were not specifically designed for an environment in which technology behaves as an actor across business, government, and consumer contexts.
In short, they were not designed for agentic AI systems, which do much more than just generate outputs; they take actions, make decisions, and operate with growing autonomy. They navigate workflows, initiate tasks and, in some cases, alter real-world environments without human prompting.
This shift forces us to confront a fundamental question: Are our current AI and data governance principles sufficient for an era in which technology itself becomes a decision-maker?
Conventional AI principles, which emphasize fairness, transparency, explainability, and human oversight, are a good foundation. Meanwhile, privacy frameworks – such as General Data Protection Regulation (GDPR), Global Cross-Border Privacy Rules (CBPR), Data Privacy Framework (DPF), or state-based U.S. AI laws – provide important baselines through data minimization, purpose limitation, and user rights.
However, as AI systems evolve, these frameworks may need to be complemented by more dynamic approaches that reflect how these technologies behave in practice.
The shift from “AI as advisor” to “AI as actor” demands a new governance architecture, and policymakers are exploring options. Singapore introduced a governance framework specifically for agentic AI. Launched at the most recent World Economic Forum, the Singapore Model AI Governance Framework identifies agentic AI systems as self-managing AI agents that can act, adapt, access external tools, and interact with systems to accomplish goals, a sharp departure from generative or predictive models that only output text or recommendations.
In the United States, federal and state efforts reflect a growing focus on how to support innovation while addressing emerging risks.
The White House introduced a national legislative framework on artificial intelligence and, just this month, President Trump issued an executive order to establish a voluntary framework for federal oversight on new AI models before they are released to the public.
NIST continues to play a central role in shaping technical benchmarks, recently completing a call for comments on the security considerations for agentic AI, to which our organization, BBB National Programs, and other respected experts and leading organizational voices have responded.
At the state level, in California, Governor Gavin Newsom signed a “first-of-its-kind executive order to strengthen AI protections and responsible use.” Virginia is focusing on an agentic AI pilot program, Delaware is launching a sandbox initiative, Tennessee released an RFI to gather information on how to leverage agentic AI for enterprise resource planning, and Connecticut’s governor is expected to sign the online safety and AI bill, the Connecticut Artificial Intelligence Responsibility and Transparency Act.
Leading think tanks like Center for Information Policy Leadership and Brookings are also contributing, such as with case studies on agentic AI that provide positive use cases, while considerable PAC funding is being directed toward efforts to minimize the passage of state-level AI laws, aiming to maintain a unified regulatory environment. Similar legislation also is being proposed.
At its core, the privacy and security of agentic AI may need to be informed by traditional data governance models, an expectation regulators will place on the data leveraged to feed and train uses of agentic AI. Among others, at least four core principles can form the basic building blocks on which any additional rules or controls for agentic AI will sit:
Certainly, there are many more difficult issues that need to be weighed, such as those identified by the Stanford AI principles, which prioritize developing technology that augments humanity, ensures ethical and equitable use, and addresses societal impacts. Key tenets include human-centered design, accountability, transparency, fairness, and safety, aiming to ensure AI benefits society.
While many governance principles for agentic AI can be drawn from existing AI and privacy playbooks, there remain some important distinctions. For instance, mainstream AI and privacy principles assume controllability, but agentic AI assumes autonomy. And while generative AI models assume human judgment, agentic AI requires machine judgment aligned with human ethics.
Agentic AI is not an extension of traditional AI. It is the beginning of a transformation of AI’s role in society. And as such, it demands a new governance playbook, one that recognizes the difference between tools that respond and tools that act.
Originally published in Forbes
Most traditional AI governance frameworks—including broad, risk-based models, such as the NIST AI Risk Management Framework—were constructed for systems that were static, user-led, and consistently under human control. They were not specifically designed for an environment in which technology behaves as an actor across business, government, and consumer contexts.
In short, they were not designed for agentic AI systems, which do much more than just generate outputs; they take actions, make decisions, and operate with growing autonomy. They navigate workflows, initiate tasks and, in some cases, alter real-world environments without human prompting.
This shift forces us to confront a fundamental question: Are our current AI and data governance principles sufficient for an era in which technology itself becomes a decision-maker?
Conventional AI principles, which emphasize fairness, transparency, explainability, and human oversight, are a good foundation. Meanwhile, privacy frameworks – such as General Data Protection Regulation (GDPR), Global Cross-Border Privacy Rules (CBPR), Data Privacy Framework (DPF), or state-based U.S. AI laws – provide important baselines through data minimization, purpose limitation, and user rights.
However, as AI systems evolve, these frameworks may need to be complemented by more dynamic approaches that reflect how these technologies behave in practice.
The New Principles Agentic AI Requires
When applied to agentic AI systems, additional layers emerge. For instance, traditional AI and privacy principles operate on predictability—systems behaving consistently over time. But agentic AI, by contrast, evolves, learning from context, adjusting strategies, and modifying actions in ways that are not always fully anticipated at deployment.The shift from “AI as advisor” to “AI as actor” demands a new governance architecture, and policymakers are exploring options. Singapore introduced a governance framework specifically for agentic AI. Launched at the most recent World Economic Forum, the Singapore Model AI Governance Framework identifies agentic AI systems as self-managing AI agents that can act, adapt, access external tools, and interact with systems to accomplish goals, a sharp departure from generative or predictive models that only output text or recommendations.
In the United States, federal and state efforts reflect a growing focus on how to support innovation while addressing emerging risks.
The White House introduced a national legislative framework on artificial intelligence and, just this month, President Trump issued an executive order to establish a voluntary framework for federal oversight on new AI models before they are released to the public.
NIST continues to play a central role in shaping technical benchmarks, recently completing a call for comments on the security considerations for agentic AI, to which our organization, BBB National Programs, and other respected experts and leading organizational voices have responded.
At the state level, in California, Governor Gavin Newsom signed a “first-of-its-kind executive order to strengthen AI protections and responsible use.” Virginia is focusing on an agentic AI pilot program, Delaware is launching a sandbox initiative, Tennessee released an RFI to gather information on how to leverage agentic AI for enterprise resource planning, and Connecticut’s governor is expected to sign the online safety and AI bill, the Connecticut Artificial Intelligence Responsibility and Transparency Act.
Leading think tanks like Center for Information Policy Leadership and Brookings are also contributing, such as with case studies on agentic AI that provide positive use cases, while considerable PAC funding is being directed toward efforts to minimize the passage of state-level AI laws, aiming to maintain a unified regulatory environment. Similar legislation also is being proposed.
At its core, the privacy and security of agentic AI may need to be informed by traditional data governance models, an expectation regulators will place on the data leveraged to feed and train uses of agentic AI. Among others, at least four core principles can form the basic building blocks on which any additional rules or controls for agentic AI will sit:
- Bounded and Justified Autonomy: Agentic systems must operate within clearly defined autonomy tiers, with explicit constraints on tools, environments, and permissions. These constraints can be far more granular than anything in traditional privacy frameworks but build on the similar expectations considered in the use of privacy controls.
- Embedded Ethical Reasoning: Agentic AI must internalize ethical reasoning, not as external policy guidance, but as part of its native decision-making logic. Training the agent on ethical parameters is essential to ensure the agent is acting with an ethical backbone.
- Continuous Evaluation, Not Periodic Audits: Traditional principles assume systems are validated at deployment. Agentic AI requires lifecycle supervision, where agents are monitored constantly, tested regularly, and reevaluated as they adapt. A human in the loop is an important element of the checks-and-balances required to power agentic AI.
- Identity, Traceability, and Zero Trust Controls: Every agent must be treated as a nonhuman identity with its own credentials, permissions, and audit trails, ones that mirror the security requirements and screening of security clearance processes for humans.
Certainly, there are many more difficult issues that need to be weighed, such as those identified by the Stanford AI principles, which prioritize developing technology that augments humanity, ensures ethical and equitable use, and addresses societal impacts. Key tenets include human-centered design, accountability, transparency, fairness, and safety, aiming to ensure AI benefits society.
While many governance principles for agentic AI can be drawn from existing AI and privacy playbooks, there remain some important distinctions. For instance, mainstream AI and privacy principles assume controllability, but agentic AI assumes autonomy. And while generative AI models assume human judgment, agentic AI requires machine judgment aligned with human ethics.
Agentic AI is not an extension of traditional AI. It is the beginning of a transformation of AI’s role in society. And as such, it demands a new governance playbook, one that recognizes the difference between tools that respond and tools that act.
Originally published in Forbes