Governing Artificial Intelligence

The Question Is No Longer What It Can Do, but What We Are Willing to Let It Do

During the first years of generative artificial intelligence, public debate was dominated by a relatively simple question: what can this technology do? Attention focused on comparing models, measuring their ability to write, code, translate, summarize, generate images, or solve increasingly complex problems. Each new release seemed to push the boundaries a little further, reinforcing the impression that the main challenge was simply to build ever more capable systems.

But as artificial intelligence moves beyond experimentation and becomes embedded in companies, public administrations, hospitals, schools, banks, and everyday services, that question is beginning to feel insufficient. The real issue is no longer only what an AI system can do, but what we want to allow it to do, under what conditions, within which limits, and who should be held responsible when its decisions produce real-world consequences.

That, in essence, is the challenge of AI governance.

Talking about governance does not mean talking only about laws, prohibitions, or regulation. Nor does it mean turning every innovation project into an endless bureaucratic process. Governance is about establishing mechanisms to decide which uses are acceptable, which risks can be tolerated, what controls should be applied, who is authorized to approve certain capabilities, and what evidence should be preserved so that we can understand what happened when something goes wrong.

The issue may sound administrative, but it is in fact deeply technological, economic, and political. The greater the ability of these systems to intervene in the real world, the more important it becomes to determine who controls that ability.

The Model Is Only Part of the Problem

One of the most common mistakes in discussions about artificial intelligence is to treat the model as if it were the entire system. We compare one model with another, analyze their outputs, and conclude that one is safer, more advanced, or more suitable than the other, even though the final behavior depends on many additional decisions made around it.

The same model can be used to summarize public documents or to access medical records. It can help draft an email or be connected to an enterprise platform with permission to modify records. It can generate a recommendation that a person reviews carefully, or participate in an automated process that immediately affects a customer.

The model may be exactly the same, but the risk is not.

A real-world AI application usually combines different elements: instructions, data, interfaces, document repositories, memory, external tools, permissions, enterprise systems, and people who intervene before or after the system produces an action. The outcome depends on how all these components are connected and, above all, on the level of authority the organization has decided to grant them.

For that reason, an organization that wants to govern artificial intelligence properly should stop asking only which model it uses and start asking what complete capability it has actually put into operation.

The Same Model Can Create Completely Different Risks

Imagine two companies using exactly the same language model. The first uses it to summarize public reports and prepare drafts that are later reviewed by an employee. The second connects that same model to corporate email, a customer database, and a tool capable of modifying records or carrying out specific actions.

In both cases, the companies could say they are using the same artificial intelligence, but that description tells us very little about the actual level of risk.

In the first scenario, the main problem might be an incomplete or inaccurate summary. In the second, an error could lead to an inappropriate communication, an incorrect change to customer data, an unauthorized action, or the exposure of sensitive information.

The key difference is not necessarily that one system is more intelligent than the other. It is that one has access to more information, more connections, and greater authority to act.

This distinction is essential for understanding AI governance. For years, discussions have often treated risk as if it automatically increased with model capability. In reality, a significant part of the risk appears when an organization turns intellectual capability into operational authority.

Intelligence Does Not Mean Authority

This distinction will become increasingly important. An artificial intelligence system may be able to analyze thousands of documents, compare alternatives, identify patterns, detect inconsistencies, and propose a solution without having permission to execute any final consequence. That design allows organizations to benefit from much of the technology’s value without automatically handing over control of the process.

A system may:

• recommend a bank transfer without being able to execute it;
• prepare a contract without being able to sign it;
• suggest a database modification without permission to apply it;
• draft a reply to a customer without sending it automatically;
• identify a medical anomaly without replacing the decision of the responsible professional.
  

Governance exists precisely in that space between what artificial intelligence is capable of doing and what an organization decides to authorize. In many cases, the most important question will not be whether the system can perform a task, but whether it should be allowed to do so without human intervention.

a Human Oversight Can Exist Only on Paper

The concept of human oversight appears frequently in policies, regulations, and corporate statements about artificial intelligence, but its formal existence does not guarantee real control. A person may officially be responsible for reviewing a decision and yet receive hundreds of automated recommendations every day, have only a few seconds to examine each one, or lack the information needed to challenge them. Under those conditions, human oversight can become little more than a procedural formality used to legitimize a decision that, in practice, has already been made by the system.

The difference between supervising and merely confirming is significant. For oversight to be meaningful, a person must have enough time, relevant information, and genuine authority to stop, correct, or reverse an action. If none of those conditions are present, inserting a human being into the process does not necessarily reduce risk. Governance, therefore, should not simply ask whether a human is involved, but what real capacity that person has to intervene.

Data Is Also Part of Governance

Artificial intelligence does not operate in a vacuum. Its capabilities expand when it can consult external information, access document repositories, retrieve business data, or use the history of previous interactions. Each new source can improve the quality of the system’s output, but it also changes its risk profile.

An application that works only with public information is very different from one connected to customer data, medical files, financial records, or confidential internal documentation. Likewise, allowing a system to read from a database is not the same as allowing it to write to it. For this reason, governing artificial intelligence also means knowing what information each system can access, who authorized that access, where the data is processed, how long it is retained, and what happens when a source changes.

Security is no longer only about protecting databases from external intrusion. It also means controlling which intelligent systems can interact with them and under what conditions.

Agents Change the Nature of the Problem

The emergence of AI agents expands this discussion even further. A traditional chatbot waits for an instruction and produces a response. An agent can receive an objective, decide which steps are required, use tools, consult external sources, and continue working until a task is completed.

That change may appear incremental, but it fundamentally alters the nature of the system.

The question is no longer only what artificial intelligence can say. It becomes what artificial intelligence can do. When an agent can use email, calendars, databases, enterprise management systems, or financial tools, concepts such as identity, permissions, authorization, and traceability stop being minor technical details and become central elements of governance. A system capable of drafting an email presents one type of risk. A system capable of sending it automatically presents another. If it can also decide who should receive it, consult internal information beforehand, and update a record afterwards, we are dealing with a much more complex capability that cannot be evaluated simply by looking at the underlying model.

When Something Goes Wrong, Someone Has to Be Responsible

Responsibility becomes particularly important when a decision is produced by a combination of people, models, applications, and automated processes.

• Who is accountable if a system harms a customer?
• The model provider?
• The company that built the application?
• The department that decided to deploy it?
• The person who accepted the recommendation?
• The manager who approved the process?

The answer will depend on the context, but an organization cannot wait for an incident before asking these questions.

Every significant AI capability should have a clearly identified owner and a description precise enough for another person to understand how it works.

At a minimum, it should be possible to answer:

• what the system is used for;
• which people may be affected;
• what information it can access;
• which tools it can use;
• which actions it can perform;
• which actions require approval;
• who supervises its outputs;
• how errors can be corrected;
• which changes require the system to be reviewed again.

This is not about producing documentation for its own sake. It is about preventing an increasingly common situation in which AI systems are widely used across an organization while responsibility is spread across so many departments that no one can fully explain how the system operates.

Governance Also Means Being Able to Reconstruct What Happened

When an AI system participates in an important decision, the organization should be able to go back and reconstruct the process. Which version of the model was used? What information did it receive? Which tools did it access? What action did it propose? Did a person intervene? Was the output later modified?

Without some degree of observability and logging, answering these questions may be impossible.

This does not mean that everything must be stored indefinitely or that every interaction requires a full audit. It means preserving enough evidence to understand significant decisions, identify failures, and establish responsibility. An organization that cannot reconstruct how a decision was produced can hardly claim that it truly governs the system that produced it.

The Challenge of Governing Systems That Constantly Change

Artificial intelligence introduces an additional difficulty: these systems evolve extremely quickly.

The model may change while the application remains the same. A new document source may be added, memory may be enabled, a new tool may be connected, a permission may be modified, the number of users may increase, or human review may be reduced.

Any of these changes can significantly alter the level of risk even when the user interface looks exactly the same.

This means governance cannot function as a one-time authorization granted at the moment of deployment. There must be some mechanism for recognizing when a change is important enough to require previous assessments to be revisited.

Not every modification needs to trigger a complete approval process, but an organization should be able to distinguish between a routine update and a transformation that changes the system’s authority, scope, or potential consequences.

Governance Is Not the Same as Slowing Innovation

There is a legitimate concern that badly designed governance can become bureaucratic and unnecessarily slow the adoption of new technology. The problem arises when organizations attempt to apply exactly the same controls to every system.

An assistant that summarizes public documents does not necessarily require the same evaluation process as an agent capable of modifying financial information. Treating both cases as equivalent does not improve safety. It simply creates controls where they add little value while distracting resources from the areas that genuinely require attention.

Effective governance should be proportional to potential impact. Systems with limited consequences can be managed through relatively simple controls, while those that process sensitive information, affect individual rights, or execute significant actions require stronger mechanisms for evaluation, identity, permissions, oversight, and logging. The objective should not be to prevent organizations from experimenting with artificial intelligence. It should be to prevent them from experimenting without understanding what they are actually putting into operation.

From the Race for Intelligence to the Debate About Power

Over the past few years, we have watched a race to build increasingly capable models. That race will almost certainly continue, but the next major debate will not be purely technological.

It will be a debate about authority.

When artificial intelligence can research, recommend, decide, negotiate, program, purchase, contract services, or interact with other systems, the central question will be who gave it permission to do so and under which rules.

That shift requires us to abandon an overly simplistic view of artificial intelligence as an isolated tool. Intelligent systems will become part of human processes, institutions, companies, and decision-making structures, and their consequences will depend as much on the rules we build around them as on the technology itself.

That is where AI governance truly begins. It is not about deciding whether technology should continue to advance, but about determining how we want to integrate it into our organizations and societies without losing our ability to understand, limit, and correct its actions. For a long time, we have asked how far artificial intelligence can go. Perhaps the time has come to ask a more difficult question: how far are we willing to let it go?

My Books on These Topics on Amazon