Autonomous AI agents can streamline entire workflows and deliver significant business value. But success depends on maintaining control over data, access and decisions.
“That is the paradox. You get the benefits when you allow the agents to make decisions and operate autonomously. At the same time, you need to be able to control them,” says Sigrun Hansen Bock, Director of Banking and Insurance at Vivicta.
So far, much of the focus on AI has been on tools that help employees write, analyse, code and compile information.
Many organisations are now entering a new phase in their use of AI, where multiple AI agents work together to complete tasks, including making their own decisions along the way.
This is where Hansen Bock believes the greatest financial gains can be found. But while some businesses want to expand their use of AI into more and more areas, others are limiting themselves to internal experiments for fear of losing control.
“That is the biggest barrier to realising the benefits of AI. Many are reluctant to move forward because they do not know which security mechanisms they need to put in place,” she says.
The potential costs of using incorrect data in agent-based decision-making need to be factored into risk assessments.
Hansen Bock draws a parallel with the Industrial Revolution. Initially, machines made individual operations more efficient, but people still had to move the work between them. It was only when factories were organised around assembly-line production that much of the manual work could be removed.
AI is facing a similar transition, moving from individual tasks to connected, automated workflows in which decisions need to be made along the way. But when people no longer control every handover, new challenges arise.
“They have no conscience, no sense that ‘this is a bad idea’,” Hansen Bock says of AI agents.
Imagine three agents, each carrying out a different part of a task. The first receives instructions about what it can and cannot do and then passes the work on. But how do you ensure that the right restrictions are applied at every stage? And what happens if the agents disagree about priorities?
In a conventional organisation, there are procedures for escalating such questions to the responsible manager. Businesses need to determine how similar conflicts should be handled between autonomous AI agents.
“Who has the overall view? At what point should we as humans step in? Are we able to trace a process that has gone through several stages and identify weaknesses that should have been corrected along the way?” Hansen Bock asks.
The key is being able to understand and follow what happens between the agents. Decisions must be traceable. And before the agents are given free rein, the limits of their autonomy need to be clearly defined.
Hansen Bock believes many of the answers can be found in well-established security principles: identity management, access management, logging and traceability.
“Zero Trust is the principle that applies. The person or entity you are dealing with is not who they claim to be until they have identified themselves.”
The principle means that identity and access must be verified. Hansen Bock believes this also needs to be incorporated into the way AI agents communicate and carry out tasks on behalf of one another. The context in which AI agents operate must follow each handover throughout the process.
“The agents need to identify themselves, state what they want to do, and receive and acknowledge that they have understood the task.”
When it comes to access management, an agent should not have greater access to systems and data than the task requires.
Hansen Bock points to decades of experience in governance and control across large, complex IT ecosystems. Vivicta works with cybersecurity, AI and digital platforms, and with how organisations can retain control over data, technology and critical services.
But this is not only about adapting security solutions to handle the autonomy of AI agents. Businesses also need to have control over the information agents use as the basis for their decisions.
Marianne Kruse-Madsen, Head of Cybersecurity at Vivicta Norway, points to a familiar challenge in information management. Businesses hold large volumes of data, but often lack clear classification, ownership and an overview of who should have access.
AI can quickly bring together information that previously had to be retrieved manually from several different sources.
“If you do it yourself, it takes time. You search across different sources and ask people, and you may not find everything. With AI, you plug it in, and seconds later you have a result on the screen.”
It is therefore important to prevent data from being made available unintentionally. Kruse-Madsen warns both about information ending up in the wrong hands and about results becoming misleading when the underlying data is not good enough.
“If, for example, you have a lot of the same type of information with very small discrepancies, there is no guarantee that AI will produce the correct answer.”
If one agent relies on incorrect or incomplete information, the next agent may build on that error. As more stages become automated, it therefore becomes essential to know what the data contains, who owns it and how it can be used.
Kruse-Madsen has seen many organisations adopt new tools before the necessary controls are in place.
“I have seen several cases in our work where we have had to stop and take a couple of steps back because the necessary governance was not in place.”
She emphasises that the problem is not only about information security. It is also about being able to trust the results, particularly when they are used to make decisions.
“If some of our customers start using the data they receive as a basis for decision-making, and there is doubt about whether the result is 100 per cent correct, but they trust it blindly, that can be quite dangerous.”
While AI creates new security challenges, the technology can also help to address them.
One potential use case is automated vulnerability management, where systems not only detect issues, but also take corrective action, report on what has been done and provide recommendations for improvement.
This is closely related to Vivicta’s work in cybersecurity.
It also illustrates one of the challenges of autonomous processes. A system that is allowed to fix a vulnerability on its own must have clear limits on the actions it is permitted to take.
These are precisely the kinds of trade-offs businesses need to make when deciding how much responsibility AI agents should be given.
Hansen Bock believes it can make sense to allow agents to carry out parts of a process entirely autonomously, while introducing control points where the risk is greatest.
“To make this work, we need to be very good at instructing the agents, as well as testing and quality-assuring the results,” she says.
“The control mechanism cannot sit on every single function.”
Both she and Kruse-Madsen emphasise the need for thorough work on information, risk and access management.
At the same time, Hansen Bock warns against turning human control into an obstacle that simply shifts the problems elsewhere.
To realise the benefits of AI, security needs to be built into the processes. As AI agents take on greater responsibility, the ability to maintain control and freedom of action becomes a prerequisite for trusting the technology.
“You should be the controller, not the gatekeeper. If you stand there blocking one door, the AI agents will try to get through another.”