Modern workforce integration -- why AI agents need the same oversight as their human counterparts [Q&A]

Agentic AI is rapidly moving from concept to reality, prompting organizations globally to rethink how they integrate these technologies into their business operations. The use of AI agents in daily workflows is set to rise dramatically in the coming years, raising questions over what organizations need to do to manage them effectively, and what might happen if they fail to do so.

We spoke with Ann Maya, EMEA CTO at Boomi, about the evolution of AI agents, the steps businesses should be taking ahead of deployment, and why the principles of human workforce management may hold the key to responsible use.

BN: How quickly are enterprises’ plans for agentic AI progressing, and what needs to happen to enable more widespread adoption?

AM: AI agents are already offering huge benefits across a multitude of functions, from software development to marketing, finance, and HR. While the mass deployment of these agents is still predominantly distributed by major technology companies like Google and Microsoft, the landscape is quickly diversifying to AI agents that are more tailored to an organization's workflow needs.

Gartner says AI agents will be deployed in over a third of all enterprise software by 2028, which underscores the strong upward trajectory we’re on. This fast-paced growth means we can’t delay the critical conversations around the need for robust guardrails and careful oversight of agentic AI. If we don’t get ahead of the risks of unmanaged agentic workforces, there could be significant consequences when we start to see more widespread adoption -- especially when it comes to security, compliance, and the accuracy of AI outputs.

We’ve already started to see some early warning signs. Recently the Financial Reporting Council released a report highlighting the lack of auditing oversight in the Big Four accounting firms’ use of AI, calling into question the risk of error and whether their AI-augmented workflows are accurate, secure, and explainable. If the major accountancy firms aren’t auditing their use of AI, the outlook for other industries is concerning.

Those responsible for deploying AI agents need to ensure they have a way to measure their performance against the core competencies of their role and functions, just as they would their human workforce.

BN: What should business leaders prioritize as they look to accelerate their agentic AI adoption plans?

AM: While it may sound obvious, readying an organization’s data for agentic AI is a critical first step, but it’s often overlooked in the name of convenience and faster ROI. The pressure to drive operational efficiency and faster investment value has arguably never been higher, so it’s tempting to accelerate AI adoption first and worry about the data later. However, if the data that AI-driven decisions are based on is fragmented or incomplete, entire workflows could be compromised, leading to bias or errors. An AI co-worker is only ever going to be as valuable as the quality of the data that it learns from.

While data quality is essential, it’s only part of the equation. Agentic AI also needs contextual awareness to make accurate decisions. That means not only interpreting data in isolation, but understanding its meaning based on criteria such as business operations and decision-making frameworks. Real-time context, like current market decisions or regulatory requirements, is also needed to ensure AI has all the information it needs to deliver meaningful outcomes.

Robust security measures for AI are equally important. Just as organizations wouldn't let employees pursue outcomes at any cost, they need to consider what it means to manage an AI workforce. Agents often operate out of sight within digital systems and make changes at a scale and speed far beyond human capability. To maintain control of these hyperproductive entities, managers need visibility of their agentic workforce. As a result, organizations can inspect AI decisions for accuracy and manage security, nurturing a system of trust.

BN: What role do APIs play in the deployment of AI agents, and what processes should organizations have to manage them?

AM: APIs (Application Programing Interfaces) are without a doubt one of the key enablers of agentic functionality. For decades, they have provided an easy way to connect systems, data, applications, and services within an organization's digital ecosystem. Today, APIs activate agents, providing the tools needed for them to act upon insights and outcomes they derive.

APIs can also manage access and authorization, making them essential for streamlining workflows. It’s why they're used extensively in nearly every business. However, not all APIs are made secure and poorly managed ones can introduce vulnerabilities. APIs that were once created and forgotten about provide an opening to an organization's digital ecosystem that cybercriminals can exploit. To minimize this risk, organizations need an API strategy that prioritizes discovery, security by design and visibility.

That’s easier said than done. Managing APIs has become more complex as organizations have shifted away from monolithic ecosystems towards local and cloud-based applications to meet responsive and high-performance computing needs. Many enterprises have embraced multiple cloud environments and now the same pattern is happening with AI. As a result, there has been a significant increase in APIs to connect systems and integrate services.

To keep up with the growing number of APIs and the complexity that comes with a growing agentic landscape, organizations need to adopt a platform management strategy that unifies data, integrates processes, and provides clear visibility and simplified oversight across all systems. Crucially, federated governance must also be in place. Without it, detecting harmful or problematic activity becomes nearly impossible and compliance becomes far more challenging.

BN: How do you envision the future of AI and human workforces evolving?

AM: Right now, we're in learning mode -- testing, experimenting and discovering how AI can reshape the way we work. AI agents give us the space to explore the full extent of this potential and we need to embrace that. But at the same time, we have never worked alongside a capability with the talent and potential that AI can offer.

To get the most out of AI’s full capabilities, organizations need an AI management strategy that is centralized and guardrails that are thoughtful and adaptable. Only strong visibility, orchestration and governance over the data and tools agents use can provide the oversight needed to trust AI outcomes. In turn, that trust will bring confidence in how staff use agentic AI and act as the catalyst for agency in agentic workflows.

In the future, we'll be working with and for AI agents. But for this vision to become reality, organizations need to rethink how they get work done -- even if that requires re-imagining the flow of events or the entire process. Done right, agentic AI gives us the ability to unlock hyper productivity with human creativity -- empowering us to achieve more. Whatever that 'more' looks like is for us to decide.

Image credit: Paradee Paradee/Dreamstime.com