How AI Agents Support Audit Without Compromising Control

The field of audit has run on a fundamental compromise for decades.

Faced with 40,000 expense reports, a common approach has been to test a sample of 40. This isn’t a shortcut; it’s a compromise, a necessary tradeoff due to the constraints of massive data volumes and the tools available. For decades, the field has been defined by this kind of work: cross-checking PDF policy documents against Excel checklists, running rigid scripts that flag known violations, and tracing anomalies one by one.

Traditional tools like CaseWare IDEA and Diligent’s ACL gave us speed, but not necessarily intelligence. In an era of complex, rapidly changing business environments, that's no longer enough. We've all seen the headlines — from Wirecard to Volkswagen — where systemic failures were missed by traditional approaches.

These constraints are beginning to change. AI and agentic systems are giving auditors the ability to move beyond small samples and rigid scripts. The ability for AI to understand context, connecting policy documents with live data, as we’ll show, is only useful if it can be trusted, governed, and audited. This blog will explore the problems and KNIME’s solution.

As auditors, we’re trained to be skeptical. When we hear about AI agents, we don’t just see the opportunity; we see the risks. These concerns typically include:

• Ensuring data security and preventing sensitive data from being shared with a third party.
• Proving reliability and having a way to trust that the AI isn’t hallucinating.
• Maintaining an audit trail and being able to explain to a regulator exactly how the AI reached its conclusion.

Failing to address these concerns is precisely why many firms are investing in AI agents but failing to get results, because they can't govern, document, or audit their "black box" solutions. They get stuck in pilot mode, unable to validate the outputs, and projects stall.

The only way to solve the auditor's dilemma is to reject "black box" AI. A black box, where data goes in and an answer comes out, is fundamentally unauditable.

The solution is a "glass box" approach.

This is where KNIME's visual, workflow-based platform becomes an advantage. It allows you to build an agent where the AI's "brain" (an LLM) is given access to a strictly controlled library of "tools." These tools are individual KNIME workflows you build and control.

Here’s the critical difference:

• The AI suggests: Your auditor asks a question. The LLM reasons about what to do and selects a tool. For example, it might say, "I need to query the sales database."
• The Workflow executes: The KNIME workflow, exposed as a tool, performs the action. It runs the SQL query inside your environment. All heavy data processing happens securely on your side, keeping sensitive data safe and never exposing it to third parties.
• The AI summarizes: The workflow gets thousands of results, processes them, and sends only a summary (e.g., "5 anomalies found") to the LLM to be put into a human-readable sentence.

This framework solves the "black box" problem. Sensitive data never leaves your environment. And, importantly for an auditor, the KNIME workflow becomes the auditable evidence of the process.

When AI is governed well, it moves auditors from data inspectors to strategic partners. Once you have a framework, you can build agents to tackle many of the audit challenges:

• Challenge: Reviewing 100% of journal entries is rarely feasible.
  • Solution: An agent can review the entire population locally, flag anomalies (like odd hours or amounts), and provide an explainable risk score.
• Challenge: Reading thousands of contracts or policy documents is slow and error-prone.
  • Solution: An AI agent can review them all, extracting key terms, identifying deviations, and summarizing complex rules in plain English.
• Challenge: Control testing is periodic, and a breach could go undetected for months.
  • Solution: An agent can monitor controls continuously, running 24/7 to reconcile system access logs with HR data and provide real-time alerts.
• Challenge: Uncovering conflicts of interest requires cross-referencing data tables.
  • Solution: An agent can read and map these relationships automatically, connecting HR and vendor lists to instantly flag non-obvious connections.

For an audit to be reliable, its process must be repeatable, transparent, and documented. This is how the "glass box" approach is built in practice. Instead of complex code, you use visual workflows to define the agent's permissions and capabilities. You grant it access to specific tool-workflows: like a Policy Reader, a Database Query Tool, and an Anomaly Detector. The entire process, from authenticating the LLM to defining the tools, is visual, transparent, and self-documenting.

This approach is what separates an enterprise-ready agent from a "black box" AI pilot. It's a blueprint you can control, adapt, and — most importantly — explain to your stakeholders.

Now, let's see that "glass box" blueprint in action. The workflow from the previous section is deployed as a practical "Agentic Audit Assistant" — a secure data app that an auditor can use to connect policy documents (PDFs) with company data (databases).

Instead of spending days trying to find the right data, the auditor can simply ask:

• "Summarize the key rules in the Global Sales and Discount Authority Policy."
• "Now, show me every expense claim for 'Business Class' air travel that lacks the required VP pre-approval."
• "Is there any correlation between employees who frequently submit out-of-policy expenses and those who approve non-compliant sales discounts?"

The agent handles the work of finding and linking data. This frees the human auditor to focus on high-value activities: asking why, identifying control weaknesses, and applying professional judgment.

This blog post has outlined the why of the "glass box" approach. To learn the full how, check out our free e-guide.You'll get the workflows, the system messages, a link to try the Agentic Audit Assistant yourself, and the step-by-step guide to move past the "black box" problems that keep AI projects stuck in 'pilot mode.'