insonl
← journal

AI Workflow Audit: What It Covers and How to Run One

date
20 November 2026
by
Insonl
read
8 min read

A common failure in operational AI projects is mundane: a team chooses a model before inspecting the workflow it is supposed to sit inside. Someone demos a prompt against clean sample files, leadership signs off, and weeks later engineers discover that real inputs arrive as skewed scans, product codes live in an unindexed ERP table, and exception rules exist only in one supervisor’s head.

An AI workflow audit prevents that sequence. Done properly, it is an engineering inspection of how work moves through a business today—what enters, which systems are read or updated, where staff spend repetitive hours, what an error costs, and whether an AI model is actually better than conventional code or a process change.

This guide covers how to run an audit, what the deliverable should include, and when to skip AI.

Step 1: Map the workflow as it runs, not as the SOP describes it

SOPs describe how a process was meant to work. They rarely show daily workarounds: copying purchase order lines out of emailed PDFs into a spreadsheet, messaging a warehouse coordinator to check stock, or correcting customer typos by hand before updating the CRM.

Pick one or two high-friction operational flows—order entry, invoice preparation, customer enquiry triage, or content screening—and trace every step from trigger to final database write:

  1. Inputs and formats. List every channel and file type: emailed PDFs, scans, faxes, chat threads, spreadsheets, and web forms. Note how inconsistent they are.
  2. Lookups and transformations. Watch what an operator opens on their second monitor. Which ERP screen, price list, or inventory table do they check to make a decision?
  3. Decision rules and exceptions. Separate routine cases from edge cases. Ask operators which cases forced manual escalation during the past week.
  4. System writes and ownership. Record every system updated at the end of the task, who controls its schema, and whether a documented API or database interface exists.
  5. Real historical samples. Pull a representative batch of past records—not cherry-picked clean ones. In Oldcastle APG’s public order-entry write-up on AWS (a third-party case, not an Insonl client), the authors recommend starting with the highest-volume document type and a sample of 50 to 100 real orders to scope a proof of concept.

If you cannot pull representative historical records paired with their final human-approved outcomes, you are not ready to build an AI pipeline. Fixing logging and data capture comes first.

Step 2: Score every candidate on four operational axes

Once real steps and samples are documented, score each candidate workflow across four axes. Novelty is not one of them.

Criterion Weak candidate for AI Strong candidate for AI Engineering implication
Volume and repetition Infrequent, ad hoc, or burns minimal staff time High volume of repetitive interpretation or triage Sets the ceiling on payback against build and hold costs
Rule clarity Senior staff disagree on valid outputs; rules shift constantly Clear definition of a correct output despite messy inputs Determines whether you can build an evaluation set
Error cost and reversibility Silent errors cause regulatory breaches or financial loss with no cheap check Reversible mistakes, or errors caught quickly in a human review UI Dictates guardrails, confidence routing, and mandatory sign-off
Data access and integration Source of truth sits in closed tools, personal chats, or dirty tables Systems of record expose reliable read/write APIs or databases Drives most of the real implementation and maintenance effort

1. Volume and repetition

Do the arithmetic before discussing models. Measure how many items arrive per month and how many minutes of manual work each item takes.

Public operational cases show what real volume looks like. In MyStorage’s Q2 2026 case study with EMIA (a Vietnam-based third-party case, not an Insonl client), automating 56% of outgoing customer enquiry replies meant roughly 1,784 automated messages per month at an average saving of about 3 minutes per outbound message—saving 89 hours of manual work per month. At larger scale, Oldcastle APG had more than 200 customer service representatives spending roughly four or more hours per day keying around 13 million order lines per year into Infor M3.

If a workflow saves only a handful of hours a month, a custom AI build will cost more to maintain than the labor it saves.

2. Rule clarity and architecture fit

Language models handle variable inputs well—unstructured purchase orders, plain-language customer enquiries, or user-submitted text. They handle contradictory business policy poorly. If two senior staff members disagree on the right outcome, an AI system will reproduce that inconsistency at speed.

Anthropic’s engineering guide on building effective AI agents separates workflows (where LLMs and tools follow predefined code paths) from agents (where an LLM dynamically directs its own processes and tool usage). Their core recommendation fits operational audits well: find the simplest solution possible and only increase complexity when needed. For operational tasks with well-defined inputs and outputs, a predefined extraction, classification, or routing workflow with deterministic checks is usually the right starting point—not an autonomous agent loop that trades predictability and cost for unneeded flexibility.

3. Error cost and verification design

Never design a production workflow around assumed perfection. Even strong deployments leave a slice of volume for people: Oldcastle APG automated 95% of order lines in its first two weeks of production while flagging the remaining 5% for CSR review.

In WellRithms’ public AWS case study on medical bill processing (a third-party case, not an Insonl client), the workflow keeps an expert review step: AI reduced manual effort from roughly 8 hours per bill to 15–20 minutes of expert review alongside 70.1 seconds of average AI runtime, cutting turnaround from up to 5 business days to up to 1 business day. The source does not state why the review step is kept, so treat it as a design choice for that workflow, not a rule for all of them.

Insonl built pre-publication AI screening on our own Vietnamese-language platform, Review Nguoi Yeu. As documented in our moderation write-up, every submitted review is screened before publication for toxicity, hate speech, personally identifiable information, and safety risks, with a design allowing review of flagged content and ongoing tuning because automated checks carry false-positive and false-negative risk. An audit defines that screening-and-review boundary before engineering starts.

4. Data access and system integration

An AI tool disconnected from live inventory, customer records, or product catalogs quickly turns into a copy-paste sidecar that staff stop using. Audit every upstream and downstream touchpoint: ERP tables, CRM records, warehouse systems, and reporting databases.

Verify authentication, API stability, and master-data hygiene. If product SKUs are duplicated across spreadsheets or customer IDs do not match between billing and support, flag data cleanup as a prerequisite rather than hiding it inside an AI build.

What the audit deliverable should contain

A useful audit ends with a written document both an operations lead and a CTO can evaluate—not a generic slide deck. Four components belong in the deliverable:

  1. Current-state workflow map. Concrete steps, input formats, manual effort, exception paths, and connected systems.
  2. Scored recommendations—including “no AI” calls. Candidate workflows ranked by operational payback, integration complexity, and risk, with explicit reasons for anything rejected.
  3. Target architecture and evaluation plan. Where deterministic code runs, where an LLM or vision model runs, how outputs are checked against master data, how staff review exceptions, and what test set from your real records defines go-live accuracy.
  4. Fixed-scope build estimate and operating assumptions. Delivery scope, timeline, ongoing inference and hosting costs, monitoring needs, and the production metrics that confirm whether the build paid off.

When the right answer is “don’t use AI”

A credible audit rejects AI whenever the workflow does not justify it. Four patterns belong in the “no AI” column:

  • Deterministic rules already cover the inputs. If data arrives in structured CSV, XML, EDI, or web-form fields, write standard code, SQL, or validation scripts. Deterministic software is cheaper, faster, easier to test, and does not hallucinate.
  • Core operational software is missing. If jobs, invoices, and customer records live in scattered spreadsheets and chat threads, an LLM only automates disorganization. Much of practical operational engineering uses conventional software with no AI at all—for example, Insonl’s work for Vu Duy Auto delivered a public website, a workshop CRM covering appointments, jobs, invoices, and inventory, and a mobile app with no AI in the codebase.
  • Volume cannot justify maintenance. Production AI requires logging, regression checks when models change, and maintenance when input templates shift. Low-volume tasks rarely clear that threshold.
  • Verification costs as much as doing the work manually. If a reviewer must re-read an entire source document line by line to trust the output, net time saved collapses.

How Insonl runs a free AI workflow audit

Insonl is a Ho Chi Minh City engineering studio working across enterprise systems (ERP, CRM, and BI) and generative AI. Our method runs in four stages: Open, Map, Build, Hold.

We start with a free AI workflow audit. We inspect your workflow as it actually operates and deliver three things in writing: a workflow map, concrete recommendations (including where conventional software or no change beats AI), and a fixed-scope estimate. The audit is free, and follow-on Map, Build, or Hold work is optional—you can hand the document to your own engineers if you prefer.

For an engineering read on where AI pays off in your operations, request a free AI workflow audit.

Sources