Buy a packaged tool when it can complete your workflow in the system of record, enforce your approval rules, and hand over usable records if you leave. Build the parts that require control the package cannot demonstrate. Test both options on the same job before you sign a subscription or commission engineering work.
Illustrative scenario. Accounts payable receives supplier invoices by email. A tool reads each invoice, matches it to a purchase order, and prepares a draft payable in the ERP. A quantity mismatch goes to the purchasing queue. The team already has a rule that changes in supplier bank details go to a separate verification process. The choice turns on what happens when a single invoice hits both exceptions, an operator resolves one, and the ERP times out as the workflow tries to save the result.
Set the pilot boundary before comparing tools
Define the input, system of record, permitted actions, and finish line. For the illustrative invoice workflow, a case counts as complete when the ERP has a correct draft or an operator has resolved the exception and recorded the outcome. An item sitting in an exception queue is still open. If the team has yet to agree on who owns each handoff, start with the AI workflow audit guide.
Give each candidate the same approved sample and expected outcomes. Keep production writes disabled and replay side effects in a sandbox. Include a normal case, conflicting exceptions, a duplicate input, and an interrupted write. First demonstrate the existing process on the same cases so reviewers can see which steps each option actually removes or adds. For a custom option, use a thin prototype through the riskiest handoff and mark unbuilt paths as unproven.
Put deterministic checks around clear rules such as whether an approver has authority for a given entity. Use a model where it helps interpret an invoice or an ambiguous message. Anthropic distinguishes predefined workflows from agents whose model directs the next steps and recommends adding complexity when the task warrants it. Ask each candidate to show the simplest workflow that meets the brief. Anthropic, Building Effective Agents
Score the same evidence for buy and build
Use 0 when a candidate offers only a claim or workaround with no demonstration, 1 when it demonstrates part of the requirement and documents the remaining work, and 2 when it demonstrates the full requirement on your pilot workflow. Record the artifact and its owner alongside each score. Mark mandatory requirements before the pilot. Any mandatory item scoring below 2 blocks adoption until the candidate passes a retest. For options that clear those gates, add the scores as a fit summary and review cost separately.
| Criterion | Packaged offering | Custom build | Evidence for a score of 2 |
|---|---|---|---|
| Integration fit | Check which connector actions and ERP fields your plan supports. Test writes and retries with your permissions. | Prototype the connector for your ERP version, custom fields, and service account. Assign ownership for schema changes. | Sandbox replay shows the correct record, one write after a retry, and a visible recovery path for failed items. |
| Policy ownership | Ask who can change approval rules, stage updates, inspect past versions, and roll back. | Put rules under version control and name who approves, deploys, and reverses a change. Give operators the access they need. | An authorized person changes one routing rule, reruns affected cases, rolls back, and can identify the version applied to each case. |
| Model flexibility | Confirm which model or version controls the package exposes, how updates are disclosed, and what fallback is available. | Keep model inputs and outputs behind a documented interface. Assign someone to maintain the evaluation set and approve changes. | A model update or fallback is exercised against the same cases and meets the agreed quality and latency thresholds. The team can identify the version used. |
| Exit readiness | Request actual exports, the access window at termination, and the scope of deletion support. | Confirm contract rights and access to repositories, configuration, data, and deployment instructions. Budget for handover. | A second operator restores sample records, approval history, and pending work into a replacement queue within the agreed recovery target. |
A connector catalogue tells you where to start. In the ERP sandbox, interrupt the response after a successful write and see whether the next attempt creates a second draft. Have an operator locate the original record from the workflow history. Also rehearse a rule change while cases are already waiting for review. Agree which rule version applies to open cases and verify that the record captures that choice.
Model flexibility depends on your operating constraints. If a managed model meets them, a documented update process and a manual fallback may be sufficient. If a particular hosting location or model version is mandatory, ask for evidence that the offering can enforce it. Owning custom code still leaves dependencies on model APIs and their output behavior. Replay changes against the same cases, including the awkward ones that account for most of the pilot’s review work.
NIST’s generative AI profile recommends defining responsibilities for third-party incidents, testing fallback procedures, and planning for deactivation and downstream dependencies. Use those points to decide which proofs belong in procurement and which belong in the operating runbook. NIST AI 600-1, §§ GOVERN 1.7 and 6.2
Put a cost on running the workflow and leaving it
Compare both options over the same planning period, case mix, and definition of completion. Count subscription or infrastructure charges, implementation, integration upkeep, and the staff time spent reviewing exceptions, reconciling records, and handling incidents. Include the cost of cases still open in the period and track the backlog alongside cost per completed case. FinOps distinguishes infrastructure units such as cost per token from business units such as cost per case resolved. The cost-per-completed-task guide shows how to build the denominator and account for review work. FinOps Foundation, Unit Economics
Estimate exit costs separately using a rehearsal. Export a completed case with an override and one pending case with its attachments. Check whether IDs and relationships survive, whether the approvals have useful timestamps and actors, and which workflow rules must be recreated. Include preparation you pay for while the system runs, such as scheduled exports and restore tests. Estimate the one-time work for remapping, reconciling, running both systems during cutover, and retiring the old path. For a custom system, include the effort for a new maintainer to deploy it and recover a stuck queue. Record estimates and unknowns separately so procurement can see which assumptions still need proof.
Make the recommendation and assign an owner
Buy when the package passes the gates and your team can sustain its operating cost and exit plan. Build when an unmet requirement warrants the engineering and there is an owner for deployment, evaluation, and incident response. A hybrid option can use a packaged extractor while your own application handles approvals, the queue, and ERP writes. Include the glue code and its support in that option’s estimate. If the viable candidates add too much review work or leave a mandatory control unresolved, keep the current workflow and narrow the pilot.
Record the selected option, evidence for each gate, the recurring and exit cost assumptions, and the person who can stop the workflow. Keep the signed-off test cases so a renewal or handover starts from the same evidence.
Sources
- NIST publication page for the Generative AI Profile and full NIST AI 600-1 report
- FinOps Foundation, Unit Economics
- Anthropic, Building Effective Agents
- Insonl, AI Workflow Audit
- Insonl, Calculate the Cost per Completed Task in an AI Pilot
- Insonl engineering services and free AI workflow audit
Request an audit for one workflow
Insonl builds custom systems and charges for implementation or other follow-on work you choose after the free audit. Bring one workflow, its current approval rules, and the system it needs to update to the free AI workflow audit.
