An AI workflow can work perfectly on the day of delivery and fail three months later. Nobody touched it. The world around it changed.
Demos never show this part of AI automation. This article explains what changes, how we notice it and what a monthly care fee pays for.
What changes
The models
AI model providers release new versions several times a year. They also retire old versions. When a version is retired, every workflow that uses it must move to a new one.
A new model is usually better on average. But "better on average" does not mean "the same on your task". A new version can:
- format numbers or dates differently
- write longer or shorter replies
- read an unclear invoice field in a different way
- follow one instruction more strictly and another less strictly
Each change is small. But in a workflow that writes to your accounting software or to your customers, a small change matters.
The APIs
Your other software changes too. An API is the interface that lets two programs exchange data, and APIs change. The accounting program releases a new API version. The email provider changes how logins work. A supplier changes the layout of its invoice PDF. Each of these changes can stop a workflow without any error message.
The people
The person who knew why a rule exists leaves the company. A new employee handles exceptions differently. A new product line appears that the workflow has never seen.
The prices
Model prices change. Usage grows as you give the workflow more work. Without monitoring, the monthly bill can grow and nobody notices.
How we notice: tests
Every workflow we build has a set of test cases. A test case is a realistic input together with the result we expect. For an invoice workflow, a test case is a PDF and the fields we expect from it. For email triage, it is an email and the category it should get.
We collect test cases from real work and anonymise them where needed. We also add difficult cases on purpose: an invoice with two VAT rates, an email in Russian, an enquiry with no address. Every time a real case goes wrong, we add it to the set.
We run the full set of tests:
- before we switch to a new model version
- after an API that we depend on changes
- after we change a prompt or a rule
- on a regular schedule, even when nothing seems to change
If a result is different from what we expect, we see it before your team does.
How we notice: monitoring
Tests check known cases. Monitoring watches real work every day. We monitor things like:
- Volume. For example, if a workflow usually handles 40 invoices a day and today handles zero, something is wrong.
- Errors. Failed API calls, timeouts and documents that the AI could not read.
- Cases sent to a person. If this number suddenly grows, the AI is less sure than before.
- Corrections. How often does a person change an AI draft before approval?
- Cost. Model usage per day and per workflow.
When a number moves outside its normal range, we get an alert. Usually we fix the cause before anyone on your side notices.
How we update a model
We do not switch models on the day a new one appears. We follow a fixed routine:
- Pin the version. Each workflow uses one exact model version, not "the latest". Nothing changes by surprise.
- Test the new version. We run the full set of tests on the new model and compare the results.
- Fix the differences. We adjust prompts or rules where the new model behaves differently.
- Run in parallel. Where it is useful, the new model runs next to the old one on real work for a short period. It does not affect the result yet.
- Switch and watch. We move the workflow to the new version and watch the monitoring closely for the first days.
Sometimes we choose a smaller and cheaper model for simple steps and a stronger one for difficult steps. We do not depend on one model provider, so we can choose for each task.
Why a monthly care fee exists
Everything above is continuous work. It does not stop when the build stops. That is why our workflows include a monthly care fee. It covers:
- Model and API updates. We follow the AI market and your software vendors, so you do not have to.
- Monitoring. We watch the numbers and fix problems.
- One small improvement a month. For example, a new category, a new rule or a new field in a report.
- A monthly report. Hours saved, errors, cases sent to a person and model cost.
The price depends on the number of workflows and on model usage. We give it on request, after the audit, when we know the scope.
Our promise for the care fee is simple: we keep you up to date. Models will change several times a year. Your workflow should continue to work anyway.
What you see as a client
Most of this work happens in the background. You see it in two places. The first is the monthly report. It shows hours saved, errors, cases sent to a person and model cost, compared with the month before.
The second is a short note when we change something important, for example a model version. The note says what changed, what we tested and what you may notice. If a change affects what the AI may do alone, you decide first.
What happens without care
Without care, a workflow usually does not fail with a loud error. It degrades. The share of drafts that need corrections grows slowly. People start to fix things by hand again. After some months, the team stops trusting the workflow and returns to the old way.
At that point, the company has paid for the build and still does the work by hand. A small monthly fee is cheaper than that outcome.
What you can ask any supplier
If you already have an AI workflow, or you plan to buy one, ask these questions:
- Which exact model version does the workflow use?
- Where are the test cases, and how many are there?
- When did they last run?
- Who gets an alert when the workflow stops?
- What did the workflow cost to run last month?
If nobody can answer, the workflow is a demo that has not failed yet. Read A demo is not a system for the full checklist.
Start with the process audit
A workflow that lasts starts with a clear process. Tests come from the rules you agree on. Monitoring comes from the numbers you want to improve. In a process audit, we map one process, write those rules and choose the first workflow. You get 3–5 automation opportunities with time saved, risk and price, and a fixed-price offer for the build. The audit costs €1,490 + VAT.