Automation maintenance: recover when an API changes

When a supplier API or source format changes, an automation should detect the problem, contain the affected work and give an owner a clear recovery path. That requires content validation, useful alerts, bounded retries, controlled replay and a manual fallback tested before go-live.

If you are choosing an automation implementation or maintenance partner, ask them to demonstrate this failure path. A workflow that transfers data correctly in a demo still needs a way to recover when an external system changes after launch.

Define what “successful” means for the business

A scheduled job can connect to a supplier, receive a valid response and finish without an execution error. Yet the result may be wrong: a price field is blank, an invoice is attached to the wrong customer, or only the first page of records was processed.

A useful success definition includes the destination. Did all expected records arrive, with the right meaning, exactly as many times as intended?

Typical changes worth designing for include:

• A field is renamed, removed or nested in a different part of the response.
• A supplier separates net price and tax, or changes the currency representation.
• Dates switch format or arrive without a time zone.
• Pagination changes and the workflow silently processes fewer records.
• A system returns an empty list while it is unavailable.
• Authentication expires or a rate limit delays processing.
• A request times out after the receiving system has already made the change.

These need different responses. Repeating a request may help after a temporary read failure. Repeating a completed write can create a duplicate order or invoice.

Build five recovery controls into the scope

1. Validate structure, meaning and completeness

Check required fields and types before applying data. Then check business meaning: currency, units, identifiers, dates and values that need explicit approval. Compare counts or reconciliation totals where there is a dependable expectation.

A fall in record count is a reason to investigate, not proof of failure; seasonal demand may legitimately change it. Similarly, zero may be valid in some fields. Choose checks with the workflow owner so alerts reflect the operation rather than an arbitrary technical rule.

Validate in a staging step before replacing trusted downstream records. Decide whether to quarantine only the affected items or pause the whole batch when consistency requires it.

2. Alert an owner with enough context to act

An alert should say which workflow and source failed, when it last worked, which records are affected and what processing has paused. Include a link to the incident or run record and the immediate fallback instruction.

Name a primary owner and a backup. State when somebody is expected to respond; an overnight alert does not create overnight support by itself. Avoid putting customer data or credentials into alert messages.

3. Separate temporary errors from changes that need a fix

Set bounded retries for failures expected to clear, such as a temporary read timeout or a rate limit, following the source system’s rules. Use increasing delays where appropriate rather than repeatedly hitting the source.

Missing fields, unexpected currencies and changed data structures need investigation. Authentication failures may need an authorised access fix. For an uncertain write result, first establish whether the destination already accepted it. Where supported, use a stable operation identifier so the same action cannot be applied twice.

4. Preserve failed work for controlled replay

Store enough information to identify and recover each affected item: source identifier, received time, relevant input, mapping version and processing outcome. Protect that store with suitable access and retention rules; a recovery queue should not become an indefinite dump of sensitive records.

After the fix, test a small replay, inspect the destination and then process the remaining items. Respect ordering when one record depends on another. Reconcile the replay against the failed set, including items already completed, explicitly skipped or still unresolved.

5. Give staff a workable manual fallback

Write down who pauses the workflow, which systems staff use, how approvals work and where manual actions are recorded. That manual log must be reconciled before replay, or recovery can duplicate work people completed during the outage.

Decide whether last-known values can remain visible. Some processes can use clearly marked stale information for reference; others must block quoting or execution until the source is verified.

Hypothetical example: a hotel-rate feed changes overnight

Consider a hypothetical destination management company in Dubai. Each night it imports contracted hotel rates from three suppliers, converts them to AED and updates the data used by its sales team. This is an illustrative failure scenario, not a client case study.

One supplier replaces a single rate field with separate net and tax fields. An unsafe workflow treats the missing field as zero and publishes invalid prices for 40 hotels. A completed run notification would give the team false reassurance.

With recovery controls in place, the workflow instead:

• Detects the missing field before publishing the affected supplier’s rates.
• Leaves the previous data marked with its age and blocks its use in new quotes until checked.
• Alerts the agreed operations owner with the supplier, affected records and last successful import.
• Holds the responses for investigation rather than repeatedly retrying the same incompatible mapping.
• Directs sales to verify rates in the supplier’s approved system and record any manual updates.

The integration owner checks what the new fields mean before changing the calculation. Is tax already included elsewhere? Are the values per room or per person, per night or per stay? Do currency, occupancy and validity dates still map correctly? Adding two fields without those checks can produce a plausible but incorrect price.

After updating and testing the mapping, the owner replays the affected records, reconciles manually checked entries and verifies representative rates in the destination. The stale restriction is removed only after the agreed checks pass. Any quotes created during the affected window are identified for review, rather than assuming a repaired feed also repaired past output.

Run one failure drill before expanding automation

Choose the workflow where silent failure would have the largest operational impact. Run the drill in a safe test environment or controlled copy, without sending real customer messages or creating live transactions.

Test at least a missing field, an unexpectedly empty response, a temporary outage and an ambiguous write result. For each, record:

1. Did the check detect it before bad data reached the next step?
2. Did the owner receive enough information to identify the affected work?
3. Did retries stop where they should?
4. Could staff use the fallback without inventing a process?
5. Could the held items be recovered without duplication?
6. Did reconciliation prove what was completed and what remained unresolved?

A failed drill is a useful delivery finding. Assign the gap, fix it and rerun the relevant scenario before treating recovery as complete.

What to agree with an automation maintenance provider

A straightforward workflow between two stable tools may need only modest checks and an internal owner. More critical or interconnected workflows justify a clearer maintenance arrangement, especially when a failure reaches customers, pricing or financial records.

Ask for explicit answers on monitoring coverage, support hours, severity levels, access to run history, responsibility for supplier changes and the scope of recovery work. Confirm who can pause or resume processing, who approves changed business rules and what documentation and workflow files you retain. Response targets should be written into the actual agreement rather than inferred from a general promise of support.

SolvTree’s integration and delivery capabilities cover scoped builds and ongoing programmes. Its process automation services are relevant when a workflow spans existing tools. Specify the recovery controls above in the project scope so implementation and maintenance expectations are clear.

If an existing automation has no tested recovery path, ask SolvTree to review that integration. Describe its inputs, destination and the consequence of incorrect output. Start the conversation with the failure you need to contain before discussing additional automation.