An AI Supplier Update Is an Operating Change, Not Just a New Feature

Procurement needs a plan for changed model behaviour, revised integrations and rollback before an AI service becomes embedded in essential work.

The purchase does not freeze the service

A business may approve an AI service based on a particular workflow and observed behaviour. Over time, the supplier can change the model, interface or integration. Some changes may improve the service. Others may alter assumptions on which the original approval depended. The organisation needs a way to recognise the difference.

The NIST Generative AI Profile is a voluntary risk-management resource published in July 2024. Its lifecycle framing is relevant to this question, but the operating recommendations below are CYPAPER's analysis. They are not a claim that NIST certifies suppliers, dictates a particular contract or guarantees that any update process eliminates risk. The focus is continuity of a locally evaluated service.

Document what the business actually approved

The approved use should describe the task, relevant data, expected output and the people or systems relying on it. A broad statement that a tool is suitable for productivity work provides little help when its behaviour changes. An operational record makes the original assumptions visible.

For illustration, a fictional team may approve assistance for classifying incoming documents into a limited set of queues. That is more specific than approval to automate document handling generally. If an update begins offering recommendations about the substance of the documents, the feature may fall outside the original evaluation. The organisation can then assess the additional role rather than treating it as automatically covered by a familiar supplier name.

Ask how changes will become visible

A supplier relationship should clarify how relevant updates are communicated and what choices the customer has. The precise arrangements vary by service and contract; they should be checked rather than assumed. A notification that arrives after an important change creates a different planning problem from one that allows preparation.

Technical teams and commercial teams should agree which changes need attention. A cosmetic interface adjustment may be routine. A new data destination, permission requirement or alteration to a critical output deserves a more deliberate review. This classification helps avoid two failures: treating every release as an emergency and allowing material changes to pass unnoticed because they are packaged among ordinary improvements.

Maintain a small regression exercise

The organisation does not need to repeat every original assessment for every update. It does need a way to check whether important behaviour still meets the task's standard. A focused collection of representative and consequential cases can provide a practical starting point.

Compare the updated service with the expected outcomes, record meaningful differences and investigate the ones that could affect delivery. The test should include exceptions already discovered in real work. It is not proof of universal reliability. Its value is local: it checks whether the conditions that supported this use remain credible. Where the service has changed substantially, a broader reassessment may be justified.

Make rollback an actual option

A plan to revert is useful only if the necessary previous version, settings or alternative process remain available. The team should verify those conditions before relying on rollback in a continuity plan. Some services may not support the kind of version control a customer expects.

If a direct technical rollback is unavailable, an alternative could be a temporary manual process or a narrower use of the tool. That option has staffing and delivery implications. It should be assessed as a real operating arrangement, not added to a plan as a reassuring sentence. A fallback that nobody has time or authority to use does not provide much practical protection when the normal workflow becomes unreliable.

Watch the integrations around the model

A business process can fail even when the model itself appears to improve. An output format may change, an integration may interpret a field differently or a downstream system may assume a sequence that no longer holds. Evaluation should therefore follow the service through its actual connections.

Use a fictional end-to-end case and observe where information moves, which system acts and how exceptions are handled. A result that looks correct on a screen may still produce an unsuitable record elsewhere. This is why change management belongs to the owner of the business process as well as the technical team maintaining the connection. Each sees a different part of the consequences.

Budget for maintaining suitability

An initial purchase decision should include the work required to keep the service suitable. Someone must monitor relevant changes, run checks and maintain the fallback. If these activities have no owner or capacity, the apparent cost of the service understates its operating requirement. This is not an argument against adoption. It is an argument for comparing alternatives on a complete basis. A lower subscription price may be attractive, but the business also needs to understand the review and integration effort associated with the actual workflow.

Preserve a decision history

When a change is accepted, record what was reviewed, which limitations remain and who owns the follow-up. This helps a future team understand why the workflow was allowed to continue. It also prevents later staff from interpreting the mere existence of an integration as evidence of a complete assessment.

My conclusion is that AI procurement needs an ongoing operating relationship with change. A supplier's improvement can create genuine value while requiring local adjustments. The responsible buyer can explain what has changed, what was checked and what happens if the result is unsuitable. That discipline turns an evolving service into something the organisation can manage, rather than a moving dependency whose consequences are discovered only after delivery is affected.

Gabriel Nicolaev — CyGuru

Sources & further reading

NIST — Generative AI Profile, published 26 July 2024 ↗

Editorial photograph for illustration.

FOLLOW THE CONNECTION

Another angle on this subject

Continue through related ideas, decisions and consequences.

World & Power

South Sudan Shows Why Early Warning Needs Spending Authority

A new WFP and FAO funding announcement highlights the decisions required to turn a forecast into practical protection before a flood.

Money & Business

Services Exporters Need a Margin Map, Not Just a Demand Forecast

The WTO’s split outlook for goods and services raises a practical question: which contracts still create value after travel, delivery and operating costs?

Money & Business

A Discount Can Sell the Customer a More Expensive Service

Discounting deserves an operating review when a lower price brings extra exceptions, support demands and commitments the original offer did not include.