Choose an AI supplier as a production risk, not just as a model

By Pascal Bouman··3 min read
AI team assessing model selection based on compute, distribution and consent

The choice: turn model selection into a production decision

Do not simply choose the highest-scoring model; choose the supplier that can demonstrate both the required quality and manageable deployment for one clearly defined workflow. This is an editorial recommendation based on the scope of the sources: NIST describes a voluntary framework for incorporating trustworthiness into the design, development, use and evaluation of AI products, services and systems. The framework therefore does not prescribe supplier selection or compare models. Use a benchmark as an initial measurement for the task, not as a standalone production decision. Also document which outcome is unacceptable: for example, unexplained output, missing audit information or a route without a workable fallback.

Compute and resilience are separate checkpoints

Available computing power is more than an infrastructure detail when a workflow becomes dependent on generative AI. The European Commission describes AI Factories as ecosystems that bring together computing power, data and talent to develop models and applications; this only supports the point that compute is a relevant building block, not that a particular supplier guarantees capacity or availability. Therefore, assess the expected load, response time, cost limit, outage route and who decides on these matters for your own application. Include security and resilience: NIST states that the trustworthiness of AI technologies depends in part on how secure they are and identifies Secure and Resilient as a primary characteristic of trustworthiness. This is not a certification of a specific implementation, but it is a reason not to infer this check from model performance.

Diagram of model quality versus operational vendor risks

Make terms, compliance and ownership assessable

The EU AI Act aims to promote human-centric and trustworthy AI and sets out, among other things, rules for placing AI systems on the market, putting them into service and using them, as well as requirements and obligations for certain situations. However, the specific rules that apply cannot be determined from this passage and depend on the role, application and context; seek specialist advice where necessary. Practical tool: before a pilot or renewal, complete this decision register: for each workflow, record the model and supplier, owner, data flow, quality threshold, capacity assumption, contractual change point, applicable obligation, evidence of verification, fallback and reassessment date. A decision register for this AI workflow: record the model route, owner, data flow, checkpoint and follow-up decision. This turns model selection into a repeatable decision rather than a one-off demo. This text does not provide individual legal, financial or professional advice about your AI application or supplier contract.

Your personal AI research team

Developments move too fast to keep up with everything yourself.

You need a research team that tracks changes, checks sources and decides what matters for your work.

Choose what you want to follow and receive only the updates that matter to you.

Updates tailored to your interests
Researched by specialist agents
Relevant insights, not daily noise

What do you want to follow?

You receive a confirmation email first and only join after clicking it. See the privacy policy.

Latest articles

Recent knowledge base articles selected for this page.