Trustworthy AI series, part four: how assurance shapes ...
Why trust looks different depending on the decision
Trust is not experienced uniformly. It takes different forms depending on the type of decision an AI system supports. Forecasting future demand, optimising constrained resources, shaping long-term strategy, or identifying risk patterns each introduce distinct expectations, risks, and accountability pressures.
This Insight focuses on how trust is built and evidenced in practice across different decision contexts. Understanding these differences is essential for decision-makers, because assurance effort is most effective when it is aligned with what is actually at stake.
Forecasting under uncertainty
Forecasting plays a central role in sectors such as energy, transport, and infrastructure. These decisions are defined less by precision than by uncertainty. Single-point predictions can create a false sense of certainty, leading to misallocation of resources and exposure to avoidable risk.
Trustworthy forecasting therefore depends on making uncertainty explicit. Probabilistic outputs, scenario ranges, and confidence intervals allow organisations to plan against risk boundaries rather than assume stability. This enables decision-makers to justify actions even when conditions change unexpectedly.
Within the National Energy System Operator’s Dynamic Reserve Setting activity, forecasting models were embedded directly into reserve-setting and system-balancing processes. Decisions are taken under significant time pressure and carry direct implications for cost, resilience, and security of supply. Trust was built not through headline accuracy alone, but through integrating uncertainty quantification and explainability into operational workflows. Operators could interrogate forecasts, explore scenario sensitivities, and understand how risk margins were derived, allowing the models to function as transparent decision-support tools rather than opaque automation.
Optimisation and trade-offs
Optimisation decisions are inherently about trade-offs. Cost, performance, resilience, and risk are balanced under operational constraints. Trust depends on whether these trade-offs are visible and defensible to those accountable for the outcomes.
If stakeholders cannot see how objectives are prioritised or how constraints shape recommendations, optimisation remains a black box, regardless of performance gains. Explainable optimisation addresses this by exposing how decisions are formed and how different priorities influence outcomes.
In a project with Inmarsat, interpretable optimisation models were applied to satellite resource allocation and network planning. Trade-offs existed between service quality, capacity utilisation, cost, and system resilience. By integrating explainable structures and uncertainty quantification into the optimisation process, engineering and operational teams could trace how constraints and objectives shaped recommendations. This transparency allowed optimisation outputs to be treated as defensible decision-support rather than opaque automation within a mission-critical environment.
Strategic planning under uncertainty
Strategic planning decisions carry long-term and systemic consequences. They influence investment priorities, regulatory posture, network resilience, and social outcomes. In this context, trust extends beyond technical performance to include how assumptions are framed and how uncertainty propagates over time.
Assurance for strategic planning requires more than accurate forecasts. It requires transparency around scenario construction, sensitivity to key assumptions, and clear representation of uncertainty. This allows decision-makers to evaluate not just likely outcomes, but the range of plausible futures they may need to respond to.
In Scottish and Southern Electricity Networks’ Vulnerability Future Energy Scenarios programme, probabilistic modelling was used to explore future network investment under different conditions, including impacts on vulnerable households. Trust depended on clear articulation of assumptions and scenario dynamics, allowing decisions to be approached as informed responses to uncertainty rather than fixed commitments to a single forecast.
Risk identification and assurance thresholds
Risk-focused systems operate where both false confidence and excessive caution have consequences. Over-sensitive systems generate noise and erode trust. Under-sensitive systems allow genuine risks to propagate unnoticed.
Trust in this context depends on calibrated thresholds, reproducible behaviour, and stable explanations that hold under scrutiny. Assurance mechanisms ensure that alerts are meaningful, consistent, and defensible over time.
In nuclear operations, this balance is particularly visible. Within the EDF Cracksmith context, AI-supported analysis informs risk identification and intervention prioritisation. Governance stakeholders require evidence that outputs are auditable, reproducible, and stable across operating conditions. By combining interpretable modelling with calibrated thresholds and formal assurance artefacts, risk identification becomes a controlled and defensible process rather than an opaque technical exercise.
A consistent pattern beneath different decisions
Across these decision types, the underlying pattern is consistent. Trust stabilises when uncertainty is made explicit, trade-offs are revealed rather than hidden, and decision boundaries are governed rather than assumed.
What differs is not the need for trust, but how it must be engineered and evidenced to support safe, defensible decision-making in each context. Forecasting prioritises uncertainty representation. Optimisation demands transparency of trade-offs. Strategic planning requires scenario clarity. Risk systems depend on calibration and reproducibility.
Recognising these differences allows organisations to focus assurance effort where it delivers the greatest operational value.
Looking ahead
Trustworthy AI is not achieved through a single technique or framework. It is shaped by how assurance principles are applied to the decisions that matter most.
In this Insight, we have explored how trust manifests across different decision contexts and why assurance must be tailored accordingly. In the next Insight, we turn to the role of mathematical rigour in making this possible, and why advanced mathematics underpins resilience, auditability, and confidence in high-consequence AI systems.
