How should health AI be tested?

Google's November 4 health AI announcement describes an evaluation framework organized around accuracy, safety and usefulness. It combines closed-question benchmarks, automated model-based raters and expert human review, covering common and higher-risk situations. Google says it uses the framework to evaluate health responses and is sharing the method with the wider sector. This is a disclosure of an assessment approach, not an independent finding that a particular consumer product improves health outcomes.

A framework follows the system through its lifecycle

The associated paper, submitted to arXiv on November 3, anchors evaluation to critical user journeys and distinguishes base-model development, product development and monitoring after deployment. That matters because a model answering an isolated question and a product supporting a sequence of decisions present different evaluation problems. A published method gives an outside reader a structure to examine; its existence does not tell that reader how a specific deployed system performed within that structure.

NEXTECH analysis: separate the questions a score conceals

Our view is that the three dimensions should remain visible separately. A technically correct statement may be difficult to understand, while a clear explanation may omit a limitation that changes how it should be interpreted. Combining those results into one attractive number would conceal the trade-off. We would want a product report to name the task, describe the intended user and show which kinds of failure prevent release. This is an editorial criterion, not a claim that Google's framework or products lack those records.

Automated review needs a challenge process

Automated raters could make repeated evaluations easier to organize, but a convincing process must also explain how disagreement is resolved. We would ask which examples experts review, whether difficult cases are kept outside the development set, and how errors found after release become future tests. An evaluator should be able to challenge a passing judgment without relying on another fluent explanation from the same system. These questions concern the independence and repeatability of the evidence; they do not establish a defect in any particular rater.

Method disclosure is a starting point for evidence

The next useful disclosure would connect this methodology to a named use case, a defined population and a dated result, including failures and the conditions outside the assessment. That would help readers distinguish a promising evaluation plan from demonstrated performance. NEXTECH has not audited the framework, reproduced the evaluations or assessed clinical outcomes. Our judgment is that publishing the method is useful because it makes more precise questions possible. It leaves the answers about any specific product to evidence that can be inspected.