What the programme covers
A validation dataset should test the intended use, not merely repeat the distribution used for training. Site independence, acquisition shift, prevalence, patient selection and reference-standard quality determine what a performance estimate actually means.
What a useful specification includes
A defensible request defines the clinical task, source evidence and acceptance criteria before patient-level data moves. The exact fields and thresholds depend on the intended model claim.
- Intended use and performance claims
- Target population, care setting and prevalence
- Independence from development patients and sources
- Comparator or reference-standard definition
- Required sample size and confidence intervals
- Subgroup, scanner and failure-mode analyses
Quality and validation controls
Validation integrity depends on frozen cohort rules, protected labels where appropriate and traceable analysis inputs. Any exclusions after evaluation begins must be recorded and justified.
- Patient and source independence
- Pre-specified inclusion and exclusion rules
- Ground-truth blinding and adjudication
- Missingness and indeterminate-case treatment
- Performance, calibration and subgroup reporting
Availability, rights and delivery
The data and evaluation design can be delivered as a controlled cohort, a secure analysis environment or an agreed evidence package depending on programme terms.
Public pages describe a sourcing and engineering capability, not guaranteed ready inventory. Each release remains subject to verified programme inventory, programme-specific authorization, privacy review, technical acceptance and buyer licence terms.
Questions, answered directly.
What makes a dataset external validation data?+
It is meaningfully independent from model development and represents the intended deployment setting under a defined protocol.
Can the same hospital contribute training and validation cases?+
Sometimes, but a stronger external design often separates sites as well as patients. The intended claim determines the required independence.
Are labels shown to the model developer?+
They may be withheld during a controlled evaluation to reduce tuning and reporting bias.
Can validation cover demographic subgroups?+
Yes when relevant fields are lawfully available, sufficiently populated and appropriate for the analysis.