AI / MODEL VALIDATION
Model performance
Two perspectives on prediction quality: unseen drug-target relationships and experimental reference data.
- Training data
- ChEMBL 23Released in 2017
- Validation data
- Through ChEMBL 33Previously unseen relationships
- Test set
- ~100,000Drug-target pairs
Validation on unseen data
To assess prediction quality, the model was reparameterised using historical data from ChEMBL 23. It was then used to predict new drug-target relationships added through ChEMBL 33, which had not been seen during training. This is a time-split validation.
Accuracy and sensitivityHow performance was measured
Accuracy
The proportion of ligand-target pairs correctly classified as active or inactive.
Sensitivity
In this benchmark, the target-averaged probability of finding a known active compound within the top 20 hits. This is the definition used for this comparison, rather than a general recall metric.
EXPERIMENTAL REFERENCE
Comparison with in vitro assays
Experimental measurements are not perfectly reproducible. This historical comparison used 20 pairs of independent assays selected from the literature, measuring the same target or biological effect. For compounds measured in both assays, one result served as the reference for assessing the other. Model predictions were evaluated against the same reference.
Model predictions and experimental agreementNeural Network
Accuracy of the model predictions against the reference assay.
Experiment
Agreement between two experimental assays measuring the same target or effect.
What this comparison tells us
In this historical comparison, model predictions and assay-to-assay agreement fall within a similar range. This does not establish equivalence across all targets or experimental conditions. Predictions help prioritise compounds; experimental testing remains necessary.
