From AI to evidence

AI / MODEL VALIDATION

Model performance

Two perspectives on prediction quality: unseen drug-target relationships and experimental reference data.

Training data
ChEMBL 23Released in 2017
Validation data
Through ChEMBL 33Previously unseen relationships
Test set
~100,000Drug-target pairs

Validation on unseen data

To assess prediction quality, the model was reparameterised using historical data from ChEMBL 23. It was then used to predict new drug-target relationships added through ChEMBL 33, which had not been seen during training. This is a time-split validation.

Historical bar chart comparing the INVENTRO model in green, ChEMBL v28 in amber and SwissTarget in grey-green. The INVENTRO model has the highest bars for accuracy and sensitivity.Accuracy and sensitivity
Historical comparison using the same training and validation datasets. Original values and labels are retained; “Our model” denotes INVENTRO.

How performance was measured

Accuracy

The proportion of ligand-target pairs correctly classified as active or inactive.

Sensitivity

In this benchmark, the target-averaged probability of finding a known active compound within the top 20 hits. This is the definition used for this comparison, rather than a general recall metric.

EXPERIMENTAL REFERENCE

Comparison with in vitro assays

Experimental measurements are not perfectly reproducible. This historical comparison used 20 pairs of independent assays selected from the literature, measuring the same target or biological effect. For compounds measured in both assays, one result served as the reference for assessing the other. Model predictions were evaluated against the same reference.

Archived box plot: neural-network predictions in green and assay-to-assay agreement in amber. Both accuracy distributions occupy a similar range. The vertical axis runs from 0.5 to 1.0.Model predictions and experimental agreement
Figure from the INVENTRO website archive. Points, boxes and axis range are unchanged. The axis uses fractions: 0.8 means 80% accuracy; the displayed range is 0.5–1.0.

Neural Network

Accuracy of the model predictions against the reference assay.

Experiment

Agreement between two experimental assays measuring the same target or effect.

What this comparison tells us

In this historical comparison, model predictions and assay-to-assay agreement fall within a similar range. This does not establish equivalence across all targets or experimental conditions. Predictions help prioritise compounds; experimental testing remains necessary.