In Silico vs In Vivo ADMET: How Accurate Is AI ADMET Prediction?

An in silico ADMET panel is a ranking instrument, not a measurement. It takes a structure and returns a score per endpoint, derived from how that structure resembles molecules in a training set of past experiments. An in vitro assay measures a real property of the real molecule in a defined system — a cell monolayer, a microsomal incubation, a patch-clamped channel. An in vivo study measures what a whole animal does with the compound. The three layers answer different questions, and the correct use of the computational layer is to decide

which compounds and which endpoints are worth the assay. If you are new to the endpoint vocabulary itself, start with our practical guide to ADMET prediction for drug candidates; this article takes the other half of the problem — how the predictions relate to experimental data.

The three layers, side by side#

The decisive row is the second-to-last one. An assay that fails usually fails loudly. A model asked about a scaffold unlike anything it was trained on returns a confident-looking number anyway.

How accurate is AI ADMET prediction, concretely?#

Accuracy is per endpoint, and the only honest figures are benchmark figures.

ADMET-AI (Swanson et al., Bioinformatics 2024) trains Chemprop-RDKit graph neural networks on 41 ADMET datasets from the Therapeutics Data Commons (Huang et al., 2021), of which 22 form the TDC ADMET Leaderboard. On its own single-task models the published performance is R² > 0.6 for 5 of the 10 regression datasets and

AUROC > 0.85 for 20 of the 31 classification datasets, with the best average rank among the models evaluated on the leaderboard. That is a useful instrument and also a plain statement that roughly half the regression endpoints explain less than 60% of the variance in their own curated test data.

ADMETlab 3.0 (Fu et al., Nucleic Acids Research 2024) covers 119 endpoints trained on over 400,000 entries using a multi-task DMPNN, and reports uncertainty estimates alongside predictions.

ProTox 3.0 (Banerjee et al., Nucleic Acids Research 2024) covers 61 toxicity endpoints — acute toxicity, organ toxicity, molecular initiating events, Tox21 pathways and toxicity off-targets — each validated on independent external sets and reported with a confidence score.

Three things follow. Benchmark numbers describe held-out splits of curated public data, not your series. An AUROC is a ranking statistic, so a good one is compatible with poor calibration of the individual probability. And a long endpoint list is coverage, not accuracy: a broad panel buys you more questions asked, not more questions answered well.

Where the in silico layer fits in a triage funnel#

Use the layers in order of what they cost to be wrong about.

1. Design and library triage (in silico). Hundreds to millions of structures, most of which do not exist yet. Here prediction has no competitor, because there is nothing to assay. Rank, deprioritize the bottom, and keep the ordering rather than the absolute values.
2. Series-level reality check (in silico, with controls). Before trusting a ranking on a new scaffold, run compounds whose experimental values you already hold — marketed drugs in the same class, or your own historical assay set — through the same panel. If the model cannot rank your known compounds, it cannot rank your unknown ones.
3. Confirmatory in vitro (on the flagged minority). Permeability, microsomal stability, CYP inhibition, plasma protein binding, hERG. This is where a predicted liability becomes a number with units.
4. In vivo PK and tox (on the survivors). The questions in silico panels structurally cannot answer: formulation and dose-dependent exposure, enterohepatic recirculation, active metabolites, tissue accumulation, idiosyncratic findings.

The funnel is also a feedback loop. Every in vitro result on your own chemistry is a data point on whether the panel is informative for your chemical space, which is the only accuracy figure that governs your next decision.

Known failure modes#

Applicability domain#

QSAR-style models are most accurate for molecules similar to their training set, and similarity to the nearest training molecule is itself a good predictor of prediction error (Sheridan et al., J. Chem. Inf. Comput. Sci. 2004). Novel scaffolds, macrocycles, covalent warheads and heavily charged species sit further out, and the score does not shrink to signal it.

Worth knowing about implementations: ADMET-AI's prediction frame carries property values and DrugBank-approved percentiles, not a per-prediction uncertainty or an applicability-domain score. On .bloom* that missing score is surfaced as unknown rather than filled in with a placeholder, because a placeholder would silently assert that your molecule resembles the training distribution. The percentile against approved drugs is the useful substitute: it tells you where a compound sits relative to things that reached the market, which is a comparative statement rather than a physical one. See ADMET-AI outputs & interpretation.

Endpoint definitions that differ from your assay#

A predicted "Caco-2 permeability" is a model of one public dataset measured in one lab under specific conditions — pH, passage number, monolayer integrity, direction. Your CRO's Caco-2 number is a different realization of the same concept, and the correlation between them is a question to measure rather than assume. Classification endpoints are worse here, because a threshold applied during curation is baked into the label. Check that prediction and assay name the same measurement before comparing them.

Species translation#

In vivo data is not a ground truth for humans either. In the ILSI/industry survey of 150 compounds with 221 human toxicity events, animal studies showed a true-positive concordance with human toxicity of 71% for rodent plus non-rodent species, 63% for non-rodents alone and 43% for rodents alone (Olson et al., Regul. Toxicol. Pharmacol. 2000). Most ADMET models are trained on human-derived or mixed-source data, so a disagreement between a prediction and a rat study is not automatically the model's error — the two are describing different organisms.

Metabolism is the hardest layer#

Metabolic fate depends on enzyme kinetics, competing pathways and metabolite activity, and computation and experiment are complementary rather than substitutable here (Kirchmair et al., Nat. Rev. Drug Discov. 2015). Treat a predicted clearance or half-life as a hypothesis about rank order within a series, and never as an input to a dose projection.

Training-set bias#

Public ADMET datasets are enriched in drugs and drug-like compounds that got far enough to be measured. Toxic and poorly-behaved molecules are systematically under-reported, and endpoints assembled from different sources may share underlying compounds, so two "independent" predictions can correlate because their training data does. Treat a clean panel as a weaker signal than a dirty one: absence of a flag is largely absence of evidence.

Physicochemical control has limits#

The combined attrition analysis across AstraZeneca, Eli Lilly, GlaxoSmithKline and Pfizer found that controlling physicochemical properties during optimization helps identify candidate-quality compounds, but that further control is unlikely to move attrition rates much and that safety-related failures need additional work (Waring et al., Nat. Rev. Drug Discov. 2015). A panel that is green on drug-likeness filters has cleared a floor, not a bar.

Deciding when a flag justifies an experiment#

A predicted liability is worth an assay when the four conditions below hold together. Fewer than three, and you are usually buying a number you will not act on.

• Consequence. Being wrong changes the program. hERG, mutagenicity and hepatotoxicity kill candidates; a borderline logP does not.
• Decision reversibility. The flag would stop a decision that is expensive to undo — committing a scaffold, scaling a synthesis, dosing an animal.
• Domain plausibility. The compound sits inside the chemistry the model has seen, judged by nearest-neighbour similarity to the training or reference set, not by the score's apparent confidence.
• Discriminating power. The flag separates your compounds. If every analog scores 0.6, the panel is telling you about the scaffold, and the experiment to run is one assay on one representative — not fifteen.

Two corollaries. A flag that is consistent across orthogonal methods — different architectures, different training data — is stronger evidence than a single model's high score, which is why complementary panels are worth running together. And a flag on a structural alert (a reactive group, a known toxicophore) is mechanistically interpretable and should usually be resolved by chemistry rather than by an assay.

How to run it on .bloom*#

ADMET-AI runs inside .bloom* from SMILES — pasted, uploaded as CSV/SDF, or carried over from a previous job — and returns 41 ADMET endpoints plus 8 RDKit physicochemical properties per compound, as a table with a downloadable CSV and a result card on the canvas. A practical sequence:

1. Screen the series, with controls in the same run. Include compounds whose experimental values you already have. Their predicted-vs-measured agreement is your local accuracy estimate.
2. Read the percentiles, not just the scores. Each prediction is paired with its percentile against the DrugBank-approved reference set, and an ATC code narrows that reference to a therapeutic class — a CNS program and an antibiotic program do not share acceptable ranges.
3. Attend to the flags. Rows marked invalid_smiles or out_of_domain are not scores; treat them as a request to standardize the structure or to stop trusting the panel for that compound.
4. Cross-check with an orthogonal model. The Screen ADMET recipe chains ADMET-AI with DeepPurpose over the same input so that agreement and disagreement are both visible.
5. Keep the run, not the screenshot. Record model_version, the option set and the preprocessing flags with each run, and hold a fixed 20–50 molecule regression subset so you can tell a real shift from a settings change. See ADMET-AI limits & performance.
6. Hand the shortlist to the assay. The output of the computational layer is a prioritized list plus a stated reason per compound — which is exactly what an in vitro plate needs as input.

Note on the other two panels named above: on .bloom* both ADMETlab 3.0 and ProTox-3 are deprecated and not callable from a run — their upstream services are unavailable. Existing job results remain viewable, and ADMET-AI's toxicity endpoints (hERG, Ames, hepatotoxicity) are the current path. The ADMET overview lists what else the section covers.

Key takeaways#

• In silico ADMET answers "which of these, and in what order"; in vitro answers "what is this compound's value"; in vivo answers "what does an organism do with it". Substituting one for another is the recurring mistake.
• Published accuracy is per endpoint and per benchmark: ADMET-AI reports R² > 0.6 on 5 of 10 regression datasets and AUROC > 0.85 on 20 of 31 classification datasets (Swanson et al., 2024). Your own chemistry needs its own estimate.
• Prediction error tracks distance from the training set (Sheridan et al., 2004), and a model outside its domain does not say so.
• Animal data is a reference, not truth: 71% true-positive concordance with human toxicity for rodent plus non-rodent species (Olson et al., 2000).
• Spend an experiment when the flag is consequential, the decision is expensive to reverse, the compound is in domain, and the flag discriminates within your series.

Run an ADMET-AI panel on your series — with your known compounds included as controls — on the .bloom* platform.

References#

1. Swanson, K., Walther, P., Leitz, J., Mukherjee, S., Wu, J.C., Shivnaraine, R.V. & Zou, J. "ADMET-AI: a machine learning ADMET platform for evaluation of large-scale chemical libraries." Bioinformatics 40, btae416 (2024). doi:10.1093/bioinformatics/btae416
2. Huang, K., Fu, T., Gao, W. et al. "Therapeutics Data Commons: Machine Learning Datasets and Tasks for Drug Discovery and Development." NeurIPS Datasets and Benchmarks (2021). arXiv:2102.09548
3. Fu, L., Shi, S., Yi, J. et al. "ADMETlab 3.0: an updated comprehensive online ADMET prediction platform enhanced with broader coverage, improved performance, API functionality and decision support." Nucleic Acids Research 52, W422–W431 (2024). doi:10.1093/nar/gkae236
4. Banerjee, P., Kemmler, E., Dunkel, M. & Preissner, R. "ProTox 3.0: a webserver for the prediction of toxicity of chemicals." Nucleic Acids Research 52, W513–W520 (2024). doi:10.1093/nar/gkae303
5. Sheridan, R.P., Feuston, B.P., Maiorov, V.N. & Kearsley, S.K. "Similarity to Molecules in the Training Set Is a Good Discriminator for Prediction Accuracy in QSAR." Journal of Chemical Information and Computer Sciences 44, 1912–1928 (2004). doi:10.1021/ci049782w
6. Olson, H., Betton, G., Robinson, D. et al. "Concordance of the Toxicity of Pharmaceuticals in Humans and in Animals." Regulatory Toxicology and Pharmacology 32, 56–67 (2000). doi:10.1006/rtph.2000.1399
7. Kirchmair, J., Göller, A.H., Lang, D. et al. "Predicting drug metabolism: experiment and/or computation?" Nature Reviews Drug Discovery 14, 387–404 (2015). doi:10.1038/nrd4581
8. Waring, M.J., Arrowsmith, J., Leach, A.R. et al. "An analysis of the attrition of drug candidates from four major pharmaceutical companies." Nature Reviews Drug Discovery 14, 475–486 (2015). doi:10.1038/nrd4609