How ProTox-3 results compare to the bench

You have a ProTox-3 result. Here is what it supports.

• ProTox-3 predicts in vivo endpoints. That is not the same as having been compared to in vivo outcomes. Rodent LD50, hepatotoxicity, carcinogenicity and immunotoxicity are animal and clinical endpoints, so it is tempting to read a ProTox-3 call as an in-vivo-grade prediction. Every figure on this page was measured against a curated database label — SuperToxic, CPDB, CEBS, ChEMBL, Tox21, ClinTox — assembled from assays and animal studies someone else ran, under someone else's inclusion criteria. Nothing in the literature grades ProTox against a fresh animal study or a fresh assay. Read the general framing before any number below.
• Read agreement per endpoint and never average across them. On the authors' external validation sets the spread is wide: cardiotoxicity reaches AUC-ROC 0.95, while immunotoxicity reaches 0.74 with a kappa of 0.29 — barely above chance agreement. One "61-endpoint toxicity profile" contains both.
• Most of ProTox-3's headline endpoints have no figure in the ProTox-3 paper. Of its 61 models, only the 28 new ones carry performance tables in the 2024 paper. Acute toxicity (LD50), hepatotoxicity, mutagenicity, carcinogenicity, cytotoxicity, immunotoxicity and the 12 Tox21 pathways are carried over, and their published figures come from ProTox-II (2018) and ProTox (2014) — earlier models on smaller data. The paper's abstract says "all the ProTox 3.0 models are validated on independent external sets"; its body text does not support that for the carried-over 33. This page uses body text wherever the two disagree.
• No number here carries a sample size, because the papers do not publish one. ProTox-II defers dataset counts to a supplementary table; ProTox-3 defers them to the web server's "Model info" page. Only one n appears in primary text: 5252 ChEMBL compounds in the cardiotoxicity training set. An accuracy without an n is a floor, not a measurement.
• No independent external validation of ProTox exists. Every figure below is author-reported by the Charité group. The two papers that come closest to a wet-lab comparison are also theirs.
• It replaces no assay, and it has no regulatory standing. Mutagenicity is the one endpoint where a (Q)SAR model can substitute for a test under ICH M7 — and only for specifically validated, documented models, which ProTox is not. See go / no-go.
• It is retired here. ProTox-3 is in the platform's permanent retired roster and is not registered with the orchestrator, so chat cannot call it; existing results remain viewable. Its acute-toxicity class and organ-toxicity endpoints have no replacement on the platform. ADMET-AI covers the overlapping ones.

Jump to agreement with wet lab or against other in silico methods.

How in silico is used currently

Multi-endpoint toxicity prediction is used to raise a flag and order a queue, not to answer a safety question. In practice that means three things, and the published literature is lopsided towards the first.

• A filter stage inside a computational pipeline. This is overwhelmingly what ProTox is used for. Search PubMed for ProTox and you get hundreds of papers in which a docking or network-pharmacology screen nominated compounds, ProTox profiled them, and the paper reported no toxicity assay for the predicted endpoints. The tool is a gate between a virtual screen and a shortlist.
• Analog comparison during lead optimisation. Reading the direction of change across a series — does this substitution move hepatotoxicity up or down — asks much less of the model than reading one absolute call, and the per-endpoint figures below support it better.
• Mechanistic triage via targets and pathways. ProTox-3's distinctive output is not its binary calls but the 15 toxicity targets, 14 molecular initiating events and 12 Tox21 pathways, which point at why a compound was flagged. This is also the part of the output with the least published grading: no paper in the ProTox series reports a concordance figure for the 15-target pharmacophore panel.

What the literature does not contain is ProTox used as a decision instrument and then graded against a fresh measurement. That absence is the central fact on this page, and it is the same absence the ADMET-AI page reports for its own class of tool.

Against other in silico methods

One real head-to-head exists in the ProTox series, and it is from 2014. The ProTox paper compared its acute-oral-toxicity prediction against Discovery Studio's TOPKAT and the US EPA's T.E.S.T. on the same external validation set, using each tool's oral rat LD50 model and, for T.E.S.T., the nearest-neighbour method. The gap is large and it is concentrated in the toxic classes:

Both comparators scored zero sensitivity in class I. That is the finding, and it is the strongest comparative result in the series — but it is eleven years old, against two tools that have each moved on, on a version of the database that has since been updated. It describes 2014 ProTox against 2014 TOPKAT.

The ProTox-3 paper's superiority claim has no data behind it. Its discussion states that "all the 61 models reported in the ProTox 3.0 web server showed relatively better performance than commercial software like Discovery Studio's TOPKAT as well as freely accessible tool like Toxicity Estimation Software Tools (T.E.S.T.)". No table, no test set, no statistic and no comparative study is cited for that sentence. ProTox-II makes the same claim by reference to the 2014 paper. Treat it as a restatement of the 2014 result, not as evidence about ProTox-3's 61 models.

For a comparison on the same endpoint that actually was independently evaluated, look at CATMoS. This is the clearest illustration on this page of what ProTox lacks. CATMoS — the QSAR suite delivered through OPERA, which predicts rat acute oral toxicity against USEPA categories — was evaluated by a US EPA, NIEHS/NICEATM and Humane Society group against empirical in vivo LD50 values for the technical grade active ingredients of 177 conventional pesticides. Results, verified in the body text rather than the abstract:

• For the 165 chemicals with empirical LD50 ≥ 500 mg/kg, CATMoS also predicted > 500 mg/kg in 145/165 cases (88%).
• Of the 150 pesticides CATMoS placed in category III or IV, 145 (97%) were empirically in III or IV.
• But the overall multiclass balanced accuracy was 0.63, and by use type 0.55 (fungicides), 0.56 (herbicides) and 0.68 (insecticides/acaricides/nematicides).
• Of 81 pesticides empirically in category IV, only 23% were predicted as IV; 73% were over-predicted into category III.
• 115 of the 177 chemicals were already in the CATMoS training dataset.

Read that set together and the "88%" is a coarse binary agreement on a low-toxicity population, not categorical accuracy — which is what the 0.63 measures. This is the shape of a real independent evaluation, and it is also a warning about how a single headline percentage reads. None of these numbers describes ProTox. They are quoted here because the comparison ProTox has never had is the one a reader of a ProTox LD50 class most wants.

Where computation does replace the assay, the model is not this kind of model. ICH M7(R1) admits two complementary (Q)SAR models in place of the bacterial mutagenicity assay for drug impurities. On an external validation set of 388 proprietary pharmaceutical impurities harvested from New Drug Applications (72 experimentally active, 316 inactive), the FDA-evaluated composite bacterial mutagenicity models reached sensitivity 65.6% with negative predictivity 90.8% (one platform) and 82.3% with 95.4% (the other), at 96% coverage — the paper summarises the range as 66–82% sensitivity and 91–95% negative predictivity. Note what the controlling statistic is: negative predictivity, because the regulatory question is whether a negative call is safe to act on. ProTox-II's mutagenicity AUC-ROC of 0.91 on a public Ames benchmark is a different claim, reports no negative predictivity, and carries no equivalent standing.

On the platform, the comparison that matters now is with the tool that replaced it. ADMET-AI covers the overlapping endpoints — DILI, hERG, Ames, ClinTox, Tox21 and CYP inhibition — and its figures are reported with the dataset and the n that ProTox's are not: Ames AUROC 0.850 ± 0.004 on n = 7278, DILI 0.887 ± 0.011 on n = 475, hERG 0.840 ± 0.007 on n = 655, all on scaffold splits. Those are lower than ProTox's corresponding numbers, and that is not a result: different label sets, different splits, different metrics and one set with an n. ProTox-3's acute-toxicity class and organ-toxicity endpoints have no counterpart at all on the platform.

Landry C, Kim MT, Kruhlak NL, Cross KP, Saiakhov R, Chakravarti S, Stavitskaya L. Transitioning to composite bacterial mutagenicity models in ICH M7 (Q)SAR analyses. Regulatory Toxicology and Pharmacology. 2019;109:104488. doi: 10.1016/j.yrtph.2019.104488

In silico vs. wet lab

There is no single wet-lab standard here, because each endpoint has its own, and they are not comparable. This is the first way a toxicity predictor differs from a structural tool: a docking pose has one ground truth and one settled agreement threshold, while ProTox-3 spans endpoints whose reference methods differ by three orders of magnitude in time and animal use.

And ProTox-3's in vivo endpoints are graded against database labels, not in vivo studies. This deserves stating twice because the endpoint names invite the opposite reading. The acute-toxicity model is a similarity search over SuperToxic, an in-house database of rodent oral LD50 values gathered from public sources and literature; where a compound had several measured LD50 values the authors kept the lowest. Carcinogenicity labels come from the Carcinogenic Potency Database and CEBS; cytotoxicity from ChEMBL HepG2 IC50 values thresholded at 10 µM; clinical toxicity from ClinTox, defined as whether a drug-like molecule failed clinical trials due to toxicity. Each of those is an assay or an animal study at origin, reaching the model through a curation step. An agreement figure is therefore a property of the model and of the label set, and it says nothing about what happens in your animal or your assay.

Read the accuracy column cell by cell, never down it. A rodent LD50 in mg/kg, a GHS-class sensitivity, an AUC-ROC against a curated label and a balanced accuracy over four regulatory categories are four different kinds of number. What the table supports is time and cost at a stated accuracy — seconds against weeks or years, at an accuracy sufficient to order a list and insufficient to decide one compound.

And note the first row carefully. The wet-lab reference for acute oral toxicity is itself only 55–80% categorically reproducible. That is a ceiling on any concordance figure for this endpoint, and it is the main reason an in silico LD50 class should be read as a band rather than a value.

Bishop PL, Mansouri K, Eckel WP, Lowit MB, Allen D, Blankinship A, Lowit AB, Harwood DE, Johnson T, Kleinstreuer NC. Evaluation of in silico model predictions for mammalian acute oral toxicity and regulatory application in pesticide hazard and risk assessment. Regulatory Toxicology and Pharmacology. 2024;149:105614. doi: 10.1016/j.yrtph.2024.105614

Agreement with wet lab

Every figure in this section is author-reported by the Charité group that built ProTox, from the three web-server papers. None of it is independent. Figures are given with the version that produced them, because most of ProTox-3's endpoints were not re-measured for ProTox-3.

The selection rule for this page. Where a paper's abstract and its body text disagree, the body text wins. That matters twice here: the ProTox-3 abstract claims external validation for all 61 models while its tables cover only the 28 new ones, and the ProTox 2014 abstract quotes the figures for a configuration that was not the one shipped.

No endpoint below carries a sample size. ProTox-II states that dataset counts are in its Supplementary Table S1; ProTox-3 states they are on the server's "Model info" page. Neither puts them in primary text, so no n could be verified for any model. The one exception is noted inline.

Newly developed in ProTox-3 (2024)

Author-reported, 10-fold cross-validation and an external set held out from the model. 224 models were built across 28 endpoints — eight sampling methods each — and the best per endpoint was shipped, which is itself a selection step applied to the same validation data.

Cardiotoxicity is the only endpoint in the ProTox series with a dataset size in primary text: 5252 compounds with hERG IC50 or Ki values from ChEMBL, which is the training set, not the test set. The paper also prints 91.80% for respiratory-toxicity cross-validation accuracy in Table 1 and 91.90% in the surrounding prose; the table value is used here.

The remaining new models — 14 molecular initiating events (THRα, THRβ, TTR, RYR, GABAR, NMDAR, AMPAR, KAR, AChE, CAR, PXR, NADHOX, VGSC, NIS) and six CYP metabolism endpoints — also carry figures in the same two tables, with external AUC-ROC from 0.85 (CYP2D6) to 0.99 (NIS). They are a different kind of claim: target-level activity against curated binding data, not an organism-level outcome. The platform's wrapper exposed a 13-token model whitelist that did not include them individually, so they were reachable only through the comprehensive run.

Carried over from ProTox-II (2018), not re-measured for ProTox-3

These are the endpoints most users actually read, and this is where the version matters most. ProTox-II is not ProTox-3. The 2024 paper states that four of the eight toxicological-endpoint models — carcinogenicity, mutagenicity, cytotoxicity and immunotoxicity — "are already reported in our previous published studies", and that the 12 Tox21 pathway models are likewise carried over; the hepatotoxicity model is described as based on the earlier method. No figure for any of them appears in the ProTox-3 paper.

Author-reported balanced accuracy, AUC-ROC and Cohen's kappa:

The 12 Tox21 pathway models (nr-ahr, nr-ar, nr-ar-lbd, nr-aromatase, nr-er, nr-er-lbd, nr-ppar-gamma, sr-are, sr-hse, sr-mmp, sr-p53, sr-atad5) are reported in the same tables, with external AUC-ROC from 0.73 (nr-ar) to 0.92 (sr-mmp) and balanced accuracy from 83% to 91%. Those balanced accuracies sit on heavily imbalanced Tox21 assays, and the paper reports no precision-recall statistic, so they flatter positive calls in the same way described on the ADMET-AI page.

Immunotoxicity is the one to be careful with. A kappa of 0.29 on external validation is fair-to-poor agreement beyond chance, and it is the weakest number in the entire series. It appears in the same radar plot, the same colour scheme and the same confidence column as cardiotoxicity's 0.95.

Acute oral toxicity: the figures are from ProTox (2014)

The LD50 and GHS-class output is not a machine-learning classifier at all. It is a 2D similarity search over SuperToxic — approximately 38 000 unique compounds with known rodent oral LD50 values at the time of the 2014 paper — combined with a search for fragments over-represented in the most toxic classes. ProTox-II and ProTox-3 both describe the method as unchanged from that paper, run against an updated database. So the only published grading of this endpoint is from 2014, on a smaller database.

External validation set: approximately 5% of the ~38 000-compound prediction set, with a similar class distribution. The paper does not state the exact count. Sensitivity, specificity and precision are averages over all six GHS classes weighted by the number of molecules per class.

The headline figure for this endpoint is for the wrong configuration. The 2014 abstract quotes "sensitivity, specificity and precision of 76, 95 and 75%", which rounds the similarity-only row. The paper's own body states that "the consensus method combining similarity and toxic fragment searches was implemented in the ProTox web server" — and that configuration is slightly worse overall (73.08 / 94.62 / 73.50) while being better in class I, the most toxic band, which is why it was chosen. The 73% row is the one that describes what the server returns.

Three further things this endpoint does not give you. Coverage is not 100%: a Tanimoto cutoff means roughly 8–10% of compounds get no prediction. There is no error figure in mg/kg — the grading is class membership, never the quality of the LD50 number, so a predicted "~192 mg/kg" is a class-III call wearing a decimal point. And class sensitivity in the toxic bands tops out around 61–73%, so roughly one in three genuinely class-I-to-III compounds is missed.

Endpoints with no published agreement figure at all

• The 15 toxicity targets. The pharmacophore panel derived from the Novartis in vitro safety targets is described in all three papers and graded in none of them. No sensitivity, specificity, AUC or concordance against panel screening is reported anywhere in the series.
• Hepatotoxicity as updated for ProTox-3. The 2024 paper says the earlier models were "updated based on the new server software specification" but reports no post-update figure, so the 0.91 above describes the 2018 model.
• Carcinogenicity against a fresh two-year bioassay. The 0.87 is against CPDB and CEBS labels. No study grades a ProTox carcinogenicity call against a bioassay the authors did not already have the result for.
• LD50 as a quantity, as above.
• Every endpoint, on your chemistry. No applicability-domain study, no scaffold-split analysis and no prospective test of any ProTox version exists. There is no figure to quote.

Banerjee P, Kemmler E, Dunkel M, Preissner R. ProTox 3.0: a webserver for the prediction of toxicity of chemicals. Nucleic Acids Research. 2024;52(W1):W513-W520. doi: 10.1093/nar/gkae303

Banerjee P, Eckert AO, Schrey AK, Preissner R. ProTox-II: a webserver for the prediction of toxicity of chemicals. Nucleic Acids Research. 2018;46(W1):W257-W263. doi: 10.1093/nar/gky318

Drwal MN, Banerjee P, Dunkel M, Wettig MR, Preissner R. ProTox: a web server for the in silico prediction of rodent oral toxicity. Nucleic Acids Research. 2014;42(W1):W53-W58. doi: 10.1093/nar/gku401

What the figure doesn't cover

• No independent external validation exists, for any ProTox version. This is the single most important caveat and it is a stated absence, not a gap in sourcing. A PubMed sweep for ProTox with validation, comparison, benchmarking, concordance or external-test-set terms returns application papers — pipelines in which ProTox was one filter stage — and no third-party evaluation. Every figure on this page is author-reported by the group that built the tool, and the best-performing of eight sampling methods per endpoint was selected on the same validation data.
• The two nearest wet-lab comparisons are also by the ProTox authors, and neither reports a concordance statistic. One compared ProTox-II and SuperCYPsPred against ex vivo results for cosmetic chemicals and concludes qualitatively that "the computational predictions are in agreement with the experimental results" — no sensitivity, specificity or n is given in the abstract, and it is ProTox-II. The other applied ProTox-3 to a cohort of nearly 2000 historically marketed small-molecule drugs and found predicted hepatotoxicity to be a consistent statistical determinant of post-marketing withdrawal probability; the authors state that the analysis "does not establish causality". An association with an outcome is not an accuracy, and withdrawal status is not a toxicity measurement.

• Version attribution is the most likely error on this page, and on any page about this tool. Most ProTox figures in circulation describe ProTox-II, which has 33 models, not ProTox-3's 61. The platform wrapped the ProTox-3.0 endpoint, so ProTox-3 is the right version here — but 33 of its 61 models are ProTox-II-era models whose only published figures are ProTox-II's. A ProTox figure quoted without its version is unusable, and the abstract of the ProTox-3 paper is not a safe source for which models were validated.
• No sample size is published for any endpoint. Both web-server papers move dataset counts off-paper — to a supplementary table and to the server's "Model info" page respectively. A balanced accuracy of 70% on an unstated number of compounds is a floor.
• Confidence scores are not graded. ProTox returns a confidence or probability per endpoint. No published study calibrates it, relates it to applicability-domain distance, or measures whether high-confidence calls are more often right. Treat it as an ordering hint within one endpoint, not as a probability, and not as an applicability-domain estimate. Compare CATMoS, which ships an explicit two-level applicability-domain assessment and a 0–1 accuracy index derived from nearest neighbours — ProTox has no published equivalent.
• Balanced accuracy and AUC flatter imbalanced endpoints. No precision-recall statistic is reported for any ProTox model. The Tox21 pathway assays in particular have few positives, and the same gap between AUROC and AUPRC documented for those datasets elsewhere applies here with no number to quote.
• The wet-lab reference is noisy, and for acute oral toxicity it is measurably noisy. Categorical reproducibility of 55–80% on the reference data bounds any concordance figure for LD50 class. A model cannot be shown to agree with a standard more closely than the standard agrees with itself.
• The platform asserted nothing about this tool. The internal reference-data manifest for protox3 holds a single example, self-reports validation_status: "pending", and its one golden file uses keys the validator does not read, so nothing in the repository verifies a ProTox prediction. No figure on this page is a platform measurement, and none should be cited as one.
• No metabolites, and no dose. ProTox predicts parent-compound endpoints. It does not predict metabolite structures or metabolite toxicity, and apart from the LD50 class it reports no dose, exposure or duration — so none of its organ-toxicity calls is a risk assessment.
• Two reviews are useful for framing and for nothing else: one on AI-driven toxicogenomics, which notes how few such models have been prospectively validated, and one surveying AI in safety pharmacology and toxicology. Neither supplies a concordance figure for ProTox or for any other tool, and neither is a source for a number on this page.

Banerjee P, Ulker OC. Combinative ex vivo studies and in silico models ProTox-II for investigating the toxicity of chemicals used mainly in cosmetic products. Toxicology Mechanisms and Methods. 2022;32(7):542-548. doi: 10.1080/15376516.2022.2053623

Niu J, Wagner S, Preissner R, Banerjee P. Data-driven in silico toxicity profiling of marketed drugs and retrospective analysis of post-marketing withdrawal risk. Toxicology Mechanisms and Methods. 2026:1-13. doi: 10.1080/15376516.2026.2704958

Islam MM, Ali Sangi SM, Meravanige G, Nagaraja S, Chohan MS, Anwer MK, Gurung A. Artificial intelligence-driven toxicogenomics for predictive environmental and pharmaceutical safety. Frontiers in Genetics. 2026;17:1858338. doi: 10.3389/fgene.2026.1858338

Miller L, Treleaven I, El Amrani AI, Rossman EI, Winters BR, Pugsley MK. The emerging use of artificial intelligence in safety pharmacology and toxicology. Journal of Pharmacological and Toxicological Methods. 2026;139:108424. doi: 10.1016/j.vascn.2026.108424

Go / no-go

Go — act on the result when:

• You are ordering a list and want obvious liabilities pushed down it. Relative rank within one endpoint over many compounds is what these figures support.
• You are reading the direction of change across an analog series, not the absolute call on one compound.
• The endpoint is one of the better-measured ones and you treat it as a flag: cardiotoxicity (external AUC-ROC 0.95), respiratory toxicity (0.93), mutagenicity (0.91, ProTox-II), hepatotoxicity (0.91, ProTox-II), neurotoxicity (0.91).
• You are deciding when to schedule an assay rather than whether to believe one. A high mutagenicity call is a reason to pull the Ames test earlier.
• You use the toxicity-target and pathway output as a mechanistic hypothesis to test, which is what it is — and you do not attach a confidence to it, because none is published.
• The compound has close relatives in the public toxicity databases, and you have read the three most-similar compounds the server returns rather than only the predicted class.

No-go — do not act on the result when:

• You need an LD50 number. The only published grading of this endpoint is GHS-class sensitivity, around 73% overall and 61–73% in the toxic classes, at 92% coverage. There is no published error in mg/kg.
• You are averaging across endpoints, or reading a radar plot as one profile. Immunotoxicity (κ = 0.29) and cardiotoxicity (AUC 0.95) are drawn in the same colours.
• You are acting on immunotoxicity at all. Fair-to-poor agreement beyond chance does not support a decision in either direction.
• You are dropping a compound on a single toxicity call. No precision-recall statistic is published for any endpoint, so the positive-predictive cost of acting on a flag is unmeasured.
• Your chemistry has no close relative in the public toxicity databases. No applicability-domain, scaffold-split or prospective study of any ProTox version exists, so there is no figure describing that case — and the confidence score is not one.
• The output is headed for a regulatory submission. ICH M7 admits specifically validated, documented (Q)SAR models for one endpoint; ProTox is not one of them, reports no negative predictivity for mutagenicity, and has no independent external validation for any endpoint. Do not read "used by regulatory agencies" claims as regulatory standing.
• You intend to skip an assay. Nothing on this page licenses that for any endpoint, and ProTox-3 is retired here in any case.
• You are reading an older ProTox figure. Check whether it is ProTox-II or ProTox-3 before using it, and check which release each endpoint's figure actually came from.

Caveats on reading this page. Every figure is quoted from the cited source with its original metric and version, and figures from different studies are not comparable because label sets, metrics and releases differ. No figure carries a sample size, because none of the primary sources publishes one. All figures are author-reported; no independent validation of ProTox exists. Where a comparison does not exist in the literature, this page says so rather than substituting a figure from a different tool, a different endpoint or a different version.