---
title: Reading an in silico result against the bench
description: >-
  What a predicted structure, pose or property score has been shown to agree
  with experimentally, how that agreement is measured, and where the figures
  stop.
tags:
  - concepts
  - validation
  - translatability
---

A prediction is not a measurement. Every tool here returns a number that was graded against something, and *what it was graded against* decides what you can do with it. A pose scored against a deposited crystal structure tells you the geometry is plausible; it does not tell you the compound binds. A toxicity score with an AUC of 0.88 against a curated label set tells you the model separates known actives from known inactives; it does not tell you what happens in your assay.

Tools that make predictions carry a **translatability** child page — `/docs/<tool>/translatability` — that states this for that tool, with citations. This page explains how to read those pages, and indexes them.

## There is no in silico to in vivo comparison, and that is not a gap in our sourcing

The literature does not contain studies that compare an in silico prediction directly against an in vivo outcome. Papers that pair the two are almost always *sequential*: a computational screen nominated a compound, and animal work followed. That is a workflow, not a concordance measurement — nothing in the design establishes how often the prediction was right.

Three comparisons do exist in quantity, and they are the ones the tool pages use:

1. **Against other in silico methods** — head-to-head on a shared benchmark, same task, same inputs.
2. **Against the standard wet-lab approach** — for structure and pose prediction, experimental structure determination, where agreement has a settled definition (heavy-atom RMSD, conventionally ≤ 2 Å); for property and toxicity prediction, the assay panel, reported as AUC or accuracy against assay-derived labels.
3. **Against the combined tooling here** — what the platform's own chain achieves, which is the only column we can measure ourselves.

Each is compared on **time, cost, and accuracy**. Anything past that — whether a confirmed pose becomes a compound that works in an animal, let alone a patient — is the part of drug discovery that no benchmark measures.

## What the wet-lab cost figures mean

**No commercial CRO publishes a list price for any of these techniques.** Across roughly 110 vendors checked in October 2026 — including SARomics, Viva Biotech, Creative Biostructure, Proteros, Charles River, Evotec and WuXi — the count of public commercial prices for co-crystallography, ITC, MST, hERG patch clamp, hepatocyte stability and cryo-EM is zero. Everything is quote-only. Eurofins Discovery and Pharmaron both run per-item catalogs with fixed specifications and turnaround times where the price itself renders as "Login to View Pricing," so a list price demonstrably exists and is simply not published.

So the wet-lab cost cells on these pages quote **academic core-facility rate cards at their industry or commercial tier**, with the facility named and the rate card dated. Read them as **recharge rates, not market prices** — a core facility recovers its costs, while a CRO prices a service, and the gap between the two is unmeasured. Treat every wet-lab cost figure on these pages as a floor.

The figures used, verified against the published rate cards:

| Technique | Commercial-tier rate | In 1-credit runs, at the plan rate | Facility, date |
|---|---|---|---|
| Crystallization screening | €200 / 96-well plate | 56,000 | Biocenter Finland, June 2025 |
| Full data collection and crystal characterization | €2,000 | 560,000 | Biocenter Finland, June 2025 |
| Synchrotron data collection | €240 / crystal | 67,200 | Biocenter Finland, June 2025 |
| Phasing, refinement, analysis | €1,000 / day each | 280,000 / day | Biocenter Finland, June 2025 |
| Crystal-to-structure, including PDB deposition | from €500 | 140,000 | Univ. of Cologne c2f (⚠︎ 2021, likely stale) |
| Crystallization plate | 150 CHF excl. VAT | — | Univ. of Zürich PCC (⚠︎ undated) |
| SPR, instrument time | $357.50 / day | 89,300 / day | CU Anschutz Biophysics, June 2026 |
| SPR, per ligand–analyte pair | $600 | 150,000 | Yurogen/ABclonal (⚠︎ the only commercial price found, and it is a lapsed promotion) |
| hERG, non-GLP electrophysiology | €250 for 1 compound at 1 concentration, 3 replicates; €500 at 4 concentrations | 70,000; 140,000 | Univ. of Siena hERG Channel Facility |
| Ames, OECD TG 471, GLP | $8,300 cytotoxicity screen plus main test, 5 strains; $4,100 confirmatory, 2 strains | 2.07 million; 1.02 million | MB Research Laboratories |
| Ames, non-GLP screening | €1,200–2,400 per sample | 336,000–672,000 | National Institute of Biology, Ljubljana (⚠︎ undated) |
| Microsomal stability, human | €346.86 per compound per species | 97,100 | IQM-CSIC Madrid, 2026 rates |
| ITC · MST · cryo-EM · hepatocyte stability | **no public price of any kind** | — | — |

The runs column is what the same money buys here: the number of 1-credit runs (ADMET-AI, ProTox-3, AiZynthFinder, P2Rank) at $0.004 per credit, the subscription-plan rate, with euros converted at €1 = $1.1205 (ECB reference rate, 22 October 2026). The 1,000- and 2,000-credit packs cost $0.005 per credit, so the same money buys about a fifth fewer runs there. A tool that costs N credits per run gets N times fewer runs, and each tool page states its own figure. Runs are rounded down.

Platform costs are stated in credits, derived from `public.job_pricing`, and are the one column measured rather than estimated:

| Tool | Credits / run | Padded runtime |
|---|---|---|
| [DiffDock](/docs/diffdock/translatability) | 149 | 14.1 min |
| [Chai-1](/docs/chai/translatability) | 42 | 4.0 min |
| [Boltz-2](/docs/boltz2/translatability) | 13 | 1.2 min |
| [ESMFold](/docs/esmfold/translatability) | 9 | 50 s |
| [REINVENT](/docs/reinvent/translatability) | 9 | 2.0 min |
| [GenMol](/docs/genmol/translatability) | 7 | 38 s |
| [MM-GBSA](/docs/mmgbsa/translatability) | 5 | 2.0 min |
| [AiZynthFinder](/docs/aizynthfinder/translatability) | 1 | 139 s |
| [ADMET-AI](/docs/admet-ai/translatability) | 1 | 27 s |
| [ProTox-3](/docs/protox3/translatability) | 1 | 80 s |

⊘ Job pricing changes, so re-derive these from `public.job_pricing` rather than citing this table.

## Accuracy is never comparable across those rows

Read an accuracy figure cell by cell, never down a column. Crystallography's accuracy is definitional, because it is the thing the others are graded against. A docking percentage is the share of poses inside a geometric threshold. An affinity method's accuracy is an energy error in kcal/mol. An ADMET model's is an AUC against a label set. Nothing meaningful comes from comparing those to each other.

What the comparison does support is **time and cost at a stated accuracy**. That is the claim these pages are built to make.

## Property predictors are graded one step removed from the bench

When a docking method is graded against a co-crystal, the crystal structure is the measurement. When a toxicity model reports an AUC, the comparison is against a **label derived from an assay someone else ran** — drawn from Tox21, ToxCast, ChEMBL, or an outcome set. The assay is still ground truth, but it reaches the model through a curation step: inclusion criteria, activity thresholds, and whatever chemistry happened to be screened.

A model's AUC is therefore a property of the model *and* of the label set. Figures in the 0.85–0.93 range on benchmark toxicity sets are routine, and they do not predict performance on a scaffold series absent from those sets. The same holds for off-target reporting: named off-targets can only be compared against panel screening that exists.

**Ask what the label actually encodes.** The sharpest failure in this class is a model that scores well because its label is a proxy for something easy. An antibody humanness model separating human from murine sequences looks strong at AUC 0.97 — but stratify the same set by species of origin and the correlation with clinical immunogenicity vanishes in every subgroup, because the score was distinguishing provenance, not predicting an outcome. One group put it directly: it "indirectly measures the ability of the model to distinguish between species." The reader's molecules all sit inside one of those subgroups, so the headline figure describes a comparison they will never make. Whenever a label is a category someone assigned rather than a quantity someone measured, check whether the model's apparent skill survives holding that category fixed — and expect the independent number to be far below the author-reported one. On [BioPhi](/docs/biophi/translatability) that gap is R² = 0.26 against R² = 0.02.

## Training overlap is the first thing to check

Benchmark sets for structure and docking methods are drawn from the PDB, and the models were trained on the PDB. When a test complex resembles a training complex, the reported number is partly a lookup. The effect is measured: deep-learning docking methods degrade as sequence identity to their training data falls.

So when a page gives two numbers that look contradictory — strong on one benchmark, weak on another — the usual explanation is that one set overlaps training data and the other was built to avoid it. Both are real. The second describes a novel target.

**Check the version, too.** Published criticism often tests an earlier release than the one running here, and a figure quoted without its version is unusable.

## What every translatability page contains

A short **verdict block** opens the page, for readers holding a result who want to know whether it is useful. Then six sections, in the same order on every page so tools can be compared side by side:

| Section | Answers |
|---|---|
| How in silico is used currently | What the field actually does with this class of output |
| Against other in silico methods | Head-to-head results, with each method's inputs stated |
| In silico vs. wet lab | Which experimental technique is the standard, and the time/cost/accuracy matrix |
| Agreement with wet lab | Published concordance figures, each with its metric and sample |
| What the figure doesn't cover | Training overlap, version, applicability domain, metric blind spots |
| Go / no-go | When to act on the output, and what to run next |

**Sourcing rules.** Figures are quoted with their source and metric. Where a tool has no published comparison for a section, the page says so rather than borrowing a figure from a different tool or version. A result from a multi-tool pipeline is attributed to the pipeline. Reviews and protocols provide context, never a concordance figure — the citation must be the primary comparative study.

## Pages by domain

Pages exist where a tool makes a prediction that can be graded against an experiment. Lookup and retrieval tools — [ChEMBL](/docs/chembl), [PubChem](/docs/pubchem), [BLAST](/docs/blast), [UniProt](/docs/uniprot) and the other [database tools](/docs/databases) — return records rather than predictions, so they have nothing to translate and carry no page.

| Domain | Tools | Status |
|---|---|---|
| Docking & MD | [DiffDock](/docs/diffdock/translatability) | Published |
| Docking & MD | [MM-GBSA](/docs/mmgbsa/translatability) | Published |
| Docking & MD | [P2Rank](/docs/p2rank/translatability) | Published |
| Structure prediction | [Boltz-2](/docs/boltz2/translatability), [Chai-1](/docs/chai/translatability), [ESMFold](/docs/esmfold/translatability) | Published |
| Structure prediction | [AlphaFold](/docs/alphafold/translatability) | Published |
| Predictors | [ESM-3](/docs/esm3/translatability) | Published — ⊘ **the page where the tool is not the model.** No ESM-3 weight loads anywhere; every surface runs Meta **ESM-2**, and the default `esm3_sm` is a **35M** checkpoint sold as 1.4B (`esm3_md` ≡ `esm3_lg`). On an independent 201-assay ProteinGym table the default scores **ρ = 0.333** against real ESM-3's 0.468, and the variant-effect recipe our docs prescribe — embedding cosine distance — has **zero** published concordance, because every DMS figure grades masked log-likelihood ratios. `esm3_inverse_fold` fails 100% of the time at 42 credits. esmGFP has never been independently re-expressed. Twenty-four defects stated with their status |
| ADMET | [ADMET-AI](/docs/admet-ai/translatability) | Published |
| ADMET | [DeepPurpose](/docs/deeppurpose/translatability) | Published — ⊘ the strongest negative result in this set. A **degree-only baseline with no chemistry in it at all** — no SMILES, no sequence — matches or beats DeepPurpose on the two easier evaluation regimes, and shuffling the training SMILES and sequences left test performance essentially unchanged. Under a cold-pair split, AUC-PR falls **67.0 → 17.2**. The default checkpoint is Davis-trained, and Davis is censored at pKd 5.0 with ~70% of labels on that floor, so the platform's `INACTIVE` class is near-unreachable. Twelve defects stated with their fixes |
| ADMET | [DeepSol](/docs/deepsol), [GraphSol](/docs/graphsol) | Planned |
| Toxicity | [ProTox-3](/docs/protox3/translatability) | Published — ⚠︎ the tool is **retired**; use [ADMET-AI](/docs/admet-ai) for the overlapping endpoints. The page is kept for anyone holding an old result |
| Predictors | [BioBERT](/docs/biobert), [Bayesian](/docs/bayesian) | Planned |
| Generative & de novo | [REINVENT](/docs/reinvent/translatability), [GenMol](/docs/genmol/translatability) | Published |
| Generative & de novo | [AiZynthFinder](/docs/aizynthfinder/translatability) | Published — the makeability comparator the generative pages point to. ⚠︎ Solve rate is a property of the configuration, not the molecule: the same tool runs **9.5% to 99.1%** depending on stock set, template library and search budget, and this platform is fixed at 100 iterations |
| Antibody engineering | [BioPhi](/docs/biophi/translatability) | Published — the most translatable tool here, and the one whose headline figure collapses once you check who measured it: humanness explains **R² = 0.26** of clinical anti-drug-antibody incidence author-reported (n = 217), but **R² = 0.02 and not significant** in the only independent measurement (n = 88 marketed products). Two groups find the correlation vanishes once the set is stratified by species, and **both published wet-lab tests of BioPhi output lost affinity** (~16-fold and ~2-fold) |
| Antibody engineering | [developability](/docs/antibody-developability/translatability) | Published — ⚠︎ the only page here where the platform's own descriptors are **hand-written threshold rules**, and where the closest published analogues are not merely ungraded but **refuted**: the charge/pI clearance heuristic fails three independent re-tests, and the viscosity rule scores *below* its own majority-class baseline. `tap_score` is not TAP, and four spurious `critical` cysteine flags make trastuzumab read `problematic`. Five defects stated with their fixes |
| Antibody engineering | [Paragraph](/docs/paragraph/translatability) | Published — the page where the **wet-lab comparator splits in two and the tool addresses only one half**. The 4.5 Å contact paratope is well measured (AUC-PR **0.725** author-reported, **0.689** on an independent retrain); the *energetic* paratope has **zero** published concordance for any paratope predictor — "alanine" and "mutagenesis" appear 0 times across six full texts. ⊘ Read AUC-PR, never ROC AUC: a positional-frequency lookup table beats Paragraph on ROC AUC (0.952 vs 0.934). ⊘ The platform folds every sequence against a **hardcoded trastuzumab partner chain** and cannot accept a cognate H/L pair, so no published figure describes this configuration. Twelve defects stated with their fixes |
| Antibody engineering | [Humatch](/docs/humatch/translatability) | Published — the class's only tool with **no published ADA coefficient at all**: its 211-therapeutic analysis is a three-bin separation, and its own paper claims only *parity* with the R² = 0.16–0.34 methods, so it cannot be placed against BioPhi's ceiling. Plain germline models beat its pairing model on an external benchmark, and its one measured-property claim is independently **r = 0.25** on thermostability and **−0.15, not significant** on expression. ★ One Humatch-humanized antibody *has* been expressed by an independent group: **44-fold affinity lost, thermostability gained.** ⊘ The paired CNN is not loaded here, yet 8 surfaces call the output "joint VH/VL". Twelve defects stated with their fixes |
| Antibody engineering | ANARCI, IgBLAST | ⊘ **Written but held, not linked.** A standing ruling excludes both from public tool pages because they return wrong answers with status `SUCCESS`. Pages exist and document the defect; publishing them is an owner call |
| Genomics & multi-omics | [scvi-tools](/docs/scvi-tools), [Scanpy](/docs/scanpy), [causal genomics](/docs/causal-genomics) | Planned |
| Targets & modalities | [CRISPR](/docs/crispr/translatability) | Published — the one tool here with **large-scale matched wet-lab ground truth**, so the page carries real concordance. ⚠︎ Scope: a genuinely state-of-the-art on-target predictor (**Rule Set 3**, not the Rule Set 2 the docs claim) and **no off-target predictor at all** — `crispr_offtarget` scans only the sequence you pasted, so "0 off-targets" means "no near-match in your input". ⊘ Our docs publish a 0–1 score band over Rule Set 3's *unbounded z-score*, which would reject 57.5% of guides as poor |
| Targets & modalities | [PROTAC](/docs/protac) | Planned |
| Materials & crystals | [XtalNet](/docs/xtalnet), [opXRD](/docs/opxrd), [PyWPEM](/docs/pywpem) | Planned |

## Adding a page for a tool

1. Create `content/docs/<tool>/translatability.mdx` with `title`, `description`, and `tags` including `translatability`.
2. Add `"translatability"` to that tool's `meta.json` `pages` array, after `results` or `outputs`.
3. Write the verdict block, then the six sections above in order. Keep a section and state that no published figure exists rather than dropping it.
4. Find the evidence by searching PubMed directly rather than relying on web search, which under-reports this literature. Resolve every PMID to a DOI before citing it.
5. Cite as NLM-style blockquotes with `doi.org` links, matching the other docs pages.
6. Move the tool's row in the table above to **Published**.