---
title: How REINVENT results compare to the bench
description: 'What a REINVENT-generated molecule has been shown to agree with experimentally, and why the comparison is makeability rather than accuracy.'
tags:
  - reinvent
  - validation
  - translatability
---

**You have a REINVENT result. Here is what it supports.**

- **A generated SMILES is a design proposal, not a finding.** Nothing about it has been measured. The question a translatability page normally answers — how often is the prediction right — has no settled definition here, because a generated molecule is not a prediction about an existing object.
- **The score on your result is probably not a quality score.** The platform's default run type is `sampling`, which samples the prior with no scoring function at all. The number the results table carries in that mode is the **negative log-likelihood** of the SMILES under the prior — how typical the molecule is of the training set, not how active, drug-like, or makeable it is. Lower NLL means more typical. Only `scoring`, `sampling_scoring`, and `staged_learning` runs apply a scoring function, and then the score is whatever you configured in [scoring functions](/docs/reinvent/scoring).
- **Do not assume the molecules can be made.** On hard multi-parameter objectives, retrosynthesis software could find no route for roughly 70% of the top-100 molecules pooled across generative methods. That figure is not REINVENT's specifically — see [agreement with wet lab](#agreement-with-wet-lab) — but it is the class's documented failure mode, and REINVENT has no published exemption from it.
- **Confirm makeability with** a retrosynthesis plan ([AiZynthFinder](/docs/aizynthfinder) on this platform) and then with a chemist. Confirm activity by synthesizing the compound and assaying it. There is no shortcut, and no step on this platform produces a measured affinity.
- **Two AstraZeneca programs have published REINVENT designs that reached potent, selective compounds** with rat pharmacokinetics. In both, REINVENT was one stage inside a Design-Make-Test-Analyze cycle that also used structure-based design, physics-based affinity prediction, machine-learned DMPK models, and medicinal chemists choosing what to make. Those compounds belong to the campaign.

Jump to [agreement with wet lab](#agreement-with-wet-lab) or [against other in silico methods](#against-other-in-silico-methods). The general framing for every tool is in [reading an in silico result against the bench](/docs/translatability).

## How in silico is used currently

Generative design is used to **enumerate design ideas inside a human-run program**, not to decide what to make. In practice that is three things:

- **Idea generation against a scoring objective.** You have a target profile — a QSAR model, a similarity constraint, physicochemical bounds — and you want structures that satisfy it so a chemist has more to choose from than they would have drawn. This is what REINVENT's reinforcement-learning loop is for, and it is where the published programs used it.
- **Local enumeration around something you already have.** Scaffold decoration, R-group replacement and linker design (the `libinvent`, `linkinvent` and `mol2mol` priors) explore around a known series rather than inventing a core. This is the mode with the least to go wrong, because the chemistry around the variable position is usually familiar.
- **Library design.** Producing a focused set to order or register, where the per-molecule bar is low and the set is what matters.

What generative design is not used for, by anyone reporting their validation honestly, is deciding that a molecule is active or synthesizable. Every published program that reached a real compound put a chemist between the generator and the flask.

## Against other in silico methods

This is the only section where REINVENT has strong, independent, head-to-head numbers — and all of them are **in silico benchmarks graded by computational oracles**, which say nothing about bench translation.

On the **Practical Molecular Optimization (PMO)** benchmark — 23 objectives, a hard budget of 10,000 oracle calls, scored as the area under the curve of the mean top-10 property score — an independent evaluation of **25 methods** found REINVENT the best of all of them. Its summed AUC top-10 was **14.196**, against Graph GA 13.751, SELFIES-REINVENT 13.471, GP BO 13.156 and STONED 13.024. The authors' own words: "REINVENT is the top-1 method", and "several older methods, like REINVENT and Graph GA, outperform more recent ones". They also found REINVENT far more sample-efficient than the SMILES LSTM hill-climbing baseline it is usually compared to.

That ranking has since been overtaken on the same benchmark. [GenMol](/docs/genmol/translatability), which this platform prefers for de novo work, reports a summed AUC top-10 of **18.362** against the same REINVENT figure of 14.196, best on 19 of the 23 tasks; f-RAG reports 16.928 and Genetic GFN 16.213.

⊘ **The REINVENT in both of those tables is not REINVENT 4.** PMO's implementation is taken from Olivecrona et al. 2017 — the original RNN-plus-reinforcement-learning REINVENT — and the GenMol paper takes its REINVENT baseline from PMO. The platform runs REINVENT 4. **No published benchmark compares REINVENT 4 against other generators on PMO or anything equivalent.** A figure quoted for "REINVENT" without its version is unusable here, and this is the most common error in reading these numbers.

On a benchmark built around reproducing known actives for five kinases and three GPCRs rather than distribution-learning statistics, REINVENT grouped with CharRNN and VAE as the models best able to reproduce known active molecules, while RNNAttn, TransVAE and GraphAF struggled despite scoring well on the usual distribution-learning metrics — an inversion of the conventional ranking.

> Gao W, Fu T, Sun J, Coley CW. Sample Efficiency Matters: A Benchmark for Practical Molecular Optimization. *Advances in Neural Information Processing Systems*. 2022. doi: [10.48550/arXiv.2206.12411](https://doi.org/10.48550/arXiv.2206.12411)
>
> Lee S, Kreis K, Veccham SP, Liu M, Reidenbach D, Peng Y, Paliwal S, Nie W, Vahdat A. GenMol: A Drug Discovery Generalist with Discrete Diffusion. 2025. doi: [10.48550/arXiv.2501.06158](https://doi.org/10.48550/arXiv.2501.06158)
>
> Weng G, Zhao H, Nie D, Zhang H, Liu L, Hou T, Kang Y. RediscMol: Benchmarking Molecular Generation Models in Biological Properties. *Journal of Medicinal Chemistry*. 2024;67(2):1533-1543. doi: [10.1021/acs.jmedchem.3c02051](https://doi.org/10.1021/acs.jmedchem.3c02051)

## In silico vs. wet lab

The standard wet-lab approach for the question REINVENT answers is a **turn of the Design-Make-Test-Analyze cycle**: a medicinal chemist designs a compound, a chemist synthesizes it, and an assay measures it. That is the comparator, and it is not a measurement technique — it is a process. This matters for how the table below reads.

**There is no RMSD or AUC here.** A generated molecule is not a prediction about a specific experiment, so there is nothing to score it against in the way a docked pose is scored against a co-crystal. What the literature actually measures, and what the accuracy column therefore reports, is one of three different things:

1. **Makeability** — does retrosynthesis software find a route, and would a chemist call it feasible?
2. **Recovery** — does the model regenerate the compounds a real project actually made and advanced?
3. **Prospective outcome** — of the molecules designed, how many were made, and how many were active?

Read each cell against its own question. None of the three is comparable to the others, and none of them is a pose accuracy or a classification AUC.

| | Time to answer | Cost basis | Accuracy, and its metric |
|---|---|---|---|
| **Standard wet lab** — chemist design, synthesis, assay (one DMTA turn) | Weeks to months per compound; synthesis may fail outright | No published basis — one design–make–test turn is custom synthesis plus assay, priced per compound. Assay components: [cost basis](/docs/translatability#what-the-wet-lab-cost-figures-mean) | Ground truth. A compound that was made either binds or does not, and the assay says which |
| **Other in silico** — other generative models with a CASP or chemist readout | Minutes to hours | No per-run price is published for this class. [cost basis](/docs/translatability#what-the-wet-lab-cost-figures-mean) | Makeability: 30.2% of top-100 molecules had an ASKCOS route on hard objectives, pooled across five generative methods, rising to 80.2% when generation was biased by SA_Score. Chemist panel: 0 of the top 10 unconstrained molecules on a multi-parameter task were called feasible by any of 7 chemists |
| **Combined tooling here** — REINVENT 4, then [ADMET-AI](/docs/admet-ai) and [AiZynthFinder](/docs/aizynthfinder) | About 2 min for a REINVENT run, plus ~30 s for ADMET screening and ~2.5 min per retrosynthesis | 11 credits | **No figure, for any of the three questions.** No published study reports makeability, recovery, or prospective outcome for REINVENT 4 — see below |

On that reading, the honest position for this chain is: **minutes instead of months to produce design ideas, with no measured accuracy of any kind attached to them.** The chain's value is throughput of hypotheses, and the bar it has to clear is a chemist's judgement, not a benchmark.

Two asymmetries matter. Synthesis can fail outright for a given structure and no amount of effort fixes it, which is the largest single reason a generated molecule never becomes data. And the wet lab reports what a specific compound *does*, while REINVENT reports what the prior finds *likely*; those are not interchangeable at any score.

## Agreement with wet lab

**This is the weakest evidence base of any tool on the platform, and the section is short because the literature is thin.** Below are the three measurable questions and what has actually been published, with each figure's metric and sample.

### Recovery of real medicinal-chemistry output

The closest thing to a concordance measurement for REINVENT trains the model on a real project's **early-stage** compounds and asks how many of the project's **middle and late-stage** compounds — the ones chemists actually made, registered and advanced — the model regenerates.

Using **REINVENT 2.0**, across five public targets (DRD2, GSK3, CDK2, EGFR, ADRB2 from ExCAPE-DB), rediscovery of middle/late-stage compounds was **1.60%** of the top 100, **0.64%** of the top 500, and **0.21%** of all 5,000 generated molecules. Across six real proprietary projects at Teijin Pharma, each with more than 1,000 compounds and real recorded synthesis dates, the same figures were **0.00%, 0.03% and 0.04%**. On real projects the top 100 scored molecules contained **no** project compounds at all. Only two of the six proprietary projects recovered anything; of the public targets, GSK3 recovered nothing.

The same study shows **why** the benchmark number is the optimistic one. Reinforcement learning drove the scoring model to 0.7–1.0, but that scoring model assigned only **0.1–0.8** to the real high-activity compounds the project actually advanced. The thing being optimized and the thing the project produced were not the same distribution. Clustering the chemical space, public targets had mostly *predictive* clusters while all six proprietary projects had more *unpredictive* clusters than predictive ones.

For scale, an earlier benchmark that split its test set randomly reported around **10%** rediscovery; a study that removed its DRD2 test set from training entirely reported **under 1%**, which is the regime real projects are in.

Secondary figures from the same work: validity above 98% on every target; novelty above 70%; uniqueness between 39.4% (GSK3) and 82.2% (DRD2) on public data but only 15.1% (project F) to 50.9% (project E) on proprietary data, recoverable to 99.0–99.8% with a diversity filter.

### Makeability

No study has measured whether REINVENT 4's output can be synthesized. The two relevant measurements in the literature evaluated other generators, and this page reports them as class context rather than as REINVENT figures:

- Against **ASKCOS** retrosynthesis (route depth ≤ 9, starting materials buyable at ≤ $100/g from a 2018 eMolecules/Sigma-Aldrich list, 60 s per molecule, first route found wins), the fraction of top-100 molecules with a route was **30.2%** on hard GuacaMol objectives for ChEMBL-trained models and **32.7%** for MOSES-trained models, pooled across SMILES LSTM, SMILES GA, Graph GA and two baselines. On trivial objectives the same figures were 60.1% and 63.5%. Biasing generation by SA_Score raised the hard-objective figure to **80.2%**, and by SCScore to **55.4%** — at a cost of more than 0.2 on the normalized objective for some tasks. For reference, the same pipeline judged **68.3%** of ChEMBL, **89.8%** of MOSES, **60.8%** of ZINC and **3.5%** of GDB17 synthesizable. **REINVENT was not among the evaluated models.**
- Against a **blinded chemist panel**, seven chemists labelled 30 generated molecules feasible or not feasible; they agreed with each other except for two chemists on two molecules, and 12 of the 30 were called feasible. On a Pi3K/mTOR multi-parameter task, the top 10 molecules from **unconstrained** generation scored **0** on the chemist panel — not one was called feasible by any of the seven, unanimously. Adding any synthetic-accessibility constraint moved that to 0.58–0.63.

The second finding is the one to carry: **SA_Score separates molecules within a congeneric series and fails across series.** Its AUC against the chemist labels was 1, 0.33 and 0.96 within each of three separate generations, but **0.63** over all 30 molecules together.

**And the referee is itself only fair-to-moderately reproducible.** The largest published study of chemist judgement collected over 5,000 pairwise preferences from **35 medicinal chemists** at one company. Agreement was measured in two pilot rounds of 220 pairs, with 9 and then 14 chemists: **inter-rater Fleiss κ was 0.40, then 0.32**. Within a chemist, on pairs shown twice, **Cohen κ was 0.60 and 0.59**, but the per-chemist spread ran from **0.16 to 1.00**, and in 4 of 23 chemist-rounds a chemist's agreement with their own earlier answer was not significant even at p < 0.1. The task was deliberately simplified to a binary "which of these two do you prefer" with an intentionally vague prompt, chosen *because* earlier work found agreement so low on anything harder — so those κ values are the optimistic end. Chemist preference also failed to correlate with any computed descriptor above Pearson r = 0.4, QED being the highest.

Note what the chemists were asked. The prompt covered "drug-likeness, synthetic accessibility, or other criteria" under a non-defined preference, so κ = 0.32–0.40 measures agreement on *which molecule to follow up*, not on whether one can be made.

### Prospective synthesis and assay

**No published study reports a REINVENT campaign with denominators** — designed, made, assayed, active. Two AstraZeneca programs report REINVENT-assisted discovery reaching real compounds, and neither gives those counts in text this page could verify:

- **Cbl-b.** A carbamate Cbl-b inhibitor series was discovered with REINVENT combined with medicinal-chemistry structure-based design, in a DMTA cycle whose Design phase ran iterative in silico rounds "strongly guided by physics-based affinity prediction and machine learning DMPK predictive models, prior to selection for synthesis". The outcome is stated as "a potent series of carbamate Cbl-b inhibitors". No number of molecules designed, made, or active is given.
- **HPK1.** REINVENT was deployed at two stages. Transfer learning on kinase-active compounds followed by reinforcement learning against QSAR-based scoring yielded **three active chemotypes**; REINVENT scaffold hopping scored by 3D pharmacophore and docking models yielded **two additional active chemotypes**; optimizing one of them delivered a compound with potent cellular activity, kinase selectivity and favourable rat pharmacokinetics.

**One prospective result in this class is fully reported, and it is not REINVENT.** A monoglyceride lipase (MGLL) campaign using **DrugEx v3** — a different generator, reported here only so the class has one honest denominator — ran this funnel: 100,000 molecules sampled, 32,907 passing both an activity and a synthesizability classifier at probability 0.8, 20,055 with an in-house retrosynthesis route, 4,675 with a route of five steps or fewer, 950 passing Lipinski, 609 after a novelty cut, then **three** chosen by chemists for their diversity, "chemical eye" and one-to-two-step routes. All three were made and assayed. One had an IC50 of **1 µM**; the other two were around **100 µM**. At the ≤ 10 µM bar conventionally used for hit finding, **one of three** was active — and the campaign's own authors attribute scaffold choice to the QSAR model rather than the generator, noting it rediscovered 145 known actives and that "key scaffolds for the target are re-used, and the sidechains are algorithmically altered".

Read that as the shape of the class's best-documented prospective result: six orders of magnitude of in silico attrition, a human choosing the final three, and one micromolar hit. It is not REINVENT's hit rate and must not be quoted as one.

> Blaschke T, Arús-Pous J, Chen H, Margreitter C, Tyrchan C, Engkvist O, Papadopoulos K, Patronov A. REINVENT 2.0: An AI Tool for De Novo Drug Design. *Journal of Chemical Information and Modeling*. 2020;60(12):5918-5922. doi: [10.1021/acs.jcim.0c00915](https://doi.org/10.1021/acs.jcim.0c00915)
>
> Handa K, Thomas MC, Kageyama M, Iijima T, Bender A. On the difficulty of validating molecular generative models realistically: a case study on public and proprietary data. *Journal of Cheminformatics*. 2023;15(1):112. doi: [10.1186/s13321-023-00781-1](https://doi.org/10.1186/s13321-023-00781-1)
>
> Gao W, Coley CW. The Synthesizability of Molecules Proposed by Generative Models. *Journal of Chemical Information and Modeling*. 2020;60(12):5714-5723. doi: [10.1021/acs.jcim.0c00174](https://doi.org/10.1021/acs.jcim.0c00174)
>
> Parrot M, Tajmouati H, da Silva VBR, Atwood BR, Fourcade R, Gaston-Mathé Y, Do Huu N, Perron Q. Integrating synthetic accessibility with AI-based generative drug design. *Journal of Cheminformatics*. 2023;15(1):83. doi: [10.1186/s13321-023-00742-8](https://doi.org/10.1186/s13321-023-00742-8)
>
> Quinn TR, Giblin KA, Thomson C, Boerth JA, Bommakanti G, Braybrooke E, et al. Accelerated Discovery of Carbamate Cbl-b Inhibitors Using Generative AI Models and Structure-Based Drug Design. *Journal of Medicinal Chemistry*. 2024;67(16):14210-14233. doi: [10.1021/acs.jmedchem.4c01034](https://doi.org/10.1021/acs.jmedchem.4c01034)
>
> Giblin KA, Song K, Chen H, Chen W, Dong Z, Escobar RA, et al. Generative AI-Assisted Discovery of HPK1 Inhibitors. *Journal of Medicinal Chemistry*. 2026;69(15):18723-18738. doi: [10.1021/acs.jmedchem.6c01048](https://doi.org/10.1021/acs.jmedchem.6c01048)
>
> Choung OH, Vianello R, Segler M, Stiefl N, Jiménez-Luna J. Extracting medicinal chemistry intuition via preference machine learning. *Nature Communications*. 2023;14(1):6651. doi: [10.1038/s41467-023-42242-1](https://doi.org/10.1038/s41467-023-42242-1)
>
> Sundin I, Voronov A, Xiao H, Papadopoulos K, Bjerrum EJ, Heinonen M, Patronov A, Kaski S, Engkvist O. Human-in-the-loop assisted de novo molecular design. *Journal of Cheminformatics*. 2022;14(1):86. doi: [10.1186/s13321-022-00667-8](https://doi.org/10.1186/s13321-022-00667-8)
>
> Menke J, Nahal Y, Bjerrum EJ, Kabeshov M, Kaski S, Engkvist O. Metis: a python-based user interface to collect expert feedback for generative chemistry models. *Journal of Cheminformatics*. 2024;16(1):100. doi: [10.1186/s13321-024-00892-3](https://doi.org/10.1186/s13321-024-00892-3)
>
> Hassen AK, Šícho M, van Aalst YJ, Huizenga MCW, Reynolds DNR, Luukkonen S, Bernatavicius A, Clevert DA, Janssen APA, van Westen GJP, Preuss M. Generate what you can make: achieving in-house synthesizability with readily available resources in de novo drug design. *Journal of Cheminformatics*. 2025;17(1):41. doi: [10.1186/s13321-024-00910-4](https://doi.org/10.1186/s13321-024-00910-4)

## What the figure doesn't cover

- **Version, and it is the dominant error here.** Every independent benchmark figure on this page is for REINVENT 1 (Olivecrona 2017) or REINVENT 2.0. The platform runs REINVENT 4, cloned unpinned from the project's default branch. The strong PMO ranking and the weak rediscovery numbers are both for software that is not what runs here, and REINVENT 4's own paper is a software description with no comparative evaluation of generated-molecule quality.
- **Nothing on this platform has been validated for REINVENT.** This page is literature only. The platform's internal validation harness is not evidence of anything about this tool, and no figure here was measured on a platform run.
- **Benchmark oracles are not assays.** PMO's objectives are property calculators and QSAR-style oracles; the AUC top-10 that put REINVENT first measures how efficiently it maximizes a computed number. A model that maximizes a computed number well is a good optimizer, which is a different claim from being a good designer.
- **Reward exploitation is the characteristic failure.** When the scoring model is the only judge, the optimizer finds the scoring model's blind spots. The rediscovery study shows this directly: reinforcement learning reached scores of 0.7–1.0 while the compounds the real project advanced scored 0.1–0.8. A high score is evidence about the scoring function.
- **Validity, uniqueness, novelty and diversity say nothing about the bench.** They are properties of a set of strings. The rediscovery study reported validity above 98% and novelty above 70% on exactly the proprietary projects where top-100 rediscovery was zero.
- **Makeability has no REINVENT-specific number, and the proxies disagree.** SA_Score reaches an AUC of 0.87 against retrosynthesis-judged synthesizability across databases but only 0.63 against chemist labels spanning three different series. Two of the published CASP studies disagree about the base rate — roughly 30% of hard-objective top-100 molecules routable by one oracle, versus 98% of GuacaMol top-100 routable by another, which that paper's authors took as evidence the benchmark was trivial. Neither number is "the" synthesizability rate.
- **The human-in-the-loop literature on REINVENT is mostly simulated.** The main study of chemist feedback steering REINVENT states plainly that "we use an oracle to simulate the responses of a chemist instead of including a real human in the loop". Its real-chemist arm was **one** chemist, who was a coauthor, over three runs of 100 queries, and the output was graded by a Random Forest DRD2 classifier rather than an assay; mean predicted activity rose from 0.18 ± 0.06 to 0.52 ± 0.15, and **one of the three runs failed to improve at all**. The companion feedback-collection tool for REINVENT ships only a fictitious JNK3 example and reports no chemist measurement. Treat "chemists improved the designs" as an unmeasured claim for this tool.
- **Published campaign outcomes belong to the campaign.** Both AstraZeneca programs ran REINVENT inside multi-stage pipelines with structure-based design, physics-based affinity prediction, DMPK models, and chemists selecting what to synthesize. Attributing their compounds to the generator is the central error in reading this literature.
- **No prospective REINVENT denominators exist.** Nobody has published how many REINVENT designs were made and how many were active. Until someone does, there is no hit rate for this tool, and any number you are offered for one is from a different generator or a different stage.
- **The default run computes nothing about quality.** A `sampling` run returns molecules scored by prior likelihood only. See [outputs and monitoring](/docs/reinvent/outputs).

## Go / no-go

**Go — act on the result when:**

- You want design ideas to put in front of a chemist, and the chemist is the gate. This is the use the published programs actually made of it.
- You are enumerating locally around a validated series — scaffold decoration, R-group replacement, linker design — where the surrounding chemistry is familiar and makeability is the smaller risk.
- You explicitly need REINVENT's reinforcement-learning machinery: a configurable multi-term [scoring function](/docs/reinvent/scoring), diversity filters, [curriculum stages](/docs/reinvent/loops), or transfer learning onto your own chemistry. Otherwise [GenMol](/docs/genmol) is the platform's preferred generator and REINVENT is soft-deprecated.
- You have a scoring model whose applicability domain actually covers the chemistry you want, and you have checked that it scores your known actives highly. If it does not, reinforcement learning will optimize away from them.
- You intend to run every surviving molecule through a retrosynthesis plan and a property screen before anyone orders anything.

**No-go — do not act on the result when:**

- You are treating a high score as evidence of activity, drug-likeness, or synthesizability. In the default `sampling` mode the score is prior likelihood and means none of those things.
- You intend to order or register molecules straight from the output without a retrosynthesis check and a chemist reading the structures.
- Your project has no seed actives to train the scoring model on. The rediscovery study's own success criterion is that promising compounds must already be in the training set; without them the generated chemistry resembled the project's failures more than its successes.
- You are quoting a REINVENT benchmark figure whose version you have not checked, or quoting a published campaign's potency as this tool's output.
- You need a hit rate to plan against. There is no published one for this tool.

**Caveats on reading this page:** every figure above is quoted from the cited source with its original metric and sample, and the version of REINVENT each figure describes is named because the versions are not interchangeable. Figures from different studies are not comparable, because generators, CASP oracles, objectives and judgement criteria differ between them. Where a number for REINVENT does not exist, this page says so instead of substituting one from another tool.