---
title: How P2Rank results compare to the bench
description: >-
  What a P2Rank pocket has been shown to agree with experimentally, and why
  almost every published success rate was measured on ligand-bound structures.
tags:
  - p2rank
  - validation
  - translatability
---

**You have a P2Rank result. Here is what it supports.**

- **A top-ranked pocket is a hypothesis about *where* a ligand would bind** — not about whether anything binds, and not about how tightly. P2Rank scores solvent-accessible surface points and clusters the positive ones; nothing in the method models a ligand.
- **Never read a success rate without its distance definition, its threshold and its Top-N rule.** The numbers on this page are not interchangeable. P2Rank's authors report **72.0%** on COACH420 at **DCA ≤ 4 Å, Top-n**; an independent benchmark reports **51.9%** on LIGYSIS at **DCC ≤ 12 Å, Top-N+2**; another reports **47.6%** on PoseBusters as a **precision** at **DCA ≤ 4 Å, Top-N<sub>L</sub>**. Those three describe different questions, and a bare percentage is unusable.
- ⊘ **Almost every published figure was measured on holo structures** — coordinates solved with a ligand bound, so the pocket is already open. On matched apo counterparts of **cryptic** sites, P2Rank's residue-level AUPRC fell from **0.34 to 0.21** and its true-positive rate from **0.84 to 0.62**. On a *general* apo set, though, pocket predictors show little measurable penalty. The question for your target is not "is this apo?" but **"does this site move?"** See [the apo problem](#the-apo-versus-holo-problem-is-the-whole-story).
- **On an AlphaFold model, P2Rank holds up — but only to the extent the model is accurate.** An independent benchmark on 688 paired PDB/AlphaFold2 structures found **top-1 DCA identical at 0.81 on both**, while fpocket collapsed from 0.75 to 0.50. The catch is that those AF2 models were near-perfect (1.6% above 2 Å backbone RMSD). Degrade them to the accuracy you actually get for a novel target with no homolog and top-1 success falls to **about 60% at 2 Å RMSD and 40% at 4 Å**. See [predicted structures as input](#predicted-structures-as-input).
- ⚠︎ **"P2Rank at 4 Å" means two different things.** Its authors measure **DCA** (centre to nearest ligand atom) and get 68.6% on HOLO4K. An independent group measuring **DCC** (centre to centre) at the same 4 Å on the same set gets **53.6%**. Same tool, same threshold, 15 points apart, because the distance is defined differently.
- **Confirm a site's location with** a co-crystal or cryo-EM structure that resolves the ligand. **Confirm that the site is functionally real with** mutagenesis — alanine scanning or binding-site point mutants assayed for loss of binding or activity. No published study pairs a P2Rank prediction with either.
- **Deciding between pocket-then-dock and blind docking?** [DiffDock](/docs/diffdock) needs no site at all, so P2Rank's error does not propagate into it. Its pose figures are on [its own translatability page](/docs/diffdock/translatability) — read them next to the numbers here before you spend a pocket prediction on constraining a search.

Jump to [agreement with wet lab](#agreement-with-wet-lab) or [against other in silico methods](#against-other-in-silico-methods). The general framing for every tool is in [reading an in silico result against the bench](/docs/translatability).

## How in silico is used currently

Pocket prediction is used to **decide where to look**, and it is almost always a routing step rather than a result. Three jobs in a live program:

- **Pre-docking site selection.** You have a structure and no annotated site, and a pocket-constrained docking program needs a box. This is the dominant use, and it is also the one where a wrong answer is cheapest to detect: the docking stage will usually fail visibly.
- **Target triage at scale.** You ask whether a novel target has anything pocket-shaped on it at all, across hundreds or hundreds of thousands of structures. P2Rank is the method of choice here because it is seconds per protein and needs no homolog, no alignment and no database — the PrankWeb authors precomputed predictions for over **800,000** AlphaFold Database models on that basis.
- **Allosteric and secondary-site hunting.** You already know the orthosteric site and want to know what else is there. This is the use with the weakest published support, and the one where the method's blind spots bite hardest — cryptic sites are the documented failure case, and allosteric sites are structurally and evolutionarily unlike the orthosteric pockets the model was trained on.

What pocket prediction is not used for, by anyone reporting their validation honestly, is deciding that a site is druggable. That claim needs a ligand.

## Against other in silico methods

On P2Rank's own benchmarks it beats everything it was compared against — fpocket, SiteHound, MetaPocket 2.0 and DeepSite — at both rank cutoffs on both sets (table in [agreement with wet lab](#agreement-with-wet-lab)). Its authors were careful about what that meant even in 2018: "nominally high results reported by the authors of respective methods may not be always indicative of their true performance on unseen proteins."

Independent evaluation has largely borne that out, and the pattern is more interesting than "it lost." P2Rank is still a front-rank baseline seven years on — it ties for first at Top-N on LIGYSIS with conservation on, leads at top-1 on AlphaFold input, and beats fpocket on every set anyone has run. Where it loses, it loses in two specific ways: on **coverage**, because methods that emit far more candidates recall more sites once the rank cutoff widens, and on **shape**, because its pockets are point clouds. Its one clear lead on a conventional benchmark is the set closest to its training data.

- **LIGYSIS, 2,775 chains, top-N+2 recall at DCC ≤ 12 Å.** P2Rank 51.9%, behind three configurations — fpocket re-scored by PRANK (60.4%), fpocket re-scored by DeepPocket (58.1%) and P2Rank's own conservation variant (53.9%) — and ahead of GrASP, plain fpocket, VN-EGNN, PUResNet and IF-SitePred. The study's conclusion is less about any single method than about **scoring**: the biggest gains came from re-ranking an existing pocket finder's output rather than from a new architecture, and redundant predictions cost every method recall. ⚠︎ The method that won is **fpocket re-scored by PRANK** — and PRANK ships inside the same P2Rank package that runs here, but only the `predict` mode is wired up as a tool, not the re-scoring mode. The best configuration in the independent benchmark is therefore one you cannot run on this platform either.
- **Astex Diverse Set and PoseBusters, precision under DCA at 4 Å, Top-N<sub>L</sub>.** P2Rank 46.4% and 47.6%, ahead of fpocket (41.3%, 40.7%), LIGSITEcs, AutoSite, CASTp, CAVITY and PocketMiner (5.4%, 9.0%), behind an ESM2-embedding ensemble (53.1%, 53.7%). On **Human BioLiP** P2Rank is the best method tested at 50.6% — the row the authors attribute to training-set overlap.
- **Membrane-embedded interface sites, top-7 success.** On extrahelical intramembrane pockets of **39 GPCR** and **59 ion-channel** complexes, graded at DCC ≤ 4 Å and DVO ≥ 0.2, P2Rank reached **31% / 23%** on GPCRs and **34% / 22%** on ion channels, against **58% / 29%** on the soluble PDBbind reference set in the same study. On top-1 median values its DVO was **0.00** on both membrane sets, and the authors conclude P2Rank "accomplished negligible DVO and DCC values across membrane protein data sets, making it less suitable for this prediction task." DeepPocket and PUResNetV2.0 led on both membrane sets. Two caveats on this row. The set construction excludes the orthosteric intrahelical pocket and the ion-channel pore deliberately, as easy to detect, so this is the hard subset of membrane sites rather than membrane proteins in general. And the DCC threshold here is 4 Å — the value the LIGYSIS group argues is too strict for a centre-to-centre measure — so these DCC numbers are harsher by construction than the 12 Å figures above and should not be read against them.

One 2025 comparison is worth quoting with its contradiction intact. GENEOnet's abstract reports that its score for "the probability that the top-ranked pocket is the correct one is 0.764, compared to 0.702 for P2Rank, the next best performing algorithm on our PDBbind test set" (BINDTEST, 6,854 complexes, disjoint from training with close homologs removed). But the body of the same paper says: "The evaluation of DCA and DCC distances using GENEOnet reveals that it **does not outperform other methods, such as P2Rank, DeepPocket, and CAVIAR**." On the two metrics this page is built on, P2Rank won.

**Against blind docking.** [DiffDock](/docs/diffdock) places a ligand across a whole protein without being given a site, which means it does not inherit a pocket-prediction error — and also that it answers a different question, since it needs a specific ligand and P2Rank does not. Choose pocket-then-dock when you have a target and no ligand in mind, or when your docking program requires a box; choose blind docking when you have a specific ligand and want the site and the pose in one step. The pose accuracy figures for that route, and the version traps in reading them, are on [DiffDock's translatability page](/docs/diffdock/translatability).

> Bocchi G, Frosini P, Micheletti A, et al. GENEOnet: a breakthrough in protein binding pocket detection using group equivariant non-expansive operators. *Scientific Reports*. 2025;15(1):34597. doi: [10.1038/s41598-025-18132-5](https://doi.org/10.1038/s41598-025-18132-5)
>
> Pliushcheuskaya P, Künze G. Evaluation of Small-Molecule Binding Site Prediction Methods on Membrane-Embedded Protein Interfaces. *Journal of Chemical Information and Modeling*. 2025;65(13):6949-6967. doi: [10.1021/acs.jcim.5c00336](https://doi.org/10.1021/acs.jcim.5c00336)

## In silico vs. wet lab

Two different experiments answer two different questions here, and the page keeps them apart.

**For where the ligand sits, the standard is a co-crystal or cryo-EM structure with the ligand resolved.** It localises the site directly, and because it is the thing predictors are graded against, the comparison has settled metrics:

- **DCA** — the distance from the predicted pocket's centre to **any atom of the ligand**. The conventional success threshold is **≤ 4 Å**.
- **DCC** — the distance from the predicted pocket's centre to the **centre of the observed site**. Four ångström is too strict for DCC, because a pocket's centroid and a ligand's centroid are not the same point. Some independent work nonetheless applies 4 Å to DCC, which costs P2Rank about 16 points; the LIGYSIS group argues for **10–12 Å** as the DCC value comparable to DCA at 4 Å. **Both conventions are in print, so the threshold alone does not tell you which measure you are reading.**
- **Top-N rule** — predictions come as a ranked list, so a success rate is quoted at a rank cutoff. P2Rank's literature uses **Top-n** and **Top-(n+2)**, where *n* is the number of relevant ligands in the structure. For a single-ligand protein these are the familiar Top-1 and Top-3.
- **Volume overlap** — DVO, the Jaccard overlap between predicted and observed pocket volumes, usually counted as a success above 0.2. It penalises a method that gets the centre right and the shape wrong.

**For whether the site is functionally real, the standard is mutagenesis** — alanine scanning across the predicted lining, or point mutants of the specific residues, assayed for loss of binding or activity. This comparison has no settled metric and, for P2Rank, no published instance at all (see [agreement with wet lab](#agreement-with-wet-lab)).

| | Time to answer | Cost basis | Accuracy, and its metric |
|---|---|---|---|
| **Standard wet lab, location** — co-crystal or cryo-EM with the ligand resolved | Weeks to months; may never crystallize, and cryo-EM may never resolve the ligand | €200 / 96-well crystallization plate (about 56,000 runs here) + €2,000 data collection (about 560,000 runs here) + €1,000/day refinement (about 280,000 runs here per day) at a core facility's commercial tier; **cryo-EM has no public price of any kind**. [What this means](/docs/translatability#what-the-wet-lab-cost-figures-mean) | Ground truth, limited by resolution and by whether the ligand is modelled at all |
| **Standard wet lab, function** — alanine scanning or binding-site point mutants, assayed | Weeks per round, plus assay development | No public price basis found for mutagenesis plus a binding assay; the [shared cost table](/docs/translatability#what-the-wet-lab-cost-figures-mean) carries no row for it | Establishes the site is required for binding or activity. Does not give coordinates, and loss of binding can be indirect |
| **Geometry-only in silico** — fpocket | Seconds, CPU | Free, open source. No per-run basis published | 56.4% / 52.4% at DCA ≤ 4 Å, Top-n on COACH420 / HOLO4K, as reported by P2Rank's authors |
| **SOTA in silico** — deep-learning site predictors: DeepPocket, PUResNetV2.0, re-scored fpocket | Seconds to minutes, GPU | Not available on this platform | Best top-N+2 recall in the largest independent benchmark: 60.4% (fpocket re-scored by PRANK) and 58.1% (DeepPocket re-scoring) at DCC ≤ 12 Å on LIGYSIS |
| **This platform** — P2Rank 2.4.2, `default` or `alphafold` model | 71 s padded runtime, CPU | **1 credit** | **72.0% Top-n / 78.3% Top-(n+2) at DCA ≤ 4 Å on COACH420** and 68.6% / 74.0% on HOLO4K, author-reported on holo structures. Independently, **51.9% top-N+2 recall at DCC ≤ 12 Å** on LIGYSIS, and **53.6% at DCC ≤ 4 Å** on HOLO4K. On AlphaFold2 input, top-1 DCA **0.81, unchanged** from the matching experimental structures — but ~0.60 once the model is 2 Å off. **No druggability figure, because no druggability is measured** |

**Read the accuracy column by cell, never down the column.** Crystallography's accuracy is definitional — it is the thing the predictors are graded against. Mutagenesis answers a question no geometric metric can express. A pocket-prediction percentage is the share of ranked predictions inside a distance threshold, and the threshold and rank cutoff differ between the rows. What the table supports is comparing *time and cost at a stated accuracy*.

On that reading, the honest position for this tool is: **seconds instead of months, at a hit rate that is the best in its class on the benchmarks its authors built and middling on the benchmark an independent group built, and with no information at all about whether a ligand would actually bind.** At one credit it is tied for the cheapest tool on the platform; the cost of being wrong is paid downstream, not here.

Two asymmetries matter. A co-crystal tells you about one ligand in one conformation, so it can confirm a site and still miss the others — which is part of why pocket prediction exists. And P2Rank returns about **6.3 pockets per protein on COACH420 and 12.6 on HOLO4K**, against **1.2 and 2.4 true sites** respectively. You are being handed a shortlist, not an answer.

## Agreement with wet lab

### Against co-crystal structures

All figures in this subsection use the **DCA criterion at a 4 Å threshold** — the distance from the predicted pocket's centre to any ligand atom — at the **Top-n** and **Top-(n+2)** rank cutoffs, where *n* is the number of relevant ligands in the structure.

**Author-reported, on the two sets P2Rank's authors designated as their unbiased estimate** (the model was trained on CHEN11 and tuned on JOINED, and the authors state that "only results on CHEN420 [sic — COACH420] and HOLO4K datasets represent an unbiased estimate of P2Rank's performance"):

| Method | COACH420 Top-n | Top-(n+2) | HOLO4K Top-n | Top-(n+2) |
|---|---|---|---|---|
| fpocket | 56.4 | 68.9 | 52.4 | 63.1 |
| fpocket + PRANK | 63.6 | 76.5 | 62.0 | 71.0 |
| SiteHound † | 53.0 | 69.3 | 50.1 | 62.1 |
| MetaPocket 2.0 † | 63.4 | 74.6 | 57.9 | 68.6 |
| DeepSite † | 56.4 | 63.4 | 45.6 | 48.2 |
| P2Rank, protrusion feature only | 64.2 | 73.0 | 59.3 | 67.7 |
| **P2Rank** | **72.0** | **78.3** | **68.6** | **74.0** |

† **These three rates are not on the same denominator as the rest of the table.** The tools failed on part of the input, so their rates are computed only over the subsets on which they completed — SiteHound finished on 2,878 of HOLO4K's 4,009 proteins and MetaPocket 2.0 on 2,575. Reading those rows against P2Rank's is a denominator mismatch. The paper's supplement gives the honest pairwise comparisons on each tool's own subset, where the ordering does not change: on MetaPocket's HOLO4K subset, MetaPocket 57.9% Top-n against P2Rank's 68.6% on the same 2,575 proteins. And DeepSite's HOLO4K figure is contaminated by its own training data — the supplement states that DeepSite "is trained on a large dataset which contains some of the proteins that are also included in our test set (733 proteins from HOLO4K)."

COACH420 is **420 single-chain structures with 511 relevant ligands**; HOLO4K is **4,009 structures (8,588 polypeptide chains) with 9,584 ligands**, mostly multimers. Both are disjoint from CHEN11 and JOINED.

⚠︎ **Top-n is not Top-1 on HOLO4K.** It collapses to Top-1 only for single-ligand proteins. COACH420 averages 1.2 relevant ligands per protein, so its Top-n is close to Top-1; HOLO4K averages 2.4, so its 68.6% is a materially more permissive cutoff. Never restate a Top-n figure as Top-1.

Note the second-to-last row: a version of P2Rank using **one** geometric feature — protrusion, essentially the count of protein atoms within 10 Å of a surface point — already beats every other tool tested on COACH420 Top-n. Most of what this method knows is shape.

**Author-reported, the later models.** PrankWeb 3 re-trained the shipped models and published all variants on the same two sets, same DCA ≤ 4 Å criterion:

| Model | COACH420 Top-n | Top-(n+2) | HOLO4K Top-n | Top-(n+2) |
|---|---|---|---|---|
| Default (2018) | 72.0 | 78.3 | 68.6 | 74.0 |
| Default + conservation (2018) | 73.2 | 77.9 | 72.1 | 76.7 |
| Default | 71.6 | 76.8 | 72.7 | 78.0 |
| Default + conservation | 74.3 | 77.2 | 74.5 | 78.4 |
| B-factor-free | 71.2 | 77.5 | 72.1 | 77.2 |
| B-factor-free + conservation | 74.9 | 78.5 | 73.9 | 77.7 |

Three things in that table change how you read a P2Rank number here.

The **conservation** rows are the best-performing configurations published, and they are **not available on this platform** — P2Rank 2.4.2's conservation config names a model file the release never shipped, so a conservation run falls back to the default model. Conservation is also not uniformly better: it was introduced in PrankWeb 2019 at 73.2% / 77.9% on COACH420 and 72.1% / 76.7% on HOLO4K, and the authors noted it "seems to perform slightly worse on COACH420 dataset but substantially better on larger HOLO4K dataset" — it loses 0.4 points on COACH420 Top-(n+2). What it mostly buys is **precision**: the conservation model predicts 3.4 pockets per protein on COACH420 and 7.7 on HOLO4K against the default model's 6.3 and 12.6, which the authors describe as "fewer but more relevant sites." The configuration running here is the one that hands you the longer list.

The **B-factor-free** rows are the `alphafold` model variant: it exists because AlphaFold files store pLDDT in the B-factor column, so the model was retrained to **ignore** that column rather than to understand it. Those rows were measured on **experimentally resolved** structures.

And the old and new rows are not strictly comparable: the footnote records that library upgrades changed how some PDB files parse and fixed a bug in the surface-point generator, "caus[ing] the scores for the Default (old) and Default models to differ." The 2018 numbers are not a worse run of the same software.

**PrankWeb 4, the current server release, publishes no new success rates at all** — no evaluation section, no DCA figure, no benchmark table. Its only quantitative claim about the method is that P2Rank 2.5 is "approximately twice as fast as the previous version." Instead the authors defer to an outside group, writing that "benchmarking results from both independent studies and papers introducing deep learning methods show that the best-performing traditional methods, such as P2Rank, are still on par with—or even outperform—many recent deep learning approaches," and recommending one paper by name for "an independent comparison with the latest deep learning methods." That paper is the LIGYSIS benchmark below. **P2Rank's own authors point you at the 51.9% figure.**

**Independently, on a benchmark built to fix the benchmarks.** The largest independent comparison of site predictors in over a decade — 13 methods and 15 variants, on the human subset of LIGYSIS, **2,775 protein chains**, with **P2Rank v2.4.2** (the release running here) — reports recall at **DCC ≤ 12 Å**:

| | Top-N recall | Top-N+2 recall | Max recall (rank ignored) |
|---|---|---|---|
| fpocket re-scored by PRANK | **48.8%** | **60.4%** | 91.3% |
| fpocket re-scored by DeepPocket | — | 58.1% | — |
| P2Rank with conservation | **48.8%** | 53.9% | 57.0% |
| **P2Rank** | **46.7%** | **51.9%** | 57.0% |
| GrASP / fpocket / VN-EGNN / PUResNet / IF-SitePred | — | 50 / 47 / 46 / 41 / 39% | 91.3% (fpocket) |

Read the two recall columns together. At **Top-N**, P2Rank's conservation variant is **tied for first** and plain P2Rank is third; at **Top-N+2** both fall behind re-scored fpocket by 6–9 points. The difference is that fpocket emits far more candidate pockets, so widening the cutoff helps it much more — its max recall is 91.3% against P2Rank's 57.0%. **P2Rank's weakness here is coverage, not ranking:** 43% of observed sites are not in its output at any rank. If your site is one of them, no rank cutoff recovers it.

That study chose 12 Å deliberately: "A threshold of 4 Å works well for the distance to closest ligand atom (DCA) but does not for DCC." It also proposes top-N+2 recall as the universal metric for this task. At the residue level on the same set, P2Rank's mean ROC AUC was **0.74** and average precision 0.42; the conservation variant reached **0.76 and 0.46**, the best of every method that reports residue scores. Mean F1 and MCC were 0.31 / 0.26 for P2Rank and 0.36 / 0.30 with conservation, behind PUResNet (0.41 / 0.39) and GrASP — though the authors caution that "the usefulness of residue-level metrics as F1 score or MCC is limited… Pocket-level metrics, particularly recall, are more representative." Turning conservation on is worth **+346 true positives** on this set, which is the largest free improvement available to P2Rank anywhere in this literature — and it is the configuration this platform cannot run.

Removing ion-binding sites from LIGYSIS raises every method's top-N+2 recall by 5–10% except fpocket's, without changing the ranking.

**Independently, as a precision rather than a recall.** A proteome-wide study ran seven pocket finders, P2Rank among them at release 2.5 with the default config on experimental structures and the alphafold config on AlphaFold2 models, and graded **precision under the DCA criterion at 4 Å** — the fraction of the top-ranked pockets whose centre lands within 4 Å of a biologically relevant ligand:

| Set | P2Rank Top-N<sub>L</sub> | Top-N<sub>L</sub>+2 | fpocket Top-N<sub>L</sub> | Best method Top-N<sub>L</sub> |
|---|---|---|---|---|
| Astex Diverse Set | 46.4 | 34.7 | 41.3 | 53.1 (hotpocketNN) |
| PoseBusters | 47.6 | 36.9 | 40.7 | 53.7 (hotpocketNN) |
| Human BioLiP | **50.6** | 38.4 | 35.8 | 50.6 (P2Rank) |

⚠︎ **This is a precision, so Top-N+2 is *lower* than Top-N** — adding two more predictions dilutes it. In the recall-based tables above, Top-(n+2) is *higher* than Top-n for the same reason inverted. Do not compare a number from this table with a number from those.

The ordering across those three rows is the finding. P2Rank wins on Human BioLiP and loses on the other two, and the authors give the reason: the Human BioLiP advantage is "likely due to overlap between the Human BioLiP dataset and the P2Rank and Fpocket training datasets," while Astex is a conventional held-out set and PoseBusters was built from structures released after these methods were trained, "making them fairer datasets for evaluation."

**Independently, with DCC in place of DCA — and this is where the two acronyms stop being pedantry.** A point-cloud GNN study re-ran P2Rank at **DCC ≤ 4 Å, Top-n** (n = the number of known sites), alongside DVO:

| Set (as de-duplicated by that study) | P2Rank success, DCC ≤ 4 Å | P2Rank mean DVO | Best method |
|---|---|---|---|
| scPDB-derived test, 482 structures / 529 sites | **64.2%** | — | PGpocket 66.2% |
| COACH420, 298 structures / 318 sites | **55.2%** | 0.34 | PGpocket 57.8% / 0.35 |
| HOLO4K, 4,055 structures / 5,444 sites | **53.6%** | **0.26 — best of the five methods** | PGpocket 56.3% (DVO 0.24) |

Comparators at DCC ≤ 4 Å: on COACH420, PUResNet 52.8%, Kalasanty 51.0%, COACH 50.1%; on HOLO4K, PUResNet 54.0%, Kalasanty 51.0%, COACH 49.3%.

⚠︎ **Put those side by side with the author-reported rows and the whole page turns on one definition.** On **HOLO4K**, P2Rank is **68.6–72.7% at DCA ≤ 4 Å** and **53.6% at DCC ≤ 4 Å**. On **COACH420** it is **71.6–72.0% at DCA** and **55.2% at DCC**. Same tool, same sets, the same number 4, roughly 16 points apart — because DCA measures to the nearest ligand *atom* and DCC to the ligand's *centre*, and a pocket centroid is systematically further from a centre than from a nearest atom. The LIGYSIS group's conclusion follows directly: "a DCC threshold of 4 Å is too conservative, and to obtain comparable results between DCA and DCC recall, a threshold of DCC of 10–12 Å should be employed." **A P2Rank percentage without its distance definition is not a weak figure, it is not a figure.**

### Predicted structures as input

This is the case most readers are actually in, and it has exactly one independent measurement.

An Oxford group built **HAP** — 688 paired PDB and AlphaFold2 structures of the same proteins, drawn from HOLO4K — and **HAP-small**, the 280 of those pairs with under 25% sequence identity to **P2Rank's own training set**. P2Rank was run with default parameters on the experimental structures and "for predicting on AF2 structures, the AlphaFold-specific configuration was used" — the same `alphafold` variant this platform exposes. Metric: **DCA, success = "the centre of the predicted binding site being within 4Å of any ligand heavy atom"**, at fixed **Top-1 / Top-2 / Top-3**.

| | HAP (n = 688) PDB → AF2 | HAP-small (n = 280) PDB → AF2 |
|---|---|---|
| **P2Rank Top-1** | **0.81 → 0.81** | **0.78 → 0.76** |
| P2Rank Top-2 | 0.90 → 0.88 | 0.86 → 0.85 |
| P2Rank Top-3 | 0.93 → 0.89 | 0.91 → 0.88 |
| fpocket Top-1 | 0.75 → **0.50** | 0.73 → **0.48** |
| IF-SitePred Top-1 | 0.76 → 0.77 | 0.75 → 0.76 |
| DeepPocket Top-1 | 0.78 → 0.78 | 0.75 → 0.75 |
| Random baseline Top-1 | 0.12 → 0.09 | 0.12 → 0.10 |

Two readings, both supported. **P2Rank is genuinely robust to this substitution** — it loses nothing at top-1 where fpocket loses a third of its accuracy, and the authors conclude that "IF-SitePred, P2Rank and DeepPocket are competitive across PDB and AF2 structures." And the HAP-small column is a second independent leakage test aimed squarely at P2Rank's training data: dropping to under 25% identity costs it **3 points** (0.81 → 0.78), consistent with the ~3–4 points the Astex/PoseBusters comparison showed.

⊘ **But the AF2 models in that set are not the AF2 models you will have.** The authors say so themselves: "The AF2 structures in the HAP set are highly accurate, with only 11 of 688 predictions having backbone RMSD values over 2Å to the PDB structure. This represents just 1.6% of the predictions, whereas the analysis of the best prediction for each target in CASP14 suggested that for free modelling targets (those targets without a known structural homologue), **over 50% of predictions are likely to have an RMSD over 2Å**." HOLO4K proteins are public complexes, so they are plausibly inside AlphaFold's own training set.

So they degraded 21 of the structures by heating them in MD from 298 K to 548 K and re-measured against structural error. The result is the number that describes a novel target: "IF-SitePred, P2Rank and DeepPocket exhibited similar correlations between RMSD of the protein structure (to the PDB structure) and top-1 prediction success: **60% success is achieved on structures with 2Å RMSD or better, while at 4Å RMSD success rate drops to around 40%**." Even at 1 Å RMSD, "performance was significantly reduced for all methods, with a reduction in success rates of up to 15%."

The authors also offer a mechanism worth knowing: P2Rank and DeepPocket "use all-atom representations of the protein to calculate the binding sites, and so this could make their predictions more sensitive to errors in the side chains of protein structure predictions," where a backbone-only representation is not. Predicted side chains are the least reliable part of a predicted model.

⊘ **One mis-citation not to inherit.** This same paper, and others after it, cite PrankWeb 3 as having found P2Rank "equally accurate on AF2 structures." PrankWeb 3 contains no such measurement — its B-factor-free rows are on experimental structures and its AlphaFold section is a single case study with no ground truth. The evidence for AF2 robustness is the HAP table above, not PrankWeb 3.

> Carbery A, Buttenschoen M, Skyner R, von Delft F, Deane CM. Learnt representations of proteins can be used for accurate prediction of small molecule binding sites on experimentally determined and predicted protein structures. *Journal of Cheminformatics*. 2024;16(1):39. doi: [10.1186/s13321-024-00821-4](https://doi.org/10.1186/s13321-024-00821-4)
>
> Zhao Y, He S, Xing Y, Li M, Cao Y, et al. A Point Cloud Graph Neural Network for Protein-Ligand Binding Site Prediction. *International Journal of Molecular Sciences*. 2024;25(17):9280. doi: [10.3390/ijms25179280](https://doi.org/10.3390/ijms25179280)

### The apo-versus-holo problem is the whole story

This is the section to read if you read only one.

Every success rate above was measured on **holo** structures — coordinates that were solved with a ligand bound. HOLO4K says so in its name. COACH420, the Astex Diverse Set, PoseBusters, PDBbind, SC6K and the human subset of LIGYSIS are all built from ligand-bound complexes. In a holo structure the pocket is already open in the coordinates, because a ligand was holding it open when the crystal was solved. A reader holding an apo structure, or a predicted model, is not in that situation.

The one published head-to-head measures exactly this gap. CryptoBench — **1,107 apo structures** assembled from apo–holo pairs, grouped by UniProt accession, clustered by sequence identity and filtered to keep only those with substantial structural change in the binding site, split 885 train / 222 test — ran off-the-shelf P2Rank on the single-chain **apo** structures of its test set and then on **the matched holo counterparts of the same proteins**. Residue-level binary classification:

| P2Rank on | AUC | AUPRC | TPR | MCC | Accuracy | FPR | F1 |
|---|---|---|---|---|---|---|---|
| Holo structures | 0.89 | **0.34** | **0.84** | **0.38** | 0.85 | 0.15 | 0.81 |
| Matched apo structures | 0.81 | **0.21** | **0.62** | **0.27** | 0.85 | 0.14 | 0.81 |

AUPRC falls by 38%, the true-positive rate by a quarter, MCC by a third. Accuracy and F1 do not move at all — they are dominated by the negative class, which is most of the protein surface, and F1 here is a weighted average. **If you grade this transition by accuracy you will conclude nothing happened.** That is the single most instructive thing on this page about reading a pocket-prediction metric.

The paper's own framing: "current prediction methodologies often rely on holo (ligand-bound) protein conformations for training and evaluation, overlooking the significance of the apo (ligand-free) states. This oversight is particularly problematic in the case of cryptic binding sites where **holo-based assessment yields unrealistic performance expectations**."

Four caveats, stated so the figure is not over-read. These are **residue-level** metrics, not DCA or DCC, so they are not comparable to the success rates above. The set is specifically **cryptic** sites — pockets selected for large apo-to-holo change — so it is the hard end of the apo problem rather than its average. The authors note they "did not control for possible data leakage between PocketMiner/P2Rank training and CryptoBench test sets," which if anything flatters both numbers. And the exact number of structures in the two matched subsets is not stated in the paper's body, which defers to its supplement; it is some subset of the 222 test structures, after multi-chain structures were excluded. The study was run by a group that includes P2Rank's own authors, with P2Rank as the baseline, and their reading of their own result is that "P2Rank shows competitive performance on apo structures" while still dropping measurably against holo.

**The counterweight, and it matters.** A larger and older independent study took the apo question to a *general* set rather than a cryptic one: 1,446 holo plus 1,082 apo crystal structures across **304 protein families**, each represented by at least two bound and two free structures, graded at the residue level against the family's unified binding site on the **top-1** pocket. Of the seven methods tested, **only fpocket showed a statistically significant preference for holo coordinates** (F 0.42 apo → 0.53 holo, p = 0.04; MCC 0.43 → 0.52, p = 0.03). The other six — including the ML method Kalasanty, 0.49 → 0.51, p = 0.12 — did not: "the other six methods show no statistically distinguishable preference for apo or holo structures. This implies that the predictive power for most of these methods is not heavily impacted by the presence or absence of a pre-organized binding site with a ligand in the starting structure."

**P2Rank is not among the seven**, so this is not a P2Rank measurement. But it fixes the shape of the claim. That set is largely rigid — "68% of the protein families display negligible amounts of backbone motion throughout their UBS" — whereas CryptoBench is filtered *for* large apo-to-holo change. The two results are not in conflict; they are different populations. **On a general apo structure the penalty is small and may not be measurable. On a pocket that actually opens on binding, it is large.** The question to ask of your own target is not "is this apo?" but "does this site move?"

> Clark JJ, Orban ZJ, Carlson HA. Predicting binding sites from unbound versus bound protein structures. *Scientific Reports*. 2020;10(1):15856. doi: [10.1038/s41598-020-72906-7](https://doi.org/10.1038/s41598-020-72906-7)

One much older and much smaller data point sits with CryptoBench. The 2015 predecessor PRANK was evaluated on B48 and U48 — the same 48 proteins in bound and unbound states — and its Top-n success rate at DCA ≤ 4 Å fell from **81.5% on the bound set to 77.8% on the unbound set**. Read it as directional only: it is PRANK re-scoring fpocket rather than P2Rank, n is 48, and the authors call that set "a rather 'easy' dataset" of "small globular proteins with one stereotypical large binding site."

> Škrhák V, Novotný M, Feidakis CP, Krivák R, Hoksza D. CryptoBench: cryptic protein-ligand binding sites dataset and benchmark. *Bioinformatics*. 2025;41(1):btae745. doi: [10.1093/bioinformatics/btae745](https://doi.org/10.1093/bioinformatics/btae745)

### What is not measured: shape

**There is no author-reported volume-overlap, IoU or DCC figure for P2Rank anywhere, and that is deliberate.** The 2018 paper states the reason plainly: "One limitation of our tool is that it does not produce exact shapes and volumes of predicted binding sites… As a consequence, in our evaluation we did not use volumetric overlap identification criteria sometimes employed in other studies." The same limitation is why `volume` and `surface_area` come back null in a result here.

So every author-reported figure on this page grades a **point** — is the pocket's centre near the ligand — and none of them grades whether the predicted pocket has the right shape or size. All the published overlap measurements are third-party, and they do not agree with each other, because shape performance depends heavily on the target class:

- **Soluble holo sets:** P2Rank's mean DVO was **0.26 on HOLO4K — the best of the five methods tested** — and 0.34 on COACH420.
- **Membrane interface sites:** top-1 median DVO **0.00** on both GPCRs and ion channels.
- **LIGYSIS, relative volume overlap over correctly located sites only:** 58.2% for P2Rank, mid-field, against 77.0% for the geometric method Ligsite⁺.
- **Residue-level:** mean ROC AUC 0.74 on LIGYSIS; the apo/holo figures in [the apo section](#the-apo-versus-holo-problem-is-the-whole-story).

There is a structural reason P2Rank's shapes under-perform its centres, and the membrane study states it: "Since the output of the method is a set of solvent accessible points and not a set of protein residues, predicted pockets tend to have smaller volumes and less pronounced shapes, leading also to smaller pocket overlaps." The same design choice is why `volume` and `surface_area` are null — **a pocket here is a cloud of surface points, not an enclosed cavity.** Do not use a P2Rank pocket to judge whether your ligand fits.

Two related absences, so you do not go looking: **no author-reported figure exists on SC6K or on any PDBbind-derived evaluation set.** SC6K originates with PUResNet, and the P2Rank authors used Binding MOAD rather than PDBbind for their alternative ground truth. Any P2Rank number you encounter on either set is third-party.

The 2018 authors also left behind the best available rule of thumb for reading anyone's number in this field: "we are suspicious when we see reported success rates that are unrealistically high, say close to or above 95% in Top-1/Top-n category… This can be indicative of a data leakage… or overfitting on a given dataset."

### Against mutagenesis

**No published figure exists.** A PubMed search on 2026-10-22 returns 19 indexed papers mentioning P2Rank, and none of them pairs a P2Rank prediction with site-directed mutagenesis, alanine scanning, SPR, ITC, or any other functional binding measurement of the predicted site. Queries combining `P2Rank` with `mutagenesis`, `site-directed`, `alanine scanning`, `SPR`, `surface plasmon` and `isothermal titration` return zero records each.

The same holds for prospective crystallographic confirmation: no study reports a novel site predicted by P2Rank in advance and then resolved by a newly solved co-crystal or cryo-EM structure. The published uses of P2Rank in discovery campaigns are pipeline stages — the pocket feeds a docking run, and any experimental result that follows belongs to the pipeline, not to the pocket prediction.

So the functional half of this page is empty, and it is empty in the literature, not in our sourcing. **If you need to know that a predicted site is real, you are running the first experiment.**

> Krivák R, Hoksza D. P2Rank: machine learning based tool for rapid and accurate prediction of ligand binding sites from protein structure. *Journal of Cheminformatics*. 2018;10(1):39. doi: [10.1186/s13321-018-0285-8](https://doi.org/10.1186/s13321-018-0285-8)
>
> Krivák R, Hoksza D. Improving protein-ligand binding site prediction accuracy by classification of inner pocket points using local features. *Journal of Cheminformatics*. 2015;7:12. doi: [10.1186/s13321-015-0059-5](https://doi.org/10.1186/s13321-015-0059-5)
>
> Jendele L, Krivák R, Škoda P, Novotný M, Hoksza D. PrankWeb: a web server for ligand binding site prediction and visualization. *Nucleic Acids Research*. 2019;47(W1):W345-W349. doi: [10.1093/nar/gkz424](https://doi.org/10.1093/nar/gkz424)
>
> Jakubec D, Škoda P, Krivák R, Novotný M, Hoksza D. PrankWeb 3: accelerated ligand-binding site predictions for experimental and modelled protein structures. *Nucleic Acids Research*. 2022;50(W1):W593-W597. doi: [10.1093/nar/gkac389](https://doi.org/10.1093/nar/gkac389)
>
> Polák L, Škoda P, Riedlová K, Krivák R, Novotný M, Hoksza D. PrankWeb 4: a modular web server for protein-ligand binding site prediction and downstream analysis. *Nucleic Acids Research*. 2025;53(W1):W466-W471. doi: [10.1093/nar/gkaf421](https://doi.org/10.1093/nar/gkaf421)
>
> Utgés JS, Barton GJ. Comparative evaluation of methods for the prediction of protein-ligand binding sites. *Journal of Cheminformatics*. 2024;16(1):126. doi: [10.1186/s13321-024-00923-z](https://doi.org/10.1186/s13321-024-00923-z)
>
> Carpenter KA, Altman RB. The Human Omnibus of Targetable Pockets. *Journal of Cheminformatics*. 2025;17(1):180. doi: [10.1186/s13321-025-01125-x](https://doi.org/10.1186/s13321-025-01125-x)

## What the figure doesn't cover

- **The holo-only composition of every benchmark is this field's training-overlap problem.** It is not that P2Rank memorised its test sets — COACH420 and HOLO4K are disjoint from CHEN11 and JOINED, and its authors checked. It is that the *whole evaluation tradition* grades on coordinates where the pocket is already open, which is a situation a reader in a live program is frequently not in. The apo column is the one that describes you, and exactly one published study provides it for P2Rank itself.
- **The benchmarks have construction defects of their own.** Both COACH420 and HOLO4K are built on **asymmetric units rather than biological assemblies**: 40% of HOLO4K structures and 56% of COACH420 structures differ in chain count between the two, so the same protein–ligand interface can appear repeated across symmetry copies. The independent group that found this notes that "considering predictions of these interfaces as independent can lead to an overestimate of the performance of a predictor." Every author-reported figure on this page inherits that. More generally, benchmark composition dominates these comparisons: one group observed that a method it found comparable to P2Rank "significantly outperforms P2Rank" in that method's own paper, concluding that "even with large test sets, the composition of the set can make a significant difference to results." **Rank orderings between two non-fpocket methods are the least durable thing in this literature; the apo/holo and predicted-structure deltas are the most.**
- **The AlphaFold figure is conditional on the model being good, and nothing in the result tells you whether it is.** P2Rank loses nothing at top-1 on near-perfect AF2 models and drops to ~60% at 2 Å RMSD and ~40% at 4 Å — but you do not get an RMSD, because if you had the experimental structure you would not be predicting. pLDDT is the only proxy you hold, and the `alphafold` variant does not *read* it: the model was retrained to **ignore** the B-factor column, not to interpret it, so a pocket sitting entirely in a pLDDT < 50 region is scored and ranked exactly like a confident one. Check the pLDDT of the pocket-lining residues yourself; [ESMFold translatability](/docs/esmfold/translatability) and [AlphaFold translatability](/docs/alphafold/translatability) have what each confidence band is and is not backed by. Predicted side chains are the specific weak point, and P2Rank uses an all-atom representation.
- **Nothing at all is published for ESMFold input.** Every predicted-structure figure on this page is AlphaFold2. ESMFold is the less accurate of the two on hard targets, so the AF2 numbers are an upper bound rather than a substitute.
- **PrankWeb 4's authors name the leakage problem themselves.** Explaining why traditional methods still match deep-learning ones, they write that "issues related to data curation—including potential **data leakage** and **limited diversity of training data**—may be contributing factors, particularly when considering performance on well-curated and independent bench[marks]." That is the tool's own maintainers, in 2025, declining to defend the benchmark literature their figures come from.
- **The best-performing published configuration is not the one you are running.** The conservation-feature models score highest in PrankWeb 3's table (74.3–74.9% Top-n). P2Rank 2.4.2's conservation config references a model file that release never shipped, so this platform runs the no-conservation `default` or `alphafold` model and a conservation request would silently fall back. Quote 71–72%, not 74–75%.
- **The headline number is not reproducible as a single measurement.** The 2018 supplement's feature-ablation runs — averages of ten independent train-and-evaluate cycles — put the full feature set at 70.1% Top-n on HOLO4K against the 68.6% reported for the shipped default model, and the authors note that "on HOLO4K the average success rates without using those features are actually better than results reported in the paper for default P2Rank model." The shipped model is one draw, and a point or two of the spread between any two figures on this page is run-to-run variance rather than method difference.
- **The ground-truth definition is a judgement call, and the authors say so.** A ligand counts toward these rates only if it has at least 5 atoms, sits within 5.5 Å of the protein by its centre of mass, is at least 4 Å from the nearest protein atom, and is not in a hand-written exclusion list of waters, sugars, glycerol, sulfate and phosphate. The authors call that filter "admittedly arbitrary," which is why they also published variants scored against Binding MOAD instead. Both are defensible and they do not give the same number.
- **Cryptic pockets are the documented failure mode.** A site that only forms on ligand binding is not in the apo coordinates, so a surface-geometry method cannot see it. The apo/holo table above is this effect, measured.
- **Membrane-embedded sites are a second documented failure mode**, with near-zero volume overlap on top-1 predictions.
- **The druggability class is not a measurement, and it is not a P2Rank output.** It is a platform heuristic over the P2Rank score and the pocket's residue composition. P2Rank 2.4.2 reports **no pocket volume and no surface area**, so both fields come back null and the volume term is dropped from the heuristic with the remaining weights renormalised; every result carries a `druggability_caveat` saying so. ⚠︎ The tool's own description still advertises "pocket volume and surface area estimates" among its outputs — a known issue, and the fix is to correct the description to match the 2.4.2 reality.
- ⚠︎ **`chains_analyzed` does not mean what it says.** On the UniProt-accession path the backend derives a target chain from the resolved structure and reports it in `chains_analyzed`, but no chain filtering is applied before P2Rank runs — the full submitted structure is analysed, every chain of it. This is a known issue; the planned fix is to either trim the structure to the named chain or stop reporting a chain that was not enforced. Until then, treat `chains_analyzed` as provenance, not as scope, and trim the PDB yourself if you need a single chain. It matters for reading a benchmark number too: COACH420's 72.0% is on single-chain structures, HOLO4K's 68.6% on multimers.
- ⚠︎ **Two result fields are stale or empty, and both are known issues.** `pocket_coverage_percent` is declared in the result schema and reaches the generated types, but nothing computes it — it is always 0. And `p2rank_version` reports `"2.4"` on every result because the field's default is never overwritten; the installed binary is **2.4.2**. Take the version from this page, not from the result envelope, when you are matching a published figure to a release.
- **A pocket score is not a probability that a drug binds.** It is the model's confidence that the surface region resembles the ligand-binding regions it was trained on. Nothing in the pipeline estimates affinity: [MM-GBSA](/docs/mmgbsa) on this platform returns no binding free energy, so there is no computational step here that converts a pocket into a potency number.
- **Allosteric sites are not orthosteric sites.** Work including P2Rank's own authors reports a sharp divergence between the two classes in kinases, with predictors doing well on orthosteric pockets and far worse on allosteric ones — the stated reason being that allosteric sites are evolutionarily permissive and sequence-degenerate where orthosteric pockets are constrained. Treat an allosteric-site prediction as weaker evidence than a top-ranked orthosteric one.

> Riedlová K, Škrhák V, Gatlin W, Ludwick M, Turano L, Novotný M, Hoksza D, Verkhivker GM. Predicting and Decoding Allosteric Binding Sites Using Protein Language Models and Structure-Based Machine Learning: An Energy Landscape-Guided Explainable AI Framework. *Journal of Chemical Theory and Computation*. 2026;22(10):5326-5347. doi: [10.1021/acs.jctc.6c00427](https://doi.org/10.1021/acs.jctc.6c00427)

## Go / no-go

**Go — act on the result when:**

- You need a docking box and have no annotated site. At one credit and 71 seconds this is the cheapest way to get one, and the downstream docking run will usually make a bad pocket visible.
- You are triaging many targets for whether anything pocket-shaped exists, and you will treat the output as a shortlist to inspect rather than a ranking to trust. Remember you get roughly 6 pockets per single-chain protein where there is about 1 true site.
- Your input is an experimental holo structure of a protein with a conventional buried orthosteric pocket. That is the case every published success rate describes.
- You will check the top three or four pockets, not just the first. Top-(n+2) beats Top-n by 3–6 points in every author-reported P2Rank row on this page, and the known site frequently ranks second or third.
- You are going to confirm the site experimentally, and you want a prediction to aim a co-crystal attempt or a round of alanine scanning at.

**No-go — do not act on the result when:**

- You are deciding whether a target is druggable. The druggability class is a heuristic over a score and a residue list, computed without a pocket volume, and no published figure validates it against anything.
- Your input is an apo structure and the site you care about might be cryptic — a pocket that opens on binding. The apo/holo table is that measurement, and it is the method's documented failure mode. A *rigid* apo site is fine.
- Your input is a predicted model of a target with **no close homolog**. The reassuring AF2 figures were measured on models within 2 Å of the crystal structure; a genuinely novel target is more often 2–4 Å out, where top-1 success is 60% falling to 40%. Switch to the `alphafold` variant, check pocket-lining pLDDT by hand, and treat the result as a shortlist.
- You are relying on a single structure for a site you believe is conformationally sensitive. Predicting across an ensemble measurably helps — P2Rank's residue-level F1 rose about 22% when predictions from ten MD structures were combined — and nothing on this platform does that for you.
- Your target is a membrane protein and the site of interest is at the protein–membrane interface rather than in the orthosteric pocket or the pore.
- You need a single answer rather than a ranked shortlist. The ranking is the product; collapsing it to the first row discards most of the published accuracy.
- You are treating a pocket as evidence that a ligand binds, or comparing two pocket scores as if they were affinities. Neither is a thing this tool measures.

**Caveats on reading this page:** every figure above is quoted from the cited source with its original metric, threshold, rank cutoff and sample, and each was read in that paper's body text or tables rather than its abstract — except the two carrying explicit review markers. Figures from different studies are not comparable to each other. Five axes vary independently between them: the **criterion** (DCA, DCC, volume overlap, residue classification), the **threshold** (DCC at 4 Å and at 12 Å are both in print), the **rank rule** (Top-n, Top-(n+2), Top-1/2/3, Top-7, Top-N<sub>L</sub>), the **direction** (recall rises with more predictions, precision falls), and the **unit** (pocket or residue). Version: the independent 2024–2025 benchmarks used releases **2.4.2 and 2.5**, which is the line running here; the author-reported 2018 figures are an earlier build whose own maintainers note is not like-for-like.