DeepPurpose DTI Prediction vs a Binding Assay: What the Score Means

A DeepPurpose drug–target interaction score is not a binding constant. It is a deep-learning model's estimate of the label it was trained on, computed from a SMILES string and a protein sequence alone. DeepPurpose (Huang et al., Bioinformatics 2021) is a library of compound and protein encoders that learn to predict binding labels from public data. Which label depends on which pretrained checkpoint you run. A Davis checkpoint predicts pKd for kinase inhibitors. A BindingDB IC50 checkpoint predicts pIC50, and a KIBA checkpoint predicts a KIBA score. A binding assay such as surface plasmon resonance (SPR) or isothermal titration calorimetry (ITC) measures the interaction directly. The score is good for one job: deciding which drug–target pairs are worth measuring. It is close to its training data when the drug and the target resemble pairs it has seen, and much weaker when either is new. Treat it as a way to order your assay queue, not as a result.

This page lines the two up side by side: what each number actually is, how far apart they can be, and how to use the predicted one without mistaking it for the measured one. It is about sequence-based affinity prediction, not docking. For pose-based methods and Boltz-2's affinity head, see DiffDock vs Chai-1 vs Boltz-2.

At a Glance#

The noise-floor row comes from Kramer et al., J. Med. Chem. 2012, discussed below. It matters because every DTI model is trained on labels carrying that noise.

What a DeepPurpose Score Actually Is#

DeepPurpose's own summary is that it implements "15 compound and protein encoders and over 50 neural architectures" for DTI prediction and reports state-of-the-art performance on benchmark datasets (Huang et al. 2021; kexinhuang12345/DeepPurpose). Mechanically, one encoder turns the SMILES into a vector: a Morgan fingerprint, or a message-passing neural network (MPNN) over the molecular graph. A second encoder, usually a convolutional network over the amino-acid string, turns the protein into another vector. A small network maps the pair to one number. No 3D structure enters at any point.

That number is a regression onto whatever label the checkpoint was trained on, so the checkpoint defines the units. The ones the deeppurpose_predict job loads come from three datasets. Each was measured differently:

The Davis panel tested "72 kinase inhibitors with 442 kinases covering

80% of the human catalytic protein kinome"

(Davis et al., Nature Biotechnology 2011). The benchmark version DeepDTA popularised keeps 68 of those ligands and 442 kinases, 30,056 interactions in all. It uses the dissociation constants, transformed as pKd = −log10(Kd / 1e9) with Kd in nM (Öztürk et al., Bioinformatics 2018). KIBA was built to reconcile heterogeneous kinase bioactivities, and it explicitly merges "IC50, K(i), and K(d)" into one model-based score (Tang et al., J. Chem. Inf. Model. 2014). BindingDB is a public collection of "over a million data entries" of experimental protein–small molecule interaction data, drawn mostly from articles and US patents (Gilson et al., Nucleic Acids Research 2016).

Three consequences follow, and none of them is visible in the number itself.

• A Davis or KIBA score outside the kinome is extrapolation. Both datasets are kinase panels. If you send the default checkpoint a GPCR or a protease sequence, it still returns a pKd-shaped number. Nothing in its training set resembles your input.
• A KIBA score cannot be read as pKd. Upstream DeepPurpose loads KIBA without a log conversion, and its loader notes that KIBA "is a modified score". The value lands in the same output field as a pKd from a Davis checkpoint.
• A BindingDB IC50 prediction carries the assay-dependence of IC50. It is an estimate of a typical reported IC50 across many labs and formats, not of the IC50 your own assay will report.

What a Binding Assay Measures#

The assay side has its own vocabulary, and the distinctions are exactly the ones a DTI score hides.

Kd, the dissociation constant, describes the drug–target complex itself: the concentration at which half the target is bound at equilibrium. SPR measures it by flowing compound over immobilised protein and recording association and dissociation in real time, so it also gives kinetic rate constants and residence time. ITC measures the heat released or absorbed as compound is titrated into protein in solution. It gives Kd together with the binding enthalpy and the stoichiometry, with no immobilisation and no label. Both report direct binding, which is why they sit at the confirmation end of a screening cascade.

IC50 is the concentration that halves an activity in one specific assay. It is not a property of the complex: change the substrate concentration and the IC50 moves. Ki is the inhibitor's dissociation constant. For a competitive inhibitor it follows from IC50 through the Cheng–Prusoff relation, Ki = IC50 / (1 + [S]/Km) (Cheng & Prusoff, Biochemical Pharmacology 1973). This is why a compound can have a single Kd but many IC50s.

Measured values are noisy too, and the noise is quantified. Kramer and colleagues compared independent Ki measurements of the same protein–ligand pairs in ChEMBL. They estimated a mean error of 0.44 pKi units, a standard deviation of 0.54 and a median error of 0.34. They put the maximum achievable squared Pearson correlation on large data sets at 0.81, and concluded that "models that yield errors smaller than the experimental uncertainty are necessarily overtrained" (Kramer et al. 2012). A companion study found the standard deviation of mixed public IC50 data "only 25% larger" than that of Ki data. It recommended a Ki-to-IC50 conversion factor of 2 when the two are mixed (Kalliokoski et al., PLoS ONE 2013).

Put those together and the ceiling is clear. A model trained on BindingDB-style public labels cannot agree with a fresh measurement more closely than two independent measurements agree with each other. Where it seems to on a benchmark, the benchmark has leaked.

How Close Does the Prediction Get?#

There are two questions here, and they have different answers. The first is how well the model ranks pairs that look like its training data. The second is how well it does on pairs that don't.

On a random split of a benchmark, the deep models are close to the kernels. DeepDTA's own Davis table, under a random split, reports these pairs of concordance index (CI) and mean squared error (MSE). A CNN–CNN model scores CI 0.878 and MSE 0.261. SimBoost, a gradient-boosting method, scores 0.872 and 0.282. KronRLS, a kernel method with no deep learning, scores 0.871 and 0.379 (Öztürk et al. 2018). The deep model's clearest advantage is in squared error, not ranking. All of these numbers come from splits where the same drugs and kinases appear on both sides of the train/test line.

On new proteins and new ligands, the advantage mostly disappears. The most direct test of DeepPurpose itself comes from an independent group (Chatterjee et al., Nature Communications 2023). They noticed that binding annotations in BindingDB follow fat-tailed degree distributions, where a few hub proteins and ligands carry most of the records. So they compared DeepPurpose with a network configuration model, which predicts binding only from how many positive and negative annotations each node has. It sees no SMILES and no sequence. Their results:

They then randomly shuffled the SMILES and amino-acid sequences in the training set, keeping each node's annotations. This "did not change the test performance." They concluded that DeepPurpose "leverages network topology as a learning shortcut and fails to generalize predictions to proteins and ligands beyond the training data."

Two caveats keep this honest. These are binary binder/non-binder models trained on BindingDB, not the regression checkpoints the platform loads, so carry over the direction of the result, not the numbers. And "fails to generalize" describes new proteins and new ligands together. A close analogue of a known inhibitor against a well-annotated kinase is the case the models handle best, and it is also the case where you learn least from them.

The practical reading: a DTI score is strongest where you need it least. For a known chemotype on a well-studied kinase, it largely restates what public data already says. For a novel target or scaffold, it tells you little more than a model that never looked at the molecule.

When to Use Each#

Use a DeepPurpose prediction when#

• You need to order a list, not answer a question. Ranking a few dozen candidates against one kinase before choosing which to send for a biochemical panel is the job the model fits.
• The target is a kinase close to the Davis or KIBA panels and the compounds resemble known inhibitors. That is the regime the checkpoints were trained in.
• You want a cheap prior for an off-target question before committing assay budget, and you will measure anything that comes back high.

Use a biochemical Ki or IC50 when#

• You are ranking a series for potency or selectivity in a screening cascade, under one fixed assay format, so that values are comparable to each other.
• The readout is functional. Inhibiting an enzyme's activity is what you care about, and binding alone is not enough.

Use SPR or ITC when#

• You need to know the compound binds the target directly, not that it changes an assay signal by some other route.
• Kinetics or thermodynamics matter. Residence time from SPR, or enthalpy and stoichiometry from ITC, are things no DTI score estimates.
• A decision depends on the number. Advancing a compound, claiming selectivity, or reporting an affinity all need a measurement.

Limitations to Keep in View#

• No structure means no mechanism. A sequence-only model cannot say where or how a compound binds. It cannot tell an orthosteric binder from an allosteric one, or a covalent mechanism from a reversible one.
• The checkpoint's training domain is invisible in the output. The same field carries pKd, pIC50 or a KIBA score depending on what ran. Always read which checkpoint produced the number.
• Benchmark figures are measured on random splits. Ask whether a quoted accuracy was measured on new proteins, new ligands or neither before you apply it to your target.
• The labels are noisy. Agreement with public data is bounded by the roughly 0.44 pKi disagreement between independent measurements, so very tight agreement is a warning sign.
• Annotations are unevenly spread. Some proteins and ligands carry far more positive than negative records, and the reverse holds for others. The AI-Bind authors show models learn that imbalance, predicting that some nodes bind "disproportionately more often" whatever the partner (Chatterjee et al. 2023). A high score for a heavily annotated kinase partly reflects how often that kinase appears in the data.

How to Run It on .bloom*#

The platform runs DeepPurpose as deeppurpose_predict, an asynchronous GPU job. It loads a pretrained checkpoint and scores the pair you send. It does not train or fine-tune a model. The binding assays in this article are wet-lab methods and are not offered on .bloom*. The job's purpose is to choose what goes to them.

What one call computes. Set task_type to dti and pass one SMILES in drug and one amino-acid sequence in target, each as a one-element array.

One call scores exactly one drug–target pair. The prediction_mode field (single, virtual_screening, repurposing) is a label, not a batch mode. To screen several compounds against a kinase, or one compound against several kinases, run one call per pair. The tool definition estimates 30–120 seconds per call.

Which checkpoint runs. With no pretrained_model set, the job loads MPNN_CNN_DAVIS, an MPNN drug encoder with a CNN protein encoder trained on the Davis kinase panel, and its output reads as pKd. You can choose a BindingDB IC50 checkpoint (morgan_cnn_bindingdb_ic50 is the general-purpose choice, and cnn_cnn_, daylight_aac_, morgan_aac_ and mpnn_cnn_ variants exist) or a kinase checkpoint: mpnn_cnn_davis, morgan_cnn_davis, mpnn_cnn_kiba or morgan_cnn_kiba. The job sets the drug and protein encodings from the checkpoint name, so they always match the model. Every checkpoint returns a continuous value. Remember the units table above: a KIBA checkpoint's value is not pKd.

Getting the inputs right. The model needs a real sequence and a real SMILES, not names. Ivy fetches the sequence with uniprot_search and the canonical SMILES with pubchem_properties before it runs the prediction, and loops one deeppurpose_predict call per pair when you ask for a panel. You can make the same calls yourself from the terminal, over MCP or over HTTP. See the UniProt and PubChem docs.

What comes back. A predictions CSV with the SMILES, the sequence and a prediction_score for the pair, plus a record of which checkpoint and which encodings produced it. Keep that record with the number, because the number's units depend on it. Parameters and examples are in the DeepPurpose docs, with the quickstart and the inference guide.

Access. deeppurpose_predict is on the guest tool list, so a single prediction runs without an account.

The same tool also runs compound property models with task_type drug_property (Caco-2 permeability, CYP inhibition, hERG and similar endpoints). That is a different question, covered in our ADMET prediction guide. Drug–drug interaction, protein–protein interaction and protein-function tasks are not offered here, because DeepPurpose publishes no pretrained checkpoint for them.

Key Takeaways#

• A DTI score is an estimate of a training label, not a binding constant. The checkpoint decides whether that label is pKd, pIC50 or a KIBA score.
• The default checkpoint is a kinase model. MPNN_CNN_DAVIS was trained on Davis kinase Kd values. Outside the kinome its output is extrapolation.
• Measured values have a noise floor too. Independent Ki measurements disagree by about 0.44 pKi on average, which caps how well any model trained on public data can agree with a new measurement.
• Generalisation is the real limit. On new proteins and new ligands, DeepPurpose's AUROC fell to 0.61 in an independent test. A configuration model with no chemistry in it matched it in cross-validation.
• SPR and ITC measure direct binding, with kinetics or thermodynamics on top. A biochemical Ki or IC50 measures function in one assay. Use the prediction to decide which of them to run.

Score a candidate list against its target, keep the pairs worth measuring and the record of which checkpoint scored them, and send those to the bench. Run it on the .bloom* platform.

References#

1. Huang, K., Fu, T., Glass, L.M., Zitnik, M., Xiao, C. & Sun, J. "DeepPurpose: a deep learning library for drug–target interaction prediction." Bioinformatics 36, 5545–5547 (2020). doi:10.1093/bioinformatics/btaa1005 · code: kexinhuang12345/DeepPurpose
2. Davis, M.I., Hunt, J.P., Herrgard, S. et al. "Comprehensive analysis of kinase inhibitor selectivity." Nature Biotechnology 29, 1046–1051 (2011). doi:10.1038/nbt.1990
3. Tang, J., Szwajda, A., Shakyawar, S. et al. "Making Sense of Large-Scale Kinase Inhibitor Bioactivity Data Sets: A Comparative and Integrative Analysis." Journal of Chemical Information and Modeling 54, 735–743 (2014). doi:10.1021/ci400709d
4. Gilson, M.K., Liu, T., Baitaluk, M. et al. "BindingDB in 2015: A public database for medicinal chemistry, computational chemistry and systems pharmacology." Nucleic Acids Research 44, D1045–D1053 (2016). doi:10.1093/nar/gkv1072
5. Öztürk, H., Özgür, A. & Ozkirimli, E. "DeepDTA: deep drug–target binding affinity prediction." Bioinformatics 34, i821–i829 (2018). doi:10.1093/bioinformatics/bty593 · code: hkmztrk/DeepDTA
6. Chatterjee, A., Walters, R., Shafi, Z. et al. "Improving the generalizability of protein-ligand binding predictions with AI-Bind." Nature Communications 14, 1989 (2023). doi:10.1038/s41467-023-37572-z
7. Kramer, C., Kalliokoski, T., Gedeck, P. & Vulpetti, A. "The Experimental Uncertainty of Heterogeneous Public Ki Data." Journal of Medicinal Chemistry 55, 5165–5173 (2012). doi:10.1021/jm300131x
8. Kalliokoski, T., Kramer, C., Vulpetti, A. & Gedeck, P. "Comparability of Mixed IC50 Data – A Statistical Analysis." PLoS ONE 8, e61007 (2013). doi:10.1371/journal.pone.0061007
9. Cheng, Y. & Prusoff, W.H. "Relationship between the inhibition constant (KI) and the concentration of inhibitor which causes 50 per cent inhibition (I50) of an enzymatic reaction." Biochemical Pharmacology 22, 3099–3108 (1973). doi:10.1016/0006-2952(73)90196-2