How BioPhi results compare to the bench

You have a BioPhi result. Here is what it supports.

• An OASis score is a sequence-composition statistic, not an immunogenicity forecast. Against reported anti-drug antibody (ADA) responses for 217 therapeutics, OASis identity reaches Pearson R = −0.51, R² = 0.26 at the threshold this platform runs. That is the measured ceiling: roughly a quarter of the variance, and the best method on that set reaches only R² = 0.34. An independent group has since argued that even this correlation is a species proxy — stratify the same set by origin and the correlation disappears inside every subgroup.
• Two independent groups have expressed a BioPhi-humanized antibody and measured its binding. Both lost affinity. A BioPhi/Sapiens variant showed a ~16-fold ELISA affinity loss where conventional CDR grafting held parental-level binding, and was discarded; an scFv lost ~2.0-fold by BLI. Total published evidence: 2 antibodies, 3 variants, no thermostability, no titre series. The direction is unfavourable and the sample is far too small to generalise — but it is no longer absent.
• Do not read immunogenicity_risk as a published metric. BioPhi and OASis publish no risk categories. The low/moderate/high field is produced on this platform by two fixed cutoffs (0.85 and 0.6) applied to the humanness score. See known issues.
• Check is_real_oasis before you use the number. Real OASis requires a ~22 GB database on a Modal volume. When it is absent the job still returns SUCCESS with a different statistic — an amino-acid composition similarity — flagged is_real_oasis: false, degraded: true, method: "composition_heuristic". The card then labels it "Humanness (est.)".
• No BioPhi-humanized antibody has been expressed and assayed in the source paper. Its humanization benchmark grades agreement with mutations that human experts made, not retained affinity. The authors state plainly that "further experimental validation is necessary."
• Confirm immunogenicity with ADA assays in a human trial — there is no substitute — and de-risk beforehand with in-vitro surrogates: T-cell proliferation / PBMC assays, MHC-II binding, and DC-T (EpiScreen-type) assays. Confirm the humanized variant with expression, a binding measurement (SPR, MST, ITC) and a thermal unfolding measurement against the parental antibody.

Jump to agreement with wet lab or against other in silico methods. The general framing for every tool is in reading an in silico result against the bench.

How in silico is used currently

BioPhi covers two different jobs with two different standings in the field, and conflating them is the most common error in reading a result here.

• Humanness scoring as a triage filter. A humanness score orders candidates and localises the problem — which framework positions look least human — so an engineering campaign knows where to intervene. This is the use the method was built for and the one its figures support. It is used to rank and flag, not to predict a patient outcome.
• Humanization as a sequence-proposal step. Computational humanization generates candidate sequences that then enter the normal experimental funnel: express, measure binding, measure thermostability, pick. Nobody in the field treats a humanized sequence as finished. The traditional process, as BioPhi's own authors describe it, is "laborious" and "performed after the candidate pool has been reduced to a handful of sequences by multiple rounds of binding assays, functional assays, and basic biophysical characterization" — the computational step is proposed as a way to explore more sequence space earlier, not as a replacement for those rounds.
• Immunogenicity risk assessment, where in silico sequence scoring is the weakest link. The regulatory and industrial practice for immunogenicity is an assay cascade — MHC-II binding and epitope prediction, then T-cell proliferation or DC-T assays on donor PBMC panels, then ADA monitoring in the trial itself. A humanness score sits upstream of all of it.

What no figure located for this page supports is a humanness score used as a decision instrument — scoring a candidate and concluding something about its clinical immunogenicity. The published correlation against clinical ADA is the R² = 0.26–0.34 band described below, and it is the central fact on this page.

Against other in silico methods

On its own benchmark, OASis is mid-field on classification and mid-field on clinical correlation. The full head-to-head is the table in agreement with wet lab: Hu-mAb leads on both axes (0.977 AUC, R² 0.34), OASis follows, and a germline-identity baseline is statistically indistinguishable from OASis. Those figures were produced by BioPhi's authors on one 553-sequence set and one 217-therapeutic ADA set, so the comparison is author-run even though the competing methods are other groups' work. To their credit the BioPhi authors ran the competitors' own published servers rather than reimplementing them, which makes that table the most internally comparable set of numbers in this literature — and it is still a table a tool's authors made about their own tool.

Where OASis is genuinely differentiated is interpretability, not accuracy. The paper's own comparison table scores methods on granularity (does the score attribute humanness to individual residues or stretches), interpretability (can it explain why), and reference-library diversity. OASis and germline content are the only two scoring positively on both granularity and interpretability; Hu-mAb, AbLSTM and MG score are neither granular nor interpretable, and IgReconstruct is interpretable but not granular. OASis draws on a reference library of order 10⁸ sequences against germline content's 10². If you need to know which 9-mers at which positions are driving a low score — which is what an engineering campaign needs — that is the argument for OASis, and it is an argument about usability rather than predictive power. Note that this integration does not expose it: the per-position and per-region breakdown is a BioPhi web-interface and CLI capability, and the fields this platform returns in its place are not region measurements. So the one axis on which OASis beats Hu-mAb is the axis the integration drops, which is worth knowing before choosing between the two scorers here.

Independent evaluations do exist, and they are harsher than the author-reported table. Four groups building competing methods have graded OASis or Sapiens on their own benchmarks. They are independent of BioPhi but not disinterested, and the page labels them that way.

★ The sharpest result: the separation collapses when the non-human species is a primate. On balanced human-versus-other classification over 10,000-sequence repertoire test sets, where a random classifier scores 0.5 PR-AUC:

OASis at 0.570 against rhesus is barely above chance, and Sapiens at 0.626 is poor. Against mouse both are strong. Read together with the ADA stratification result above, this is the same story twice: these scores are good at telling rodent from human and weak at everything finer. ⚠︎ Use the test-set column, as quoted here. The same paper's "diverse" column scores OASis and Sapiens higher, and its authors explain why — that set "was built using sequences from the training sets of Sapiens and OASis, and hence are overclassified". A benchmark drawn from a model's own training data is not a measurement of that model.

The same group's re-derivation of the therapeutic task independently confirms the relaxed-threshold figure. On 549 IMGT therapeutics (196 fully human against 353 of non-human origin — BioPhi's set minus 4 light-chain-only antibodies), running OASis at the relaxed stringency, they report OASis ROC-AUC 0.975 / PR-AUC 0.963, against AbNatiV 0.979 / 0.971 and Hu-mAb 0.979 / 0.956. That 0.975 is an outside reproduction of the ~0.972 relaxed figure, on nearly the same data, and it is the best independent corroboration of any number on this page. ⚠︎ But note the same paper's methods: the other methods' scores in its table were "obtained as computed by the authors of BioPhi and deposited in their GitHub", so only the AbNatiV and OASis values there are independent recomputations.

★ A specific, exploitable failure mode: sequences "between" two human genes. OASis scores 9-mer peptides, so a sequence that starts in one human V-gene and ends in another is made of locally human peptides and scores as human overall. Tested on artificially constructed mixed-gene designs, OASis ranked only 1% of them as non-human, against Hu-mAb 8%, AbNatiV 28% and Humatch 51%. If a humanization or design process can produce chimaeric-framework sequences, OASis is close to blind to it. ⚠︎ The 51/28/8/1% figures are stated in that paper's body; their construction detail is in supplementary material that was not retrieved.

On structural consequence, BioPhi output ranks worst of three ML baselines at the residues that matter. An independent diffusion-model paper refolded each baseline's humanized sequences for 22 test antibodies and measured RMSD over Vernier-zone residues against the reference structure: BioPhi 0.5304 Å heavy chain, against AbNatiV 0.3042 Å and Hu-mAb 0.6246 Å. BioPhi was best of the three on light-chain Vernier RMSD (0.1229 Å). That paper expressed and assayed only its own designs, so this is a structural-prediction comparison, not a binding one — but the heavy-chain Vernier result lines up with the two wet-lab losses above.

Sapiens is far and away the fastest, which is a real advantage. On paired VH/VL humanization: Sapiens 3 s, Humatch 33 s, AbNatiV 218 s, Hu-mAb 1,093 s.

And the whole family's position is stated plainly by a competitor. Verbatim, from the Humatch authors in late 2024: "Currently, Humatch, and all other computational humanization tools lack experimental validation. Unfortunately, only the experimental end-points are known to be sufficiently safe to administer to patients. Our designs, and those from Hu-mAb, AbNatiV, and Sapiens are only predicted to be safe." That remains close to true: the entire wet-lab record for BioPhi output is the two antibodies above.

Ramon A, Ali M, Atkinson M, Saturnino A, Didi K, Visentin C, Ricagno S, Xu X, Greenig M, Sormanni P. Assessing antibody and nanobody nativeness for hit selection and humanization with AbNatiV. Nature Machine Intelligence. 2024;6(1):74-91. doi: 10.1038/s42256-023-00778-3

One finding here is a warning about self-graded metrics. When the authors scored Hu-mAb-optimized sequences with OASis and T20, both read lower than the expert-humanized counterparts the Hu-mAb variants were designed to match on the Hu-mAb score — and Hu-mAb's 25 sequences carried 224 framework 9-mers with no occurrence in OAS, against 12 for the experimental sequences and 7 for Sapiens×3. The authors' conclusion is the generalisable one: "the Hu-mAb metric was no longer an unbiased estimator when applied to Hu-mAb-optimized sequences." Optimising a sequence against a score and then grading it with that score measures nothing. This is the reason BioPhi deliberately separates the humanization model (Sapiens) from the scoring metric (OASis), and it is a reason to cross-check any humanized sequence with a metric that did not produce it — Humatch is the second metric on this platform.

Chinery L, Jeliazkov JR, Deane CM. Humatch - fast, gene-specific joint humanisation of antibody heavy and light chains. mAbs. 2024;16(1):2434121. doi: 10.1080/19420862.2024.2434121

In silico vs. wet lab

There are two comparators here, not one, because BioPhi answers two questions.

For the humanness / immunogenicity question, the wet-lab standard is ADA incidence measured in human trials — a tiered screening, confirmatory and titre/neutralising assay cascade run on patient samples. It is not a service you can buy for a candidate: it comes with the trial. The in-vitro surrogates that stand in for it earlier are T-cell proliferation / PBMC assays, MHC class II binding assays, and DC-T (EpiScreen-type) assays.

None of those appears in the shared cost basis, which covers crystallography, SPR, hERG, Ames and microsomal stability. This page therefore quotes no wet-lab cost figure at all rather than deriving one, and the cost column below says so. Read what the wet-lab cost figures mean for why the shared table is the only sanctioned basis.

For the humanization question, the comparator is an experimental humanization campaign: CDR grafting onto a human framework, a back-mutation scan over the framework and Vernier positions, then expression and assay of each variant for retained affinity, expression titre and thermostability. The agreement metric there is not a score — it is whether the variant still binds and still folds.

Read the accuracy column by cell, never down the column. An ADA incidence from a trial, a measured KD, a donor response frequency, a classifier AUC against a species label, and a mutation-overlap percentage against an expert's choices are five different kinds of number. Nothing meaningful comes from comparing them. What the table supports is comparing time and cost at a stated accuracy.

On that reading, the honest position for this chain is: seconds instead of weeks, for a filter — and the filter has never been shown to answer the clinical question. A humanness score at one cent is an excellent way to order a candidate list and a poor way to predict what a patient's immune system will do.

All three tools bill at the same rate. There is one price for this tool family — 1 credit per run — so humanisation, humanness scoring and the OASis analysis each cost the same regardless of how much work they do. The tool definition's own comment records real humanness compute at ~136–162 s in production, so the padded runtime that the price derives from is roughly an order of magnitude shorter than the observed job.

Two asymmetries matter. A clinical ADA rate is not a property of the molecule alone — it moves with the indication, the dosing regimen, concomitant immunosuppression, and the assay used to detect it, which is why the same antibody is reported at different incidences in different trials. And humanization can succeed on humanness while failing on binding; the score cannot see that, because binding is not in it.

Agreement with wet lab

This section is in two halves because the two outputs are graded against two different things, and neither grading is a bench measurement of the BioPhi output itself.

Humanness against clinical immunogenicity

This is the comparison that matters and the one with the lowest ceiling. BioPhi's authors tested OASis and six other humanness metrics against reported ADA responses for 217 therapeutics, a set curated by the Hu-mAb authors. Performance was reported as Pearson correlation and explained variance:

All of these figures are author-reported, from the BioPhi paper's Table 1, read in the primary full text.

Three things in that table are the whole story.

First, the best R² anything achieves is 0.34, and the platform's configuration sits at 0.26. Roughly three quarters of the variation in reported ADA across 217 therapeutics is unexplained by any of these scores. A score is a weak population-level signal, not a per-molecule prediction. Nothing here supports telling a programme that its candidate will or will not provoke ADA.

Second, OASis does not beat a trivial baseline on classification. Germline content — plain percent sequence identity to the nearest human germline, built from a reference library four orders of magnitude smaller — reaches ROC AUC 0.968 against OASis medium's 0.966, and R² 0.25 against 0.28. The authors say so themselves, and they draw the correct conclusion: the value of the repertoire-scale reference is granularity and interpretability (which 9-mers, at which positions), not raw discriminative power.

Third, the platform runs the relaxed threshold, so the relaxed row is the one that describes your result. The Modal app fixes min_fraction_subjects = 0.10, which is BioPhi's ≥10%-of-subjects setting, and converts to a percentile. The paper's headline figures (0.966 AUC, R² 0.28) are for the medium ≥50% threshold. The relaxed row is better on classification and worse on ADA correlation. A figure quoted without its prevalence threshold is unusable.

The one fully independent measurement of this relationship finds no correlation at all. Every figure in the table above comes from a tool's own authors. A Boehringer Ingelheim survey took 89 marketed antibody-based biotherapeutics (88 with ADA data), computed percent humanness as germline sequence identity with IgBLAST 1.15, and regressed it against the ADA incidence reported in the package inserts and BLA pharmacology reviews. The result, verbatim: "the linear correlation coefficients (r²) values for both VHs and VLs are 0.02 and the p-values are 0.16 (VLs) and 0.17 (VHs). Therefore, the correlation between % ADAs and % Humanness is statistically insignificant for the antibody-based biotherapeutics in this dataset."

That is R² = 0.02 against R² = 0.26–0.34 — roughly fifteenfold less variance explained, from not significant to the field's headline. Both are primary and both were read in full text, so the discrepancy is the finding, not an error to resolve. The mechanical reason is the population: the 217-therapeutic set is a score author's curation that includes murine and chimeric therapeutics with very high ADA, and that tail at the non-human end is what produces the slope; the 88-product set is marketed products only, where murine and chimeric together are 16 of 88. A humanness score separates murine from human. Within the molecules you would actually be choosing between, nothing published shows it predicting ADA.

By antibody class, the trend is not even monotonic. In the same independent survey (Table S3, 6 murine / 10 chimeric / 42 humanized / 31 human): average ADA incidence murine 18%, chimeric 5%, humanized 9%, human 5% — humanized scores worse than both chimeric and fully human. The authors state the medians "do not decrease significantly among murine, chimeric, humanized, or human antibodies".

The older class surveys say the same thing where they can speak. A review of reported anti-antibody responses graded 44 murine, 15 chimeric and 22 humanized therapeutics against fixed thresholds (negligible <2%, tolerable 2–15%, marked >15%): murine 84% marked; chimeric 40% marked, 27% tolerable, 33% negligible; humanized 9% marked, 36% tolerable, 55% negligible. Humanization clearly moves the murine and chimeric classes. But the authors also found that "if the marked category is discounted, humanized and chimeric mAb both have about equal proportions of tolerable and negligible examples", that "the overlap between HAHA and HACA incidence is extensive and blurs the advantage of humanization", and — critically — that whether fully human antibodies are less immunogenic than humanized ones "cannot be answered at present". ⚠︎ That paper has no fully-human class at all, so it is not a source for a "fully human = low ADA" figure.

And fully human does not mean non-immunogenic. A compilation of approved fully human antibodies reports panitumumab 3–4%, ofatumumab 0%, canakinumab 0%, golimumab 2–16% in RA, and adalimumab 5–12% on the label against 20–89% in the literature — with the explicit conclusion that "within the subclass of currently approved fully human antibodies ... the percentage of negligible, tolerable and marked immune responses is not clearly different from humanized antibodies", and that fully human antibodies "may have reached the limit of our ability to select for reduced immunogenicity antibodies without more directed engineering". A humanized antibody, A33, reached 66% HAHA — "a value that would be typical for a fully murine mAb".

The sharpest single counterexample is ATR-107, a fully human anti-IL-21R antibody, reported to have elicited ADA in 76% of healthy subjects in a phase 1 study. No sequence humanness score would have flagged it.

An ADA incidence is not a property of the molecule, and this is measurable. A compiled per-study table shows adalimumab — one fully human product — reported between 0.67% (Crohn's, n = 299) and 50% (psoriatic arthritis, n = 48), and infliximab between 5.4% (psoriasis, n = 168) and 61% (Crohn's, n = 125). Across the primary literature adalimumab alone carries figures of 0.67%, 12%, 28% (n = 272 over 3 years), 37% (n = 27, by SPR) and 87% (n = 15, by a non-drug-tolerant ELISA). The spread within one molecule exceeds the spread between classes. Any regression of a sequence score against "the" ADA rate is regressing against a number that depends on indication, co-medication, dosing, assay sensitivity and drug tolerance — which is a large part of why the independent R² is 0.02.

Martin KP, Grimaldi C, Grempler R, Hansel S, Kumar S. Trends in industrialization of biotherapeutics: a survey of product characteristics of 89 antibody-based biotherapeutics. mAbs. 2023;15(1):2191301. doi: 10.1080/19420862.2023.2191301

Hwang WYK, Foote J. Immunogenicity of engineered antibodies. Methods. 2005;36(1):3-10. doi: 10.1016/j.ymeth.2005.01.001

Karle A, Spindeldreher S, Kolbinger F. Secukinumab, a novel anti-IL-17A antibody, shows low immunogenicity potential in human in vitro assays comparable to other marketed biotherapeutics with low clinical immunogenicity. mAbs. 2016;8(3):536-550. doi: 10.1080/19420862.2015.1136761

Xue L, Hickling T, Song R, Nowak J, Rup B. Contribution of enhanced engagement of antigen presentation machinery to the clinical immunogenicity of a human interleukin (IL)-21 receptor-blocking therapeutic antibody. Clinical and Experimental Immunology. 2016;183(1):102-113. doi: 10.1111/cei.12711

Even the author-reported correlation may be measuring species, not immunogenicity. An independent group re-examined the same 217-therapeutic benchmark and concluded, verbatim: "Rather than measuring the ability of a model to predict immunogenicity, it indirectly measures the ability of the model to distinguish between species." Their test is a stratification: "if we consider either humanized antibodies, mouse antibodies, chimeric antibodies or human antibodies in this dataset, Hu-mAb scores show no statistically significant correlation with ADA in any group." A second independent group reports the same thing for Sapiens — no significant separation of immunogenic from non-immunogenic antibodies within the human-origin subgroup, and that "for a large subset of non-immunogenic ABs, Sapiens gives low human-ness scores". The R² in the table above is therefore best read as the score doing what its classification AUC says it does — telling murine from human — and not as evidence about immunogenicity.

Carry the n with every ADA correlation. What reads like one benchmark is reported at n = 217 (BioPhi, Hu-mAb), 216 (AbNatiV2), 211 (Humatch), 126 (AbNatiV), 109 (an MHC-II study's own set), 88 (the independent survey) and 65 (T20). These are not interchangeable, and a correlation quoted without its n and its ADA source is unusable.

The classification AUC is a property of the label set as much as the model, and this label set is unusual. The task was defined with human sequences as the positive class and sequences that were "humanized or from other species" as the negative class, over therapeutic antibodies from IMGT mAb DB. Antibodies that successfully completed clinical humanization are therefore labelled non-human. An AUC of 0.972 means the score reproduces that labelling, which is a provenance label, not an immunogenicity outcome. This is the general caveat the index describes in property predictors are graded one step removed from the bench, in a sharper form: here the label is the molecule's manufacturing history.

Fully human antibodies score low, and the paper names two. Among the 198 human-origin therapeutics, elipovimab (anti-HIV) scored 27% and navivumab (anti-influenza A) 54% OASis identity, both because long CDR3s and heavily mutated frameworks generate many non-human 9-mers. Somatic hypermutation is what an ordinary human immune response does to its own antibodies; a score that reads it as non-human is measuring distance from germline, not foreignness to a patient. By group, median OASis percentile across thresholds runs 5–7% for murine, 37–40% for humanized and 80–81% for human therapeutics — a clear separation of provenance classes with substantial overlap at the edges.

Adding MHC-II prediction made it worse. Because immunogenicity is driven by peptide display on MHC-II, the authors also scored the count of non-human 9-mers predicted to bind MHC-II by netMHCIIpan 3.1. It reduced performance and they did not pursue it, suggesting improvement would need a newer netMHCIIpan or a predictor specialised on antibody-derived epitopes. The honest reading is that bolting a generic MHC-II predictor onto a humanness score did not improve the clinical correlation. The numeric result is omitted here because the two statistics the paper reports for it do not reconcile with each other.

Prihoda D, Maamary J, Waight A, Juan V, Fayadat-Dilman L, Svozil D, Bitton DA. BioPhi: A platform for antibody design, humanization, and humanness evaluation based on natural antibody repertoires and deep learning. mAbs. 2022;14(1):2020203. doi: 10.1080/19420862.2021.2020203

Marks C, Hummer AM, Chin M, Deane CM. Humanization of antibodies using a machine learning approach on large-scale repertoire data. Bioinformatics. 2021;37(22):4041-4047. doi: 10.1093/bioinformatics/btab434

Where the residual epitopes actually are, and what that means for a framework-focused tool. Mapping CD4+ T-cell epitopes across eight humanized and fully human therapeutic antibodies — overlapping 15-mer peptides, autologous monocyte-derived dendritic cells, around 100 donors per antibody — produced a clean split. Across 188 framework-derived peptides compiled over 793 donors there was "a complete absence of prominent CD4+ T cell responses". Across 364 CDR-containing peptides, 2.5% (9 of 364) induced significant responses, distributed across all three CDRs of both chains with no single CDR position more immunogenic than the others.

Read both halves of that result, because they point in opposite directions for this tool. The same study substituted human frameworks into chimeric cetuximab while leaving its CDRs identical, and the prominent T-cell responses present in the chimeric peptide set disappeared in the humanized one. So framework humanization demonstrably removes epitopes when the frameworks are non-human — which is exactly the situation BioPhi is for, and is a point in its favour. What the result also says is that once the frameworks are human, the epitopes that remain are in the CDRs, and those are the residues Sapiens discards every mutation in by construction and that CDR grafting preserves by definition. A rising humanness score tells you the first job was done. It carries no information about the second, and the authors conclude that "immunogenicity will always be present in some antibody molecules due to the nature of the antigen-specific combining sites".

And CDR deimmunization is not a free fix. Of 202 peptide variants across five epitopes, double mutants reduced responses more reliably than single mutants — but when those double mutants were built into full-length antibodies, "only about 40% of the mutants retained full bioactivity", giving an expected yield of "approximately 11% of all variants" as effective deimmunizing changes. Nothing on this platform performs that step.

Harding FA, Stickler MM, Razo J, DuBridge RB. The immunogenicity of humanized and fully human antibodies: residual immunogenicity resides in the CDR regions. mAbs. 2010;2(3):256-265. doi: 10.4161/mabs.2.3.11641

The in-vitro surrogates, and why they are not a better answer than they look

The assays that sit between a humanness score and a clinical ADA rate — CD4 T-cell proliferation on donor PBMC panels, DC-T assays, MHC class II binding and MAPPs — are the field's standard pre-clinical immunogenicity surrogates, and they are what you should run on a humanized candidate. They are also not a validated predictor of clinical ADA, and the page states that so nobody reads "run a T-cell assay" as "then you will know".

No published study reports a correlation coefficient between an ex-vivo T-cell donor-response frequency and clinical ADA incidence. What exists instead is one sensitivity/specificity pair, from 45 monoclonal antibody homologs binned by clinical treatment-emergent ADA from FDA labels and publications and run in a CD8-depleted PBMC CD4-proliferation assay:

• Specificity is high. Every antibody that gave a strong assay response (≥40% positive donors) was one with high or intermediate clinical ADA.
• Sensitivity is about half. Verbatim: "only half of the mAbs with high clinical ADA (9 out of 17) elicited a response in this assay, suggesting the assay is not sensitive enough for a standalone assay for preclinical immunogenicity risk assessment."
• Format changes the answer for the same molecule. An anti-IL-21R homolog of ATR-107 gave 78% positive donors in the PBMC assay and 8% in the DC:CD4 assay. Two homologs of that same molecule gave 8% and 36% in the same DC:CD4 assay. Four antibodies with 62–100% clinical ADA were missed by every format tested, and adding a restimulation step did not improve prediction — it turned a non-immunogenic control positive.
• The authors' own conclusion: "there are no CD4 T cell assay formats that can predict the clinical immunogenicity of all biotherapeutics."

The MHC-II axis is weaker still. No study reports a correlation between an MHC class II binding or MAPPs readout and observed clinical ADA across a panel, computed out of sample. The three numeric attempts located are: BioPhi's own netMHCIIpan 3.1 epitope count, which reduced performance against the 217-therapeutic ADA set and was abandoned; a vendor-authored platform whose large correlation is score-against-own-prediction rather than against observed ADA; and a statistical-inference humanness paper reporting that its MHC-II-based score correlated with experimental data "even worse" than its sequence score. In one head-to-head, a MAPPs cluster count ranked adalimumab and infliximab in the inverted order relative to the T-cell assay on the same antibodies.

And the step that would close the loop has never been taken. No published study demonstrates in humans that removing a CD4+ T-cell epitope lowers clinical ADA incidence. The epitope-mapping work above named this as the outstanding requirement in 2010 — "the final step ... will be to conclusively demonstrate in humans that the removal of CD4+ T cell epitopes reduces the immunogenicity of a particular antibody ... the sine qua non will be information from clinical trials" — and industry assessments in 2017 and 2024 still describe the predictive link as unestablished.

So the honest shape of the cascade is: a humanness score is a weak species-discriminating filter, the in-vitro assays are specific but roughly half-sensitive and format-dependent, and the clinical ADA assay is the only measurement — and it is not comparable between products. Nothing in this chain lets you skip a step, and the page's go/no-go is written on that basis.

Walsh RE, Nix A, Ackaert C, Mazy A, Schockaert J, Pattyn S, Malherbe L. Preclinical immunogenicity risk assessment of biotherapeutics using CD4 T cell assays. Frontiers in Immunology. 2024;15:1406040. doi: 10.3389/fimmu.2024.1406040

Humanization against experimental humanization

No variant in the BioPhi paper was expressed or assayed. The humanization benchmark is 25 parental–humanized antibody pairs, curated by others, where the humanized partner is an antibody a human expert actually made and developed. The grading is mutation overlap: what fraction of the mutations the method proposed were also made by the expert. The authors are explicit that this is not a ground truth — "since a sequence can be successfully humanized in various ways, even across different germlines, we could not consider the experimentally validated sequence as a single ground truth" — and they close the section by naming the limit: "Ultimately, the only way to evaluate reduced immunogenicity risks and preserved binding affinity and functional activity is through in vitro or even in vivo assays," and "we understand that further experimental validation is necessary."

Author-reported, 25 pairs, read in the primary full text (Table 2). Humanness improvement is the absolute change in OASis medium identity; preservation is percent identity to the parental sequence; precision is shared mutations divided by mutations the method made:

Which row describes which platform tool. biophi_humanize runs sapiens_humanize_chain with default parameters and a single pass, so Sapiens×1 is its row — +30% humanness, 89% preservation, 77% mutation precision. The paper's better-balanced Sapiens×3 requires three iterations, and the integration does not expose an iteration count. biophi_graft runs cdr_grafting_humanize_chain with backmutate_vernier=True hardwired, so its row is Vernier CDR graft — +32%, 85% total and 100% Vernier preservation, 70% precision — not the Straight graft row. Do not quote a Straight-graft or Sapiens×3 figure for a result from this platform.

What this says and does not say. Sapiens×3 matches the experts' humanness gain (+34%) while preserving more of the parental sequence (86% against 80%), and between 72% and 77% of its mutations are ones the experts also chose. That is a real and useful result about proposal quality. It is not a result about binding: roughly a quarter of Sapiens' mutations were not made by the expert, Vernier-zone agreement is only 40–49% — and Vernier positions are precisely the ones that support the CDR loops and most often cost affinity — and the authors note that "deviation across the 25 antibodies was substantial." The averages hide per-antibody failures.

On a larger set of 152 humanized therapeutics with computationally reconstructed parental sequences, Sapiens×1 reached +30% OASis improvement at 89% preservation and 72% mutation precision, against Hu-mAb's +4% and 36%. Note that the parental sequences there are reconstructions, not known inputs, so this set grades rediscovery under an extra inference step.

An independent group reproduced the preservation half. A third group with no stake in Sapiens re-ran the same 25-pair benchmark and reported Sapiens×1 at 89% total preservation, 91% Vernier preservation and 76% expert-mutation overlap — against BioPhi's own 89% / 90% / 77%. That is a genuine independent confirmation of the preservation and overlap numbers. The same study's verdict was that its own much simpler mixture model achieved results "statistically equivalent to those of BioPhi/Sapiens", and that both "significantly outperform Hu-mAb".

⚠︎ A competing group measuring the same thing differently gets a lower number. The Humatch authors report Sapiens mutation overlap at 0.57 heavy / 0.69 light on these same 25 antibodies, the lowest or joint-lowest of the four methods they compared. Their metric is a mean per-chain exact-match overlap; BioPhi's is shared mutations divided by total predicted, pooled across chains. 77% and 0.57 are not the same quantity and must not be presented as one — but the direction flips depending on who defines the metric, which is itself the finding.

Expressed and assayed: two antibodies, three variants, both negative

This is the comparison the bench actually produces, and it now exists — barely. Two independent groups have humanized an antibody with BioPhi, expressed it, and measured binding. Both report a loss.

1. A trispecific programme chose conventional grafting over BioPhi. An anti-GPRC5D murine clone was humanized two ways in parallel: hu481 by "deep learning-based optimization ... via the BioPhi platform ... in combination with natural antibody libraries (Sapiens)", and hu482 by conventional CDR grafting. Both were expressed as scFv-hFc fusions and purified to >90%. Verbatim: "Indirect ELISA indicated that hu481 exhibited a ∼16-fold fold reduction in affinity, whereas hu482 maintained parental-level binding." The programme carried hu482 forward into its clinical trispecific and discarded the BioPhi variant. n = 1 antibody, 1 BioPhi variant; ELISA and flow cytometry only — no KD, no Tm, no expression-titre comparison.

2. An anti-CD99 scFv lost affinity through a Vernier-zone mutation. Framework-only BioPhi humanization of a previously CDR-grafted anti-CD99 scFv produced variant BP, plus a hybrid HY. All three were expressed in E. coli, purified, and measured by BLI in triplicate: KD 1.35 ± 0.59 × 10⁻⁷ M (parental) against 2.64 ± 0.79 × 10⁻⁷ M (BioPhi) and 3.95 ± 1.14 × 10⁻⁷ M (hybrid) — about 2.0-fold and 2.9-fold worse. ELISA and flow cytometry agreed. Association rates were comparable; the loss came from faster dissociation. The authors' conclusion: the work "underscores the limitations of BioPhi in optimizing HuScFvMT99/3".

★ The mechanism in that second paper is the single most actionable finding on this page. Verbatim: "Although BioPhi focuses on framework mutation, the modification could be introduced in the vernier zone. The phenylalanine 27 (F27) and asparagine 28 (N28) positions were identified as vernier zones in the VH domain ... Modification of residues in antibody vernier zone can lead to alteration of CDRs loop structure, leading to impaired binding activity." Framework-only humanization is not binding-site-safe, because the Vernier positions are framework positions that support the CDR loops. This is the same place the 25-pair benchmark shows Sapiens agreeing with experts only 40–49% of the time. Two independent lines of evidence point at the same residues.

Read the limits of this honestly. n = 2 antibodies and 3 variants, two different formats (scFv-hFc and scFv), one ELISA-only and one with BLI, no thermostability measured in either, no expression-titre series, no in vivo work, and no immunogenicity measurement of any kind on a BioPhi design. The second paper also notes its BioPhi variant retained "a nanomolar range of binding affinity, which is comparable to the previously reported potent anti-CD99 antibodies" — a 2-fold loss is not a failure in every programme. The direction of the published evidence is unfavourable and the sample is far too small to generalise. What it does settle is that the comparison is no longer absent, and that the first two attempts both went the wrong way.

Li J, Lu Q, Zhu Z, Wang X, Chen Y, Zhang Z, Yu Z, Wang Z, Chen Y, Xie H, Yang N, Lu H, Niu T, Nie C, Tong A, Li H. YMN-V115: a novel humanized BCMA/GPRC5D/CD3 trispecific antibody in relapsed/refractory multiple myeloma. Journal for ImmunoTherapy of Cancer. 2026;14(2):e013986. doi: 10.1136/jitc-2025-013986

Sornsuwan K, Pamonsupornwichit T, Juntit OA, Thongkum W, Takheaw N, Kodchakorn K, Tayapiwatana C. Plasticity of BioPhi-driven humanness optimization in ScFv-CD99 binding affinity validated through AlphaFold, HADDOCK, and MD simulations. Computational and Structural Biotechnology Journal. 2025;27:369-382. doi: 10.1016/j.csbj.2025.01.001

Parkinson J, Wang W. For antibody sequence generative modeling, mixture models may be all you need. Bioinformatics. 2024;40(5):btae278. doi: 10.1093/bioinformatics/btae278

What the figure doesn't cover

• Humanness is not a validated predictor of immunogenicity in a patient, and no figure on this page makes it one. R² = 0.26 at the platform's threshold and 0.34 for the best published method, against reported ADA across 217 therapeutics — and R² = 0.02, not statistically significant, in the one fully independent measurement on 88 marketed products with package-insert ADA. Those reports come from different trials, indications, dosing regimens and assay formats; one fully human product spans 0.67% to 50% across indications.
• The correlation that does exist may be a species proxy. Stratify the 217-therapeutic set by origin and the ADA correlation vanishes inside every subgroup — murine, chimeric, humanized and human alike. Two independent groups report this. On repertoire data the same pattern appears: OASis separates human from mouse at 0.897 PR-AUC and human from rhesus at 0.570, where chance is 0.5. These scores are good at telling rodent from human and weak at everything finer, which is not the distinction a humanization programme needs. Humanized and even fully human antibodies carry residual immunological risk — the CDR-epitope mapping above is the mechanism — and the BioPhi paper's own data shows fully human therapeutics scoring as low as 27% OASis identity. Treat a low score as a reason to look harder, never as a prediction, and treat a high score as no assurance at all.
• A humanness improvement is a framework improvement, and the residual epitopes are in the CDRs. Framework humanization does remove T-cell epitopes when the frameworks are non-human — that is measured, and it is the job BioPhi is for. But in antibodies whose frameworks are already human, the epitopes that remain sit in CDR-containing peptides (2.5% of 364 tested, against none of 188 framework peptides over 793 donors), and both platform modes leave CDRs untouched by construction. A score that rises after humanization reports the first job, and is silent on the second.
• The classification label is provenance, not outcome. The 0.972 AUC separates sequences labelled "human" from sequences labelled "humanized or other species". Humanized clinical antibodies sit in the negative class. A score that perfectly reproduced that labelling would tell you how a molecule was made, not how a patient will respond to it.
• The score reads distance from germline, and somatic hypermutation is not foreignness. Long CDR3s and mutated frameworks drive the score down whether or not they are immunogenic. For a target that requires an unusual CDR3, expect a low score that means little.
• The humanness_class cutoffs disagree with the paper's own distribution — known issue. The backend classifies on the percentile with fixed thresholds: ≥ 90 → human, ≥ 70 → humanized, ≥ 40 → chimeric, else murine. The BioPhi paper measured the median OASis percentile of real therapeutics at 80–81% for human, 37–40% for humanized and 5–7% for murine antibodies. Put those together and the median fully human therapeutic is classified humanized by this platform, while the median humanized therapeutic lands at or below the chimeric/murine boundary. The classification is therefore systematically stricter than the reference distribution it is nominally built on, and the schema's documented oasis_percentile_threshold default of 80.0 matches neither the cutoff actually applied (90) nor any documented value, and it is ignored. Read the oasis_percentile number, not the class label.
• immunogenicity_risk is not a published metric — known issue. The low/moderate/high field is computed on this platform as humanness_score >= 0.85 → low, >= 0.6 → moderate, else high. Those cutoffs appear in no publication and are not calibrated against any ADA dataset. The field is surfaced as a headline (tier-1) value on the result card, above the provenance flags. Read the underlying humanness_score and oasis_percentile instead. A fix to re-label or de-emphasise this field is not yet implemented.
• immunogenicity_reduction_percent is always null — known issue. The output schema declares the field and the backend sets it to None unconditionally; no code path populates it. What is measured during a humanization run, when real OASis is available, is oasis_percentile_before, oasis_percentile_after and oasis_percentile_improvement. Use the percentile delta. The outputs page currently describes this field as an "estimated immunogenicity reduction", which overstates it; that description is a known documentation defect pending correction.
• Humanness can silently be a different statistic — check is_real_oasis. Real OASis needs the ~22 GB OASis_9mers_v1.db on a Modal volume. If it is missing, the job returns SUCCESS with method: "composition_heuristic", is_real_oasis: false, degraded: true, fallback_reason: "oasis_path_not_wired", and the score is a cosine similarity between the sequence's amino-acid composition and a fixed human frequency table — no 9-mer lookup and no OAS data at all. Every figure on this page describes real OASis and none of them describes the fallback, for which no published validation exists. The card renders the degraded case as "Humanness (est.)" in a warning colour with warning severity, never success, and the is_real_oasis and degraded flags are surfaced. The platform is honest about the substitution; you have to read the flag. One blind spot: result shapes that predate this disclosure carry neither flag, and the card treats an absent flag as real. A result from before the flags existed can therefore render as a real OASis score, so date an old result before trusting it.
• framework_humanness and cdr_humanness are not region measurements — known issue. On the real OASis path, framework_humanness is a copy of the overall score and cdr_humanness is not emitted at all. On the fallback path, framework_humanness is the overall score multiplied by a constant 1.05 and cdr_humanness comes from fixed sequence slices with no antibody numbering applied. Do not read the per-region breakdown described on the outputs page as a per-region score. BioPhi's own web interface does produce a genuine per-position and per-region humanness breakdown; that capability is not exposed through this integration.
• Three input parameters are accepted and ignored — known issue. target_humanness, oasis_percentile_threshold and preserve_binding_residues are all in the tool schemas and none is read by the Modal app's compute path. In particular, biophi_humanize is described as humanizing "while preserving binding residues", and no residue is protected: Sapiens runs with default parameters, preserving CDRs by definition (mutations in CDRs are discarded) but nothing else. biophi_graft always applies Vernier back-mutation, which is hardwired on and not exposed. The humanness threshold likewise has no effect — the classification cutoffs are fixed in the backend. Treat these as known issues; the fix is to either wire them through or remove them from the schema.
• "OASis germline identity analysis" is a mis-description — known issue. The biophi_humanness tool description calls OASis a germline identity analysis. It is not: OASis is 9-mer peptide prevalence across human repertoires, and germline content is the separate baseline metric that OASis was compared against (and did not beat). The two are different measurements.
• biophi_graft takes no donor antibody — known issue. Its description reads "graft CDR loops from a donor antibody onto a human germline framework", but the backend grafts the input sequence's own CDRs onto its nearest human germline framework. There is no second, donor sequence input. The description should read as humanizing the submitted antibody.
• The stated durations for humanize and graft are unmeasured. Both carry estimatedDuration: "30-60 seconds". Only the humanness tool has a measured range (~136–162 s, recorded in a dated comment in its definition after the original estimate under-sized the staleness budget and swept a live job). Treat the humanize and graft estimates as unverified.
• The humanness number is a percentile, not a fraction of human 9-mers. humanness_score here is the OASis percentile divided by 100, where the percentile is ranked against 544 therapeutic mAbs from IMGT mAb DB, including preclinical and terminated programmes. A score of 0.80 means "as human as the 80th-percentile clinical-stage antibody", not "80% of my 9-mers are human". The reference set is a historical snapshot of industry practice, so the scale moves if the reference moves.
• The humanized sequence is not on the card. humanized_vh and humanized_vl are in the hide list for the result card's field surfacing, so the sequences come back in the job result but are not rendered as card fields, contrary to what the quickstart and outputs pages describe.
• Version is unpinned. The Modal image installs biophi from a git HEAD with no tag or commit pin, so the running version is unspecified and can change between deploys. Sapiens weights are the published v1 models. A humanness figure quoted without its BioPhi version and prevalence threshold is not reproducible.
• Independent evaluation exists for the method, but none for this integration. OASis and Sapiens have been graded by four competing groups (see against other in silico methods), and one of those re-derivations corroborates the relaxed-threshold classification figure. Whether the deployed Modal app reproduces any of it depends on the OASis database being present and on which unpinned BioPhi commit is installed, and no published study has graded that.
• The score is close to blind to mixed-gene frameworks. Because OASis scores 9-mers, a sequence assembled from two different human V-genes reads as human: it flagged only 1% of artificial mixed-gene designs as non-human, against 51% for the best method tested. If your pipeline can emit chimaeric frameworks, this score will not catch them.
• Framework-only humanization is not binding-site-safe. The Vernier positions sit in the framework and support the CDR loops. In the one published case where the affinity loss was traced, BioPhi had mutated VH F27 and N28 — both Vernier positions — and the authors attributed the impaired binding to exactly that. Sapiens agrees with expert choices at Vernier positions only 40–49% of the time. Inspect every proposed mutation against a Vernier-position list before you express anything.
• Nothing here measures developability. Humanization moves charge, hydrophobicity and aggregation propensity, and the score is blind to all of it. Run antibody developability on the humanized candidate.

Go / no-go

Go — act on the result when:

• You are ordering or triaging a candidate list on humanness, and you want the relative rank. That is what the classification figures support.
• You want Sapiens or a CDR graft to propose humanized sequences that you will then express and assay, alongside a conventional CDR graft as a control. 73–77% agreement with expert choices is a good starting pool, mutations_made is a genuine parental-to-humanized diff, and in the one published head-to-head the conventional graft was the variant that kept its affinity — so run both.
• You will inspect every proposed mutation against a Vernier-position list before expressing. This is the step that would have caught both published affinity losses.
• You cross-check the humanized sequence with a metric that did not generate it — Humatch is the second scorer here — because a score used to optimise a sequence cannot also grade it. That is the one lesson on this page with a measured demonstration behind it.
• You have confirmed the humanness percentile moved in the right direction across a humanization run, using oasis_percentile_before and oasis_percentile_after, which are real measurements when OASis is wired.
• You want the 9-mer-level localisation that is OASis's actual advantage over the better-correlating Hu-mAb — in which case use BioPhi's own web interface or CLI, not this integration, which surfaces no per-position or per-region breakdown (see known issues).
• You have confirmed is_real_oasis: true on the result, and you quote the prevalence threshold with the number.

No-go — do not act on the result when:

• You are predicting whether a candidate will provoke ADA in patients, or comparing two candidates' clinical immunogenicity risk. R² = 0.26 does not support either, and immunogenicity_risk is an uncalibrated platform-side binning of that same score.
• You are treating a high humanness score as clearance. Fully human antibodies are immunogenic, and the paper's own set contains human-origin therapeutics at 27% OASis identity.
• You intend to ship a humanized sequence without expressing and assaying it. Both published wet-lab tests of BioPhi output lost affinity — one by ~16-fold against a conventional graft that lost none — and Vernier-position agreement with expert humanization is only 40–49%.
• You are humanizing away from a rodent parent, for instance from a primate or a non-germline human lineage. OASis separates human from rhesus at 0.570 PR-AUC, against 0.5 for chance.
• You need to know that a humanized sequence has not been assembled from two different human V-genes. OASis flagged 1% of such designs.
• You are relying on preserve_binding_residues, target_humanness or oasis_percentile_threshold to have shaped the result. None of them is read.
• You are reading framework_humanness, cdr_humanness or immunogenicity_reduction_percent as measurements. The first two are not region scores and the third is always null.
• You are acting on the humanness_class label. Its cutoffs classify the median human therapeutic as humanized. Use oasis_percentile.
• The result is flagged degraded: true or is_real_oasis: false and you need a humanness number rather than a placeholder. Nothing on this page validates the composition heuristic.
• You need an immunogenicity risk assessment for a regulatory filing. That requires the assay cascade — MHC-II and T-cell assays, then clinical ADA monitoring — and no sequence score substitutes for it.

Caveats on reading this page: every figure above is quoted from the cited source with its original metric, threshold and sample, and was read in the primary full text. The one abstract-only source is the ATR-107 paper, marked as such, from which only the 76% incidence is taken. Each figure is labelled author-reported (the tool's own authors), competitor-authored (independent of BioPhi, not disinterested) or independent; the distinction matters more here than anywhere else on these pages, because every favourable ADA correlation is author-reported and the one fully independent measurement is null. Figures from different rows are not comparable — label sets, prevalence thresholds, ADA curations, metric definitions and sample sizes all differ, and an ADA correlation quoted without its n is unusable. Where a measurement does not exist — thermostability or expression titre for any BioPhi-humanized variant, a correlation between a T-cell assay donor-response frequency and clinical ADA, an MHC-II readout against observed ADA out of sample, or a human demonstration that removing a T-cell epitope lowers ADA — this page says so rather than substituting a figure from a different method, a different tool or a different antibody.