How Humatch results compare to the bench

You have a Humatch result. Here is what it supports.

• Humatch publishes no correlation coefficient against clinical immunogenicity at all. Its anti-drug antibody (ADA) analysis is a three-bin separation over 211 therapeutics, not a regression, and its own paper claims only that rival scorers "offer similar separation." So Humatch cannot be placed above or below the R² = 0.34 ceiling that BioPhi's page establishes for this class — there is no comparable number to place. The only published coefficient for the same statistic Humatch uses is its predecessor's: R² = 0.31.
• Its headline accuracies are V-gene family assignments, not immunogenicity predictions. Humatch reports per-gene classifiers at ROC AUC 1.000 for almost every class — separating human repertoire sequences from mouse and rhesus repertoire sequences. That is a species-and-germline label. The index explains the general form of this caveat in property predictors are graded one step removed from the bench; here the label is the sequence's species of origin.
• ★ One Humatch-humanized antibody has been expressed and fully assayed, by an independent group, and the result is split. Affinity fell from KD 0.47 nM (chimeric parent) to 20.6 nM by BLI — roughly 44-fold — with a ~24-fold rise in competitive-ELISA IC50. Thermostability improved: the humanized variant showed an apo Tm of 80.4 °C and none of the parent's low-temperature instability. The programme carried the chimeric variant into its in vivo work. This is more wet-lab evidence than exists for any BioPhi design, and it is still n = 1.
• ⊘ The paired CNN — Humatch's one distinctive contribution — is not running on this platform. The Modal app loads the heavy and light models only and scores one chain per call. Eight platform surfaces nonetheless label the output "Joint VH/VL humanness," and vl_sequence is silently dropped at the default chain_type: "H". See known issues.
• ⊘ humanness_score is not "probability this is human." It is the largest softmax probability over a single human V-gene family, with the non-human class zeroed out of the argmax and filtered out of the result. Heavy chains are scored over 7 classes and light chains over 17, so a VH and a VL number are not on the same scale.
• The authors state the limit themselves: "Currently, Humatch, and all other computational humanization tools lack experimental validation."
• Confirm immunogenicity with ADA assays in a human trial, de-risked beforehand with T-cell proliferation / PBMC assays, MHC class II binding and DC-T (EpiScreen-type) assays. BioPhi's page covers how weak that surrogate cascade is, and that analysis applies here unchanged. Confirm any humanized variant by expressing it and measuring binding, titre and thermal unfolding against the parent.

Jump to agreement with wet lab or against other in silico methods. The general framing for every tool is in reading an in silico result against the bench.

This page is the sibling of BioPhi's. The two tools answer overlapping questions, so the clinical-endpoint analysis, the independent R² = 0.02 survey, the CDR-epitope mapping and the in-vitro surrogate figures are all stated there and are not repeated here. This page covers what is specific to Humatch, and says where it differs.

How in silico is used currently

Humatch covers three jobs with three different evidential standings, and the platform exposes only the first.

• Humanness scoring as a triage filter. A score orders candidates and localises the problem — which framework positions look least like the closest human germline — so a campaign knows where to intervene. This is the use the figures support, and humatch_score is the only Humatch mode on this platform.
• Germline-targeted humanization as a sequence-proposal step. Humatch's own emphasis is humanization: iterative single-point mutation guided by its classifiers toward one chosen V gene, in about 33 seconds per antibody. Those sequences then enter the normal funnel — express, measure binding, measure thermostability, pick. None of this is available here; BioPhi is the humanization route on this platform.
• VH/VL pairing assessment. Humatch is the first humanization tool to score heavy and light chains jointly, on the argument that separate optimisation "risks disrupting expression, lowering stability, and creating immunogenic epitopes that span both chains." It is the paper's distinctive claim and the reason to prefer it over Hu-mAb or Sapiens. It is also not running here (see known issues).

Humatch is also used in practice as a pass/fail gate on generated sequences — a 2026 light-chain generative model used it to filter its outputs, where it called 99.73% of generated light chains human. Read that as a ceiling effect, not a validation: a filter that passes everything has not been shown to discriminate. ⚠︎ That model is from the same Oxford group as Humatch, so it is not an independent use.

Capel HL, Ellmen I, Murray CJ, Mignone G, Black M, Clarke B, Breen C, Tierney S, Dougan P, Buick RJ, Greenshields-Watson A, Deane CM. LICHEN enables light-chain immunoglobulin sequence generation conditioned on the heavy chain and experimental needs. Communications Biology. 2026;9(1):468. doi: 10.1038/s42003-026-09727-3

What no figure located for this page supports is a humanness score used as a decision instrument about a patient's immune response. Humatch publishes no coefficient for that relationship, and the independent reading of the benchmark it uses is that the benchmark measures species.

Against other in silico methods

On its own humanization benchmark, Humatch leads — and the comparison is author-run. The Table 2 head-to-head above is the full result: Humatch at 0.77 / 0.82 expert-mutation overlap against Hu-mAb 0.67 / 0.77, Sapiens 0.57 / 0.69 and AbNatiV 0.57 / 0.65. Three qualifications belong with it.

• Hu-mAb is the same laboratory's previous tool. Both papers are from the Oxford group, with a shared senior author. A method beating its own lab's predecessor is a self-comparison, and this page labels it as one.
• Hu-mAb's numbers were taken from Hu-mAb's paper; Sapiens and AbNatiV were run by Humatch's authors with "default and paper-recommended parameters." Two of the three competitors were operated by the group they were being compared against.
• The metric is not the one BioPhi reports. Humatch's 0.57 for Sapiens is a mean per-chain exact-match overlap; BioPhi reports Sapiens at 77% mutation precision, pooled across chains and divided by mutations the method made. The sibling page flags this — 0.57 and 77% are not the same quantity, and the ranking flips depending on who defines the metric. Neither number is wrong. Quoting either without its definition is.

On mixed-gene rejection, Humatch is genuinely differentiated. Sequences that sit "between" V genes are not observed naturally and are argued to carry immunogenic risk. Constructing artificial mixed-gene designs, the authors report that Humatch ranks 51% of them as non-human, against AbNatiV 28%, Hu-mAb 8% and OASis 1% (stated in the body; the analysis is in the paper's SI Section 15). That is a large margin on a real design pathology, and it is the best argument for using Humatch alongside OASis rather than instead of it. ⚠︎ It is also graded against constructed negatives — the label is how the sequence was built, not an immunogenicity measurement — and 51% means Humatch misses half of its own artificial mixed-gene set.

On the pairing task, the one independent benchmark puts plain germline information ahead of Humatch. The 2025 germline-aware benchmark evaluated three published pairing models — p-IgGen, Humatch and ImmunoMatch — across random, V-gene and germline splits against a Topline and Bottomline reference. Its findings, quoted as directions because the numbers are in a figure and a supplementary table (see the review note above):

• "Humatch and ImmunoMatch ... perform consistently across splits, achieving strong AUC-ROC scores, particularly on v-gene, but fall short of Topline performance in F1 Score and Accuracy, highlighting limitations in binary classification calibration."
• On the external PairedAbNGS dataset: "germline-based models (V and VDJ) consistently achieved the highest accuracy, closely followed by Humatch. These findings confirm the robustness of germline information as a key determinant of pairing compatibility."

That is the germline-baseline result for Humatch's distinctive claim, and it reads the same way the germline-content baseline reads for OASis: a plain germline model matches or beats the deep-learning method on the classification task, and the deep-learning method's advantage is elsewhere. Humatch's "elsewhere" is the thermostability correlation reported under agreement with wet lab, where it does beat the germline baseline — r = 0.25 against 0.16 — with both effects small.

A 2025 diffusion-based humanization model independently reproduced the shape of Humatch's ADA result — stratifying therapeutics by score and finding high-ADA antibodies concentrated in the low-score bin, noting that "the trends observed in our model outputs mirror those reported in previous investigations about ADA evaluation, including Hu-mAb and Humatch." It reports no Humatch figure, so it corroborates the direction and adds no number.

Liu D, Hao X, Fan L. HuAbDiffusion: a discrete language diffusion model used for antibody humanization. Briefings in Bioinformatics. 2025;26(6):bbaf658. doi: 10.1093/bib/bbaf658

In silico vs. wet lab

There are two comparators, because Humatch makes two kinds of claim.

For humanness and immunogenicity, the wet-lab standard is ADA incidence measured in human trials — a tiered screening, confirmatory and titre/neutralising cascade on patient samples, which comes with the trial rather than being purchasable per candidate. The earlier in-vitro surrogates are T-cell proliferation / PBMC assays, MHC class II binding assays and DC-T (EpiScreen-type) assays.

For the pairing claim, the comparators are unusually concrete and were actually measured: Fab melting temperature by differential scanning fluorimetry and expression titre in HEK cells, both available for 137 clinical-stage antibodies from Jain et al.

For humanization, the comparator is an experimental humanization campaign — graft, back-mutation scan, then expression and assay of each variant for retained affinity, titre and thermostability. The agreement metric there is not a score; it is whether the variant still binds and still folds.

None of those appears in the shared cost basis, which covers crystallography, SPR, hERG, Ames and microsomal stability. This page therefore quotes no wet-lab cost figure at all rather than deriving one. Read what the wet-lab cost figures mean for why the shared table is the only sanctioned basis.

Read the accuracy column by cell, never down the column. An ADA incidence from a trial, a measured KD, a Tm in °C, a donor response frequency, and a multiclass V-gene F1 are five different kinds of number. What the table supports is comparing time and cost at a stated accuracy.

On that reading, the honest position for this tool is: seconds instead of weeks, for a germline-family assignment — and the assignment has never been regressed against the clinical question at all. Humatch is a fast, well-characterised classifier of what species and which V gene a sequence resembles. That is a real and useful thing to know. It is not an immunogenicity forecast, and unlike BioPhi it does not even come with a weak one.

The runtime this price is derived from is shorter than the runtime the tool actually needs. The 1 credit price assumes a runtime of about 52 seconds. The tool definition states estimatedDuration: "2-3 minutes", and a dated comment immediately above it records the measured figure as "the real ~132s runtime" after an earlier "10-30 seconds" estimate caused live jobs to be swept as stale. Two older catalog rows still carry that superseded "10-30 seconds" / "10-60s" text. Where they conflict, trust the ~132 s comment: it is the only one of the five written as a measurement rather than an estimate, and it is dated. The padded 52 s the price is derived from is less than half of it.

⚠︎ And the Modal container timeout is 120 seconds, shorter than the ~132 s the tool records as real. A job that genuinely takes the recorded runtime is killed by its own container. Treat this as a known issue.

Two asymmetries matter. A clinical ADA rate is not a property of the molecule alone — it moves with indication, dosing, co-medication and assay, which is why the same antibody is reported at different incidences in different trials. And a sequence can become more germline-like while losing affinity; the score cannot see that, because binding is not in it. The one published expression experiment on a Humatch design is exactly that case.

Agreement with wet lab

This section is in four parts, because Humatch's outputs are graded against four different things and none of the gradings is a bench measurement of immunogenicity.

What the classifier's labels actually are

Humatch is three convolutional networks. CNN-H and CNN-L are multiclass: each outputs a probability vector over one non-human class plus the human V-gene families (HV1–7 for heavy; LV1–10 and KV1–7 for light). CNN-P is binary over "true" and "fake" pairs. All three were trained on the Observed Antibody Space repertoire database, not on therapeutics.

This is a materially different label set from BioPhi's, and the difference runs in Humatch's favour on one axis and against it on another. BioPhi's and Hu-mAb's classification task puts therapeutic antibodies that successfully completed clinical humanization into the negative class, so their AUC partly reproduces a manufacturing provenance label — the sibling page states that caveat in full. Humatch's unpaired negatives are not humanized therapeutics; they are genuine non-human repertoire sequences. Its classification is therefore a cleaner species question. But that is also the problem: a cleaner species classifier is further from, not closer to, an immunogenicity prediction. Nothing in the human-versus-mouse-and-rhesus label touches the clinical endpoint.

CNN-P's negative class has the sharper label problem, and the authors say so. The artificial bad pairs were built by joining chains from different B-cell classes and different studies, which, verbatim, "means that the chains are unlikely, but not guaranteed, to match poorly." The label is co-occurrence provenance, not a measured pairing failure. The authors add that "a lack of ground truth 'badly' paired sequence data for training CNN-P could result in both false positive and negative predictions," and the independent benchmark below makes the same point about its own evaluation — "we cannot test the data on the negative (experimentally validated unpaired) sequences." No dataset of experimentally confirmed badly-paired antibodies exists. Every pairing figure anywhere in this literature is graded against synthetic negatives.

Author-reported classification performance, read in the primary full text (Table 1). Two rows are worth carrying; the rest are 0.99+:

Read the PR AUC and the F1, not the ROC AUC. ROC AUC is 1.000 for every V-gene class including KV7, whose F1 is 0.587 — a class the model identifies barely better than half the time by harmonic mean of precision and recall. Under the extreme class imbalance of a 24-way repertoire problem, ROC AUC saturates and stops carrying information. A Humatch figure quoted as "ROC AUC 1.000" is not wrong and is not useful. The authors are candid about the cause: KV7 and LV9 are data-sparse, and "gene classes with the lowest scores tend to have little training data available."

⚠︎ Humatch was trained on complete variable domains and requires them. Sequences lacking cysteines at IMGT 23 and 104, or missing the first or last position, were removed from training, and the authors state that "users should only input complete sequences for reliable results." The platform's only length gate is a 90–130 residue window and a dominant-residue check; it does not verify the conserved cysteines. A truncated or mis-trimmed domain can score, and the score means nothing.

Chinery L, Jeliazkov JR, Deane CM. Humatch - fast, gene-specific joint humanisation of antibody heavy and light chains. mAbs. 2024;16(1):2434121. doi: 10.1080/19420862.2024.2434121

Humanness against clinical immunogenicity

Humatch reports no correlation coefficient, no R², and no regression against ADA. What it reports is a binning. Its authors took the Marks et al. set of 217 therapeutics with recorded ADA levels, found sequences for 211, scored each with all three CNNs, and took the minimum of the three scores — "reasoning that if one of the CNNs ranked a therapeutic poorly, then it should be classed as higher risk." The result, verbatim:

Using a loose Humatch cutoff of 0.01 to retain most low ADA therapeutics removed 89% of high ADA (red), 59% of medium ADA (yellow), and just 23% of low ADA (green) therapeutics.

And immediately after, the comparison: "Alternative humanness classifiers, including Hu-mAb, AbNatiV, and OASis, offer similar separation of low and high ADA antibodies." Humatch's own paper therefore claims parity with the methods on the R² = 0.16–0.34 band, not an improvement on them, and declines to publish the coefficient that would let you check.

That matters because the minimum-of-scores construction has a published coefficient — from the same laboratory's previous tool. Hu-mAb's paper, read in its primary full text, assessed "the correlation between the percentage of patients who develop ADAs and the minimum humanness score of a therapeutic's VH and VL chains, as the least human chain is expected to dictate the level of immunogenicity, across 217 therapeutics," and found: "the correlation was weak with an R² of 0.31." Humatch uses the identical statistic on the identical dataset, extended from two chains to three CNNs, and reports a bar chart instead. A page that quoted "Humatch correlates with ADA" without a number would be reporting Hu-mAb's 0.31 by implication. This page does not do that: the honest statement is that Humatch's position against the 0.34 ceiling is unmeasured in its own publication, and the two nearest anchors are its predecessor's 0.31 and BioPhi's recomputation of Hu-mAb at 0.34.

An independent group argues the whole benchmark measures species. Re-examining the same Marks et al. set, they concluded, verbatim: "Rather than measuring the ability of a model to predict immunogenicity, it indirectly measures the ability of the model to distinguish between species." Their test is a stratification: "if we consider either humanized antibodies, mouse antibodies, chimeric antibodies or human antibodies in this dataset, Hu-mAb scores show no statistically significant correlation with ADA in any group." And the mechanism: "a model that assigns low scores to mouse or chimeric antibodies will correlate with ADA even though it cannot predict whether a mouse or a humanized antibody is more or less likely than other mouse or humanized antibodies to be immunogenic." The same group reports that "SAM, BioPhi, AbNativ, and Hu-mAb achieve statistically equivalent results for correlation with ADA."

⚠︎ That critique predates Humatch and does not name it. It is a critique of the dataset and the framing, published in May 2024 against a benchmark Humatch adopted in November 2024 — the same 217-therapeutic set, the same minimum-of-scores logic, and a model whose explicit negative class is non-human species. It transfers by dataset identity, not by citation, and this page says so rather than claiming the authors tested Humatch.

Humatch's own data makes the same point from the other direction. Its classifiers rank some antibodies that human experts humanized and regulators approved as non-human, and the paper notes the predecessor's version of this: "Marks et al. find Hu-mAb ranks only ~66% of experimentally humanized VH/VL pairs as human" — a figure confirmed in Hu-mAb's primary text as "two-thirds of the humanized therapeutics had both VH and VL classified as human." A score that rejects a third of the molecules that passed the clinic is not reading clinical risk.

For the full picture of what is known about humanness against ADA — the independent 88-product survey at R² = 0.02, the non-monotonic per-class incidences, the fully human antibody at 76% ADA, and the within-molecule spread that exceeds the between-class spread — read BioPhi's agreement section. Every one of those findings constrains Humatch exactly as it constrains OASis, and none is restated here.

Marks C, Hummer AM, Chin M, Deane CM. Humanization of antibodies using a machine learning approach on large-scale repertoire data. Bioinformatics. 2021;37(22):4041-4047. doi: 10.1093/bioinformatics/btab434

Parkinson J, Wang W. For antibody sequence generative modeling, mixture models may be all you need. Bioinformatics. 2024;40(5):btae278. doi: 10.1093/bioinformatics/btae278

The paired model against something measured

This is the one place Humatch connects to a bench measurement, and the connection is thin in its own paper and small when measured independently.

Humatch's argument for CNN-P is that joint optimisation protects expression and stability. The evidence offered for it is one group comparison, stated in the body and computed in the supplement: using 137 therapeutics with melting temperature data from Jain et al., "those with high CNN-P predictions (above 0.5) had higher melting temperatures than those with lower predictions (p < 0.001)." The discussion restates it as "high CNN-P predictions are shown to correlate with higher melting temperatures."

Three things about that sentence. It gives a p-value and no effect size, so the body reports no correlation coefficient and no ΔTm. It splits at 0.5, while the threshold Humatch recommends and uses for humanization is 0.95 — the validated split point is not the operating point. And the authors note they "could only find stability data for 13 of the 20 experimental endpoints predicted to pair well and none for the five predicted to pair less well," so the subgroup that would falsify the claim had no data at all.

An independent group has since measured the effect size, on the same 137 antibodies. A 2025 germline-aware VH–VL pairing benchmark computed Pearson's r and Spearman's ρ between each model's pairing score and the Jain et al. measurements, for HEK expression titre and Fab melting temperature. Its Table 3, read in the primary text, with bold marking p < 0.05:

Humatch has the strongest thermostability association of any model tested, and it is r = 0.25 — about 6% of the variance. The authors' verdict: "The observed correlations with developability properties are modest. Humatch shows the strongest association with thermostability, while p-IgGen displays the clearest trend with expression. However, the effect sizes remain small, indicating that current AI models struggle to capture developability signals from VH–VL pairing alone." They flag the n = 137 limit themselves.

On expression — the other property the paired model is argued to protect — Humatch's correlation is −0.15 Pearson and 0.01 Spearman, neither significant. The direction is wrong and the magnitude is nil. Nothing published supports reading a pairing score as an expression forecast.

Jain T, Sun T, Durand S, Hall A, Houston NR, Nett JH, et al. Biophysical properties of the clinical-stage antibody landscape. PNAS. 2017;114(5):944-949. doi: 10.1073/pnas.1616408114

Joubbi S, D'Arco E, Maccari G, Milazzo P, Micheli A. Germline-aware deep learning models and benchmarks for predicting antibody VH-VL pairing. mAbs. 2025;17(1):2570749. doi: 10.1080/19420862.2025.2570749

Humanization against experimental humanization

No variant in the Humatch paper was expressed or assayed, and the authors say so without hedging:

Currently, Humatch, and all other computational humanization tools lack experimental validation. Unfortunately, only the experimental end-points are known to be sufficiently safe to administer to patients. Our designs, and those from Hu-mAb, AbNatiV, and Sapiens are only predicted to be safe. However, validating their true immunogenic risks is complex and expensive. Instead, future efforts may first focus on testing all designs' affinity and melting temperatures to ensure that suggested mutations do not disrupt binding or affect stability.

The benchmark is the same 25 precursor / experimentally-humanized pairs that BioPhi used, curated by the Hu-mAb authors. The grading is mutation overlap: what fraction of the mutations the method proposed were also made by the human expert. Author-reported, read in the primary full text (Table 2), where overlap of 1.00 would mean every proposed mutation was also made experimentally:

This is a real result about proposal quality and nothing more. Humatch's overlap is the highest of the four while making the second-largest number of mutations, which the authors reasonably read as evidence that it "does not simply find the 'easy' mutations." The number of mutations it proposes tracks the number the experts made at Pearson 0.89. On a throughput test over 1,000 non-human paired OAS sequences, 12 pairs failed to reach target within the 60-edit cap and the mean was 36 s per pair on 16 CPUs.

⚠︎ Humatch mutates Vernier and interface positions by design, at the same rate the experts did. Its mutation profile puts 19.3% of heavy and 12.6% of light mutations inside Vernier and interface zones, against 17.9% and 14.6% experimentally. The authors offer this as a match, and as a profile match it is one. But a proportion match is not a position match, and Vernier positions are the framework residues that support the CDR loops and most often cost affinity. BioPhi's page documents two independent lines of evidence that this is where framework humanization loses binding — a Vernier mutation at VH F27/N28 costing an anti-CD99 scFv its affinity, and Vernier-zone agreement with experts of only 40–49% on these same 25 antibodies. The one Humatch design that has been expressed lost 44-fold affinity, and its authors attributed that to framework residues adjacent to the CDRs. Three independent lines now point at the same residues. Humatch lets users fix Vernier and interface positions; it does not do so by default.

The one germline-only baseline the paper runs is graded on Humatch's own scores. The authors ablated the CNN guidance, making only the top germline mutations with the same edit counts, and report that "no baseline-designed paired sequences matched Humatch's designs exactly and only two designs passed all three CNN target thresholds of 0.95." That grades the germline baseline against Humatch's own classifiers, not against the experimental-overlap metric on which Humatch's headline 0.77 / 0.82 is reported. It is the same self-grading trap BioPhi's authors identified when Hu-mAb-optimized sequences were scored with Hu-mAb. Since Humatch's humanization begins with a germline-frequency lookup — mutating to a target germline-likeness of 0.40 before any CNN is consulted, and the paper notes "the majority of these mutations were to common germline residues" — how much of the 0.77 / 0.82 overlap a plain germline lookup would have achieved on its own is not reported. That is precisely the question the germline-content baseline answered for OASis, where it was not beaten.

Expressed and assayed: one antibody, one variant, split result

This is the comparison the bench actually produces, and for Humatch it exists — once. An independent group with no stake in the method humanized a murine anti-xylazine antibody with Humatch, expressed it in Expi293 cells, purified it, and characterised it by BLI, competitive ELISA, DSF and a cross-reactivity panel.

Affinity fell substantially. BLI, read in the primary text: KD 0.32 nM for the murine parent, 0.47 nM for the chimeric, and 20.6 nM for the Humatch-humanized variant — roughly 44-fold worse than the chimeric and 64-fold worse than the murine parent, computed from the paper's own dissociation constants. Competitive ELISA agreed: "humanization resulted in approximately a 24-fold increase in the IC50 reflecting the reduction in affinity observed by BLI." The programme carried the chimeric variant into its in vivo efficacy work.

Thermostability improved, clearly. The murine and chimeric variants "displayed high initial fluorescence in the absence of ligand, which is indicative of exposed hydrophobic regions in the apo state" and no resolvable apo Tm. The humanized variant "demonstrated no instability at low temperatures and a relatively high intrinsic Tm of 80.4 °C in the apo state, indicative of stabilization of the antibody scaffold upon humanization," rising to 87.2 °C with ligand. The authors' summary: "despite the loss in affinity, the humanized construct displayed high biophysical stability, indicating a stable and well-folded scaffold suitable for manufacturability."

Specificity changed in both directions. The humanized variant lost the chimeric's cross-reactivity with tizanidine and naphazoline — an improvement — but gained measurable binding to free fentanyl, which the chimeric did not have. For an antibody intended for polysubstance overdose, a new opioid cross-reactivity is not a neutral change.

★ The authors' mechanism is the same one BioPhi's failures point at. Verbatim: "Structural analysis revealed that xylazine binds in a deep aromatic and hydrophobic pocket, which also involves framework regions in close proximity to the Complementarity Determining Regions (CDRs). We hypothesize that minor structural changes to the framework regions altered the binding pocket, leading to the observed decrease in affinity. This is well-documented in antibody engineering, where CDR grafting often disrupts Vernier zone residues." A 2.0 Å crystal structure of the parent in complex with xylazine shows the binding pocket forming at the VH/VL interface, which is the thing Humatch's paired model is meant to protect — and which was not running in the version the platform exposes.

Read the limits honestly. n = 1 antibody and 1 Humatch variant. It is a small-molecule hapten binder with an unusually buried, framework-adjacent pocket, which is close to the worst case for framework humanization and not representative of a protein-antigen antibody. No expression titre was reported for the humanized variant, no immunogenicity measurement of any kind was made on it, and the humanized variant was not the one taken in vivo. The direction on affinity is unfavourable, the direction on stability is favourable, and one antibody settles neither. What it does settle is that the comparison is no longer absent — which is one antibody more than BioPhi could say until recently, and which went the same way on affinity.

Kang J, Xu R, Lee S, Luengas D, Kim CM, Marecki C, Gallant J, Moeller N, Shi K, Jahan R, Hicks D, Runyon S, Aihara H, Pravetoni M. Discovery, Structural Characterization, and Preclinical Evaluation of Monoclonal Antibodies against Xylazine Poisoning. ACS Pharmacology & Translational Science. 2026;9(7):1881-1892. doi: 10.1021/acsptsci.6c00117

What the figure doesn't cover

• No published figure relates a Humatch score to clinical immunogenicity by any coefficient. Not R², not Pearson, not AUC against an ADA label. The only published statistic is a three-bin separation, and the paper's own comparison claim is parity with methods whose ceiling is R² = 0.34. Treat a low score as a reason to look harder and a high score as no assurance at all.
• The classification label is species and V-gene family, not outcome. The negatives are mouse and rhesus repertoire sequences. A model that perfectly reproduced that label would tell you what animal a sequence came from. This is the index's one-step-removed caveat at its sharpest: there is no assay anywhere in the label.
• ROC AUC 1.000 is a saturation artefact, not a result. Read PR AUC and F1. KV7 sits at F1 0.587 with ROC AUC 1.000.
• No independent evaluation of Humatch's humanness classification exists. A full-text search of PMC for "Humatch" returns eight articles, which is the whole census this page could reach; of those, one is the paper itself, one independently benchmarks the pairing model, one expressed a humanized design, three use the score as a filter or a citation, and none re-grades the humanness classifier on an external set. PMC does not index every journal, so read that as a floor.
• ⊘ The paired CNN is not loaded on this platform — known issue. The Modal app loads heavy.weights.h5 and light.weights.h5 only and selects exactly one per job by chain_type. There is no paired code path. Humatch's own position is that it "as standard uses the outputs of all three CNNs for paired VH/VL humanness classification," so the integration runs a configuration the paper treats as the non-default, and the distinctive claim — the only one with a link to a measured property — is absent. The fix is to load the paired model and add a joint mode.
• ⊘ Eight platform surfaces describe the output as "Joint VH/VL" — known issue. The result card's description and primary-metric tooltip, the pipeline recipe line, and the catalog rows all say joint, and one catalog description goes further: "Humatch evaluates antibody humanness by jointly scoring paired VH/VL chains ... providing a more accurate assessment than single-chain methods." Every clause of that is false for what runs. The tool's own description is the one surface that gets it right — "scores one chain at a time." The fix is to correct the description in all eight places, including the generated mirrors; until then, read the tool description, not the card.
• ⊘ vl_sequence is accepted and silently dropped at the default chain_type — known issue. Submit both chains without setting chain_type: "L" and only the heavy chain is scored, with no warning and status SUCCESS. To score a light chain you must make a second call with chain_type: "L". The fix is to reject or warn on an unused chain.
• ⊘ humanness_score is max P(one human V-gene family), not P(human) — known issue. The backend takes the largest softmax probability after zeroing the non-human class out of the argmax, and then filters the non-human probability out of v_gene_scores entirely, so no consumer can recover it. Two consequences. A sequence whose probability mass is spread across several human families scores low even when its non-human probability is near zero. And heavy chains are scored over 7 classes while light chains are scored over 17, so the light-chain scale is systematically compressed and a VH and a VL number are not comparable — while the same fixed cutoffs are applied to both. The fix is to emit the non-human probability and to document the per-chain denominator.
• ⊘ The is_human cutoff is a fixed 0.5, and the card's colours disagree with it — known issue. score_threshold defaults to 0.5 in the Modal schema and is not exposed in the tool schema, so it can never be changed from a prompt. No in-repo provenance ties 0.5 to any published Humatch distribution, and it is far below the 0.95 Humatch recommends for all three classifiers "to ensure high precision and recall." Meanwhile the result card paints the score success at ≥ 80%, warning at ≥ 60% and error below — so a sequence at 0.55 is is_human: true in the result and red on the card. The 0.50–0.60 band is a straight pass/fail inversion between two layers of the same platform, and neither cutoff is a published boundary. Read the number, not the colour, and not the label.
• ⊘ scheme is accepted and explicitly never applied — known issue. The Modal schema accepts imgt, chothia and kabat; the code states that "numbering is always IMGT" and records the request as requested_scheme, which is "recorded, not applied." scheme_used is therefore always imgt. Humatch is IMGT-only by construction — it pads to 200 canonical IMGT positions. The parameter is also not exposed in the tool schema at all, yet the inputs page documents it as a user-settable Param. The fix is to remove it from the schema and correct the inputs page.
• ⊘ Three further documented parameters are unreachable — known issue. score_threshold, the sequences batch (documented as "One to 50 per job") and sequence_names all exist on the Modal side and none is exposed by humatch_score, which always submits exactly one sequence. The inputs page presents all three as user inputs. Treat that page's parameter list as describing the Modal app, not the tool.
• ⊘ positional_scores is not per-position — known issue. The outputs page describes it as "per-position humanness, on the numbering scheme used." The backend writes at most one key into it, germline_likeness, and marks the location "legacy." No per-position score is ever emitted. What is measured is germline_likeness — the mean, over the aligned sequence, of how often each observed residue appears at that position in the predicted V gene's germline frequency table — and flagged_positions, which are framework IMGT positions (CDRs excluded) whose residue differs from the most-observed germline residue, ranked by germline-frequency gain. Those are germline-frequency statistics, not humanness scores, and they are the honest fields to read.
• ⊘ Four of the result card's metrics are dead by construction — known issue. The "VH", "VL", "Germline" and "Deviations" metrics read thirteen field names that no Humatch code path emits, so they never render; and top_human_germline, which is emitted, is not read by the "Germline" branch — it appears only inside a tooltip. The live metrics are the primary humanness score, germline likeness, flagged framework positions and the failure count. The outputs page's promise of a "per-chain read" on the card is not met. The fix is to point the extractors at the emitted field names.
• Five runtime figures, one of them shorter than the container timeout — known issue. 52 s padded in pricing, "2–3 minutes" in the tool definition, "~132s" in the tool's dated measurement comment, and "10-30 seconds" / "10-60s" in two older catalog rows, against a hard Modal timeout=120. A job at the recorded real runtime exceeds its own timeout. The price derives from the shortest figure.
• Several fields a result can carry are not documented anywhere. germline_likeness, germline_likeness_error, flagged_position_details, flagged_positions_error, partial, requested_scheme, humatch_git_rev, germline_likeness_method and flagged_positions_method are all emitted and none appears on the outputs page. humatch_git_rev is the useful one: it records the exact upstream commit, which is the version a figure needs to be reproducible.
• A partial job returns SUCCESS. When germline likeness cannot be computed, the row carries germline_likeness: null with an error string and partial: true, while humanness_score is still reported. The platform is honest about it — and a card can therefore show a humanness number with no germline measurement behind it. Where a sequence fails outright the job fails rather than returning an empty success, which is the right behaviour and was a deliberate fix.
• No humanization, and no humanness improvement measurement. The platform scores; it does not rewrite. Every humanization figure on this page — the 0.77 / 0.82 overlap, the edit distances, the 33-second runtime — describes a capability you cannot reach from here. BioPhi is the humanization route, and its own expression evidence is unfavourable.
• The version is pinned, which is unusual and good. The Modal image installs Humatch from a fixed upstream commit, and an image-build assertion fails the build unless a reference sequence reproduces a known germline-likeness value. A figure quoted with humatch_git_rev is reproducible. That is the opposite of BioPhi's unpinned git HEAD.
• Nothing here measures binding, expression or developability. Humanness and germline likeness are blind to all three. The one expressed Humatch design lost 44-fold affinity while gaining thermostability. Run antibody developability on any candidate, and read its translatability page first, because the platform's descriptors there are hand-written rules.

Go / no-go

Go — act on the result when:

• You are ordering or triaging a candidate list by resemblance to the human germline repertoire, and you want the relative rank. That is what the classification figures support, and the classifier is well characterised on that task.
• You want to know which human V gene a sequence is closest to, and where it departs from that gene's germline consensus. top_human_germline, germline_likeness and flagged_positions are real measurements of exactly that, computed on the IMGT-aligned sequence.
• You are cross-checking a sequence that another tool humanized. A score cannot grade a sequence it generated, and Humatch did not generate yours — it is the independent metric against a BioPhi output, and that cross-check is the one lesson in this literature with a measured demonstration behind it.
• You are screening for mixed-gene designs. Rejecting 51% of artificial between-gene sequences against OASis's 1% is the largest margin Humatch has over any competitor.
• You quote the figure with its chain type, its denominator and humatch_git_rev, and you read humanness_score rather than is_human or the card's colour.

No-go — do not act on the result when:

• You are predicting whether a candidate will provoke ADA in patients, or comparing two candidates' clinical immunogenicity risk. No published coefficient supports either for Humatch, and the benchmark its paper uses is independently argued to measure species.
• You are treating a high score as clearance, or a low score as a liability. The score drops for long or heavily mutated CDR3s and for sequences between germlines — and it rejected a third of the humanized antibodies that reached the clinic, in its predecessor's measurement.
• You are comparing a VH number to a VL number. They are maxima over 7 and 17 classes respectively.
• You are reading the output as a joint VH/VL assessment. The paired model is not loaded, and a result from a single call describes one chain.
• You passed vl_sequence without setting chain_type: "L". Only the heavy chain was scored.
• You are relying on scheme, score_threshold, sequence_names or a multi-sequence batch to have shaped the result. None of them reaches the compute path.
• You are reading positional_scores as per-position humanness, or the card's absent "VH" / "VL" / "Germline" / "Deviations" metrics as missing data rather than as metrics that cannot render.
• You intend to ship a humanized sequence — from any tool — without expressing it and measuring binding, titre and thermostability. The one Humatch design ever expressed lost roughly 44-fold affinity, and its authors traced it to framework residues next to the CDRs.
• You need an immunogenicity risk assessment for a regulatory filing. That requires the assay cascade — MHC-II and T-cell assays, then clinical ADA monitoring — and no sequence score substitutes for it.

Caveats on reading this page: every figure above is quoted from the cited source with its original metric, threshold and sample, and every quantitative figure was read in the primary full body text rather than an abstract. Author-reported, self-compared and independent figures are labelled as such throughout: the classification and humanization tables are author-reported, the Hu-mAb comparison is a same-laboratory self-comparison, and the thermostability correlation, the expression correlation, the pairing benchmark and the expressed-and-assayed result are independent. Figures from different rows are not comparable, because label sets, thresholds, metrics and denominators differ between them. Where a comparison was not located — notably any coefficient relating a Humatch score to clinical ADA, any independent re-grading of its humanness classifier, any expression titre for a Humatch design, and any immunogenicity measurement of any kind on one — this page says so rather than substituting a figure from a different method, a different tool or a different version.