PETase ANnotation and Triage System

Methods and limitations

Spec section 7 calls this page non-negotiable "given how easily this kind of tool is over-read". This project has more to disclose than most.

Trained on the measured set alone, the activity head reaches AUC 0.790 (± 0.010) across 126 independent sequence clusters at 30% identity, against 0.565 for a classifier using amino-acid composition alone. It clears that baseline by a real margin.

It does not clear the one that matters. Retrieval — scoring a sequence by its similarity to the nearest enzyme already known to degrade PET, outside its own cluster — reaches 0.787. The learned head trained on measured labels comes in 0.003 against it. On labels somebody actually measured, the model does not beat looking the answer up. Trained on the annotated set instead it reaches 0.859 and clears retrieval by +0.072 — on labels a computer assigned by similarity, which is the rule retrieval already encodes.

These figures are read from the evaluation run's own output rather than written here, because this paragraph previously quoted an AUC of 0.976 against a composition baseline of 0.829 for some time after the protocol had been re-run and returned different numbers. Prose does not get recomputed.

Measured negatives, and why they changed everything

The hardest question in this project is not "is this a polyesterase" — that separates at AUC 0.975 — but "does this polyesterase degrade PET". Answering it needs enzymes that were tested and found not to, and for most of this project's life there were 29 of those, all on the weakest possible basis: PAZy records only positive substrate associations, so an enzyme missing from its PET list might have been assayed and failed, or might never have been assayed at all. Those two are not the same claim and the database cannot tell them apart.

Two 2025 papers screened panels under a single protocol and, crucially, reported what did not work. Both deposited their source data openly. Ingesting them takes the measured negative class from nothing to 150.

Both screens also measured melting temperatures, and the ACS screen assayed at 40 °C as well as 60 °C. Almost every optimum in this catalogue sits between 50 and 78 °C; the therapeutic target is 37 °C, so a screen that reports what happens near body temperature is worth more here than its enzyme count suggests.

And the same question asked of geometry

Active-site geometry was the one signal here measured off coordinates rather than inherited from somebody's annotation, so it was the obvious thing to re-test once real labels existed. On the inferred negatives it looked convincing: cleft depth separated PET-active from PET-inactive at AUC 0.808, p 1.7×10⁻⁷, falling to 0.534 under cluster-grouped splitting — which was read as an underpowered evaluation waiting for more data.

Asked of 237 PET-active against 139 measured-inactive enzymes, all ESMFold on both sides so no coordinate-source confound is in play, the answer is AUC 0.398 (0.360–0.453 across 10 random splits).

Below a coin flip, and consistently so — every one of the 10 splits lands under 0.5. That is worth reading carefully, because it does not mean the classifier is useful inverted. It means whatever relationship the features carry inside the training clusters reverses in held-out ones, which is the signature of fitting lineage rather than activity: the geometry says which family an enzyme belongs to, and family does not say whether it degrades PET.

The trajectory is the strongest part of the evidence. As the measured negative set filled in, the result went 0.507 at 95 negatives, 0.498 at 125, 0.459 at 139, and 0.398 with the positive set complete. A real effect emerging from a thin sample strengthens and narrows around a non-null value; moving toward and past chance as the sample grows is a confound being diluted. That distinction is exactly the one that let the earlier reading survive as "underpowered" for as long as it did.

The raw features are where it is clearest. Not one of the nine survives at p < 0.01, and cleft depth — the 0.808 above — comes in at 0.547, p 0.12. That signal was never measuring PET activity. It was measuring whatever separates the enzymes a database happens to list from the ones it does not, and it disappeared the moment the labels came from somebody running the assay.

Two limits worth stating with it. Only 8 clusters contain both classes, so the grouped estimate rests on few independent lineages — though its stability across seeds argues the answer is real. And every structure here is a prediction: if ESMFold idealises the active site, which the paired crystal-versus-model comparison above shows it does, a genuine difference could be flattened by the modelling rather than absent from the enzymes.

What the rest of that supplement held

The Science paper ships seven data files. Only two of the remaining three were worth ingesting, and saying which is part of the point.

One result falls out of S7 directly. Under the paper's own sequence-cluster framework, 14 clusters contain both a PET-active and a measured-inactive enzyme, against 8 under the 30% identity clustering used for the splits here. The shortage of lineages measured on both sides is real under either definition, and slightly less severe under theirs.

What 123 more negatives actually bought

Not what was expected, and the disappointment is the finding. The within-family contrast — measured PET-active against measured PET-inactive, both polyesterases — now runs on 149 negatives instead of 26. The number that governs whether it can be evaluated barely moved: clusters holding both an active and an inactive enzyme went from 7 to 8. The constraint was never how many negatives exist. It is how many independent lineages anyone has measured on both sides, and five times the data bought one more.

Two readings, and both matter. On the full measured contrast the head scores 0.648 and amino-acid composition scores 0.644 — no advantage at all over counting residues. Restricted to the clusters that contain both classes, the head reaches 0.625 against composition's 0.519, which is a real margin.

The clearest gain is in the error bars. That hardest test previously read 0.435 ± 0.301, an interval so wide it excluded nothing; it now reads 0.625 ± 0.081. The model did not get better. The experiment got powered enough to give a stable answer, and the stable answer is a modest signal that is not composition. That is a smaller claim than "we solved it" and the first one here that survives its own error bar.

A protein is only labelled from a condition it was actually assayed under. An enzyme absent from a condition is not a zero in it, so comparable_group_id keys on substrate, temperature and pH together and a crystalline-powder result at 60 °C is never pooled with an amorphous-film result at 40 °C.

Why none of it transfers: the determinants are lineage-specific

Three independent methods had failed the same way — sequence embeddings, a learned head, and active-site geometry all worked out-of-family and collapsed within it. The usual reading of that is too little data. This test asks a different question, and it needs no train/test split at all, so it cannot be leakage in disguise.

Fit the same model separately inside each of the three lineages large enough to support it, then compare the directions the fits point in. A single global rule for what makes a PETase can only exist if those directions broadly agree.

The comparison needs its own reference, and that is the part that makes it a test rather than a number. A cosine similarity of 0.3 between two coefficient vectors means nothing on its own, because with a few dozen enzymes the vectors are noisy and even two samples of an identical process would not agree perfectly. So the reference is not zero: it is how well two bootstrap resamples of the same lineage agree with each other — an empirical ceiling on what agreement looks like when the true direction is the same by construction.

Two fits of the same lineage agree at about +0.70. Two different lineages agree at roughly zero, and in two of three pairs they point in opposite directions — what predicts activity in one family predicts inactivity in another. The same pattern appears in two feature sets that share nothing: one derived from 3D coordinates, one from a language model over sequence.

That the ceiling is high is what makes this a result rather than a shrug. The within-lineage directions are well determined; it is specifically the comparison between lineages that fails, so this is not the test being too noisy to answer.

It also explains the below-chance results. A model trained across lineages learns a direction that reverses on a lineage it has not seen, which is exactly how geometry reached 0.398 and why an eighteen-fold larger language model changed nothing. The problem was never capacity, and it was never only sample size: there may be no single global rule to learn. Established on three lineages, so it is a strong indication rather than a settled law — but it is the first explanation offered here that accounts for every failure at once.

Where the signal holds, and what to ship instead

With a global rule ruled out, one question remains and it is the one a user would actually ask: given a candidate resembling enzymes we have characterised, how close does it have to be before the prediction means anything? The 262-member lineage — 171 measured-active, 91 measured-inactive — is the only place with enough data to answer. Re-cluster inside it at successively stricter identities, hold out whole sub-clusters, and read where the curve meets chance.

Read the point estimates alone and there is an operating band: 0.703 against retrieval's 0.544 at 70% identity. Read the intervals and it is not there. The paired lead — head minus retrieval on the same held-out sub-clusters — is +0.159 [−0.263, +0.390], spanning zero. Seventeen sub-clusters cannot resolve a difference that size, and the curve is not even monotonic: amino-acid composition beats both at 80%.

One thing does survive. At 70% identity the head's own interval, [0.521, 0.798], excludes 0.5 — so within a lineage, among relatives sharing about 70% of their sequence, there is weak evidence of better-than-chance discrimination. It just cannot be shown to beat looking up the nearest known PETase.

So the honest deliverable is retrieval, not a classifier. Nearest-neighbour similarity to a measured-active enzyme is simple, needs no training, carries no risk of learning a lineage direction that reverses, and nothing built here has been shown to beat it on labels somebody measured. That is a smaller claim than this project set out to make and it is the one the evidence supports.

Positives by evidence tier

Training runs

The composition baseline uses amino-acid fractions and length only. Any model claim must clear it as well as the retrieval baseline.

Pipeline runs

Data sources

Limitations

1. Positives number in the low hundreds and most carry family annotation rather than measured PET activity.
2. Published activity data is not harmonised across assay formats. Absolute rate predictions should not be trusted.
3. Crystalline PET degradation at 37 °C by any known enzyme is slow. PANTS ranks relative promise, not therapeutic viability.
4. Predicted structures are predictions. Cleft geometry on a metagenomic sequence with no close homologue carries real uncertainty, and a brittle version of that measurement scored IsPETase's own prediction at half its crystal value before being fixed.
5. Predicted and experimental coordinates are not interchangeable. Measured on the same protein both ways (51 enzymes with a crystal structure and an AlphaFold model), the oxyanion hole differs systematically: the second donor's angle is 23.6° in the crystal against 15.6° in the model, p 1.1×10⁻⁶. Predictions build a tighter, more idealised active site than the protein has. Cleft depth is the one geometric feature that survives the comparison, and it is also the strongest activity feature. Pooling sources without accounting for this dropped a geometry model from 0.749 to 0.553, which is why every structure here records where its coordinates came from.
6. Leave-one-family-out cannot be run at all. All 13 ESTHER families in this catalogue are wholly positive or wholly negative, so holding one out removes an entire class. That is not a gap in the protocol; it is the clearest statement of the problem, which is that in this dataset the label is family membership.
7. That AUC is measured against hard negatives from other α/β-hydrolase families. Tested against the near misses instead it reaches 1.000, which is a warning rather than a result: every near miss is a single ESTHER family (Cutinase), and one homogeneous family separates as a block (composition alone scores 0.910). Neither contrast shows that the head ranks PET activity within the polyesterase family.
8. The within-family question has no dataset. It needs PET-inactive polyesterases, measured and published, and databases record what works: nobody systematically publishes a failed assay. That is publication bias, not a curation gap, and more database mining will not fix it.
9. Nothing here addresses delivery, immunogenicity, biodistribution, or what happens to liberated TPA and EG in vivo.
10. Metagenomic candidates may come from unculturable organisms, may not express in a standard host, and may be fragments or misassemblies.
11. Cleft width currently separates a fungal cutinase from the polyesterase family, but does not order that family by PET activity. Family-level separation is a much easier task than the one that matters.

Recent manifests