Structure module: a symmetry you can prove, or one you learn

Evoformer and Pairformer produce a pair representation — the model's hypothesis about which residues touch. This page turns that into coordinates, and the interesting part is a decision AlphaFold3 made that AlphaFold2 did not.

The shapes involved — pair_rep in, a rigid frame per residue out — are laid out in what goes in, what comes out.

A protein has no preferred position or orientation, so a model predicting coordinates must satisfy

$$\text{predict}(R\mathbf{x} + \mathbf{t}) = R\,\text{predict}(\mathbf{x}) + \mathbf{t}$$

for every rotation $R$ and translation $\mathbf{t}$. There are two ways to get that, and they are not equally verifiable.

Measured

uv run structure.py, error relative to the molecule's own size, three seeds, float64:

head	equivariance error	guaranteed?
ipa	3.6e-16	yes — by construction
diffusion	4.5e-02	no — must be learned
mlp	2.7e-01	no — and not trying

The same molecule, rotated, rendered twice. Grey underneath is where the prediction should be. On the left IPA sits exactly on it through the whole rotation. On the right the AF3-style head wanders off it and back, because it is reading raw coordinates and has only ever learned that orientation should not matter.

The drift is exaggerated 6× to be visible at this size, and the figure says so — at true scale it is 5% of the molecule's width, which is large enough to matter and small enough to be invisible in a thumbnail.

uv run scripts/make_protein_animations.py --only se3-equivariance

IPA's figure is float noise. Invariant Point Attention compares points only after mapping them into per-residue local frames, and the distance between two points in a shared frame cannot change when the whole molecule moves:

$$\text{logit}_{ij} = \frac{q_i \cdot k_j}{\sqrt{c}} + b_{ij} - \frac{\gamma}{2}\sum_p \left\| T_i(\mathbf{q}^p_i) - T_j(\mathbf{k}^p_j) \right\|^2$$

The third term is the guarantee. Apply a global transform and every point moves with its frame, so every distance — and therefore the output — is unchanged. Arithmetic, not training.

Does the augmentation work?

AlphaFold3 removed IPA and recovers the symmetry by randomly rotating every training example. Measured on 1 × RTX 3080 Ti, equivariance taken from the trained model:

arm	FAPE	vs do-nothing	equivariance
--head ipa	1.7972	+18.8%	1.086e-15
--head diffusion	2.2785	−3.0%	5.075e-02
--head diffusion --augment	2.2917	−3.6%	7.807e-03
--head mlp	2.2662	−2.4%	3.335e-01

Yes — 5.1e-02 → 7.8e-03, a 6.5× reduction. AF3's strategy is real.

And no — it is still thirteen orders of magnitude from IPA. A guarantee holds on inputs you never tested. A learned symmetry holds on inputs that resemble the ones you did, and degrades quietly elsewhere, which a held-out split drawn from the same distribution cannot detect.

Scope this one carefully

Only IPA beats the do-nothing baseline here, and that is partly an artefact of the lab. This setup gives the head no sequence and no pair features — s and z are zeros — so geometry is the only signal, and reading geometry is exactly what IPA can do and plain attention cannot. Inside a full AlphaFold the trunk supplies features the other heads could use. This is a statement about this lab, not about AF2 versus AF3.

So why did AlphaFold3 give up the guarantee?

Generality, not carelessness. IPA needs a residue frame built from N, CA and C atoms. AlphaFold3 predicts ligands, ions, nucleic acids and modified residues — most of which have no backbone and therefore no frame to build. Dropping the architectural guarantee is what let the model treat everything as atoms.

A symmetry you can prove, against a model that can represent more things. Neither choice is obviously right, which is why both shipped.

Prediction on truth, in the CASP style

Green is the experimental backbone of a real CATH chain. Blue is this repository's structure module. They are superimposed by a real Kabsch fit and scored with a real GDT-TS — the metric on DeepMind's CASP14 panels, computed the standard way: the mean, over 1, 2, 4 and 8 Å cutoffs, of the percentage of residues within that distance.

This is not AlphaFold2, and the difference is the whole point

DeepMind's CASP14 figure shows AlphaFold2 predicting a target it had never seen, from sequence alone. This shows a ~50k-parameter structure module, trained here for 300 steps, refining a backbone that was deliberately noised. Those are enormously different problems, and a figure that blurred them would be exactly the fabricated result this course has rules against.

What is honest about it: the chain is real, the superposition is real, and the GDT-TS is computed properly — with a single global Kabsch fit rather than the per-cutoff search official CASP uses, which means the number is under-reported, never inflated. What it demonstrates is that the course's own model produces something you can superimpose and score. Not that it competes with AlphaFold2, which it emphatically does not.

For the comparison, the Evoformer page carries AlphaFold2's actual CASP14 submissions against the same kind of experimental structure. That figure and this one deliberately use the same visual language so the difference in difficulty is visible rather than asserted.

uv run scripts/make_protein_animations.py --only prediction-vs-truth

An open question this lab is positioned to ask

The measurement above — IPA exact, the AF3-style head at 5.1e-02, augmentation closing it to 7.8e-03 — was taken in distribution. Every test structure resembled the training structures.

That leaves a question nobody appears to have answered:

Does a learned symmetry decay off-distribution, while an architectural one stays flat?

A guarantee holds on inputs you never tested. A learned symmetry is a claim about the training distribution, so it ought to weaken as you move away from it — but "ought to" is not a measurement, and the size of the effect is what decides whether AlphaFold3's trade was cheap or expensive.

What the literature has, and what it does not

There is a substantial theory literature on augmentation versus built-in equivariance: Data Augmentation vs. Equivariant Networks (2022) derives generalization bounds, and Emergent Equivariance in Deep Ensembles (2024) shows ensembles become equivariant even off the data manifold. Almost none of it concerns proteins — the empirical work sits in robotics, turbulence and vision.

On the protein side the debate is acknowledged but not measured. AlphaFold3 dropping IPA reopened the question of how physical invariances should be captured, and Transformers trained on proteins can learn to attend to Euclidean distance (2025) argues plain Transformers suffice — without reporting an equivariance error.

A citation-graph check in October 2026 found that of 332 works citing OpenFold (Nature Methods, 2024), only 5 mention equivariance and 2 mention symmetry, none measuring it; and of 16,107 works citing AlphaFold 3, 83 mention equivariance, overwhelmingly as review-level commentary.

The experiment exists already, with the wrong metric

OpenFold elided large regions of fold space to test how AlphaFold2 generalises — and measured accuracy. Swapping the metric gives a well-posed study:

Train an augmentation-only head, then measure equivariance error on held-out fold space against seen fold space.

Falsifiable in both directions, and the negative result is as useful as the positive: a flat curve would say augmentation is sufficient and the field can stop arguing about it.

How much to trust this

These are abstract- and citation-level searches, not a systematic review. They can miss work buried in a methods section. A lead, not a clearance.

The loss has to be invariant too

Frame-Aligned Point Error compares predicted and true coordinates inside each residue's own frame, so a globally rotated but otherwise perfect prediction costs nothing. A plain RMSD would punish it, and the model would burn capacity learning an arbitrary orientation.

That distinction is sharper than it looks. Transforming prediction and truth together leaves a raw RMSD unchanged too — both move the same way. The property that separates them is transforming the prediction alone: FAPE ignores it, RMSD does not. The first version of this course's test got that wrong and passed on a sabotage that replaced FAPE with RMSD.

Three attempts at the data

--data synthetic builds backbones from helices and strands. It took three tries, and the failures teach more than the success:

generator	improvement over do-nothing
random walk, 3.8 Å steps	+1.0%
helix/strand, N and C at fixed global offsets	−0.7%
helix/strand, N and C in each residue's local frame	+18.8%
real CATH backbones	+33.1%

A random walk has nothing to denoise toward — the true positions are themselves random, so +1.0% is near the information-theoretic ceiling, not a weak model. And structure in the CA trace was not enough: with N and C at constant global offsets every frame pointed the same way, carrying no local information, and the model did worse than nothing.