fredzzp/esm-teddymer-pseudodimers

ESM-Teddymer pseudo-dimers

60,177,402 intra-chain domain pairs ("pseudo-dimers") cut out of ESM Metagenomic Atlas monomers at Chainsaw/TED domain boundaries. Each row is a target domain and a binder domain that were adjacent in one real folded chain, so the pair comes with a real interface without anyone having to dock anything. Built to train target-conditioned binder-design models.

All-atom structures for both chains ship alongside as foldcomp.

What is in here

data/train-*.parquet              80 shards   60,177,402 rows x 32 cols   ~17 GiB
structures/dimers_*.tar        2,000 tars    120,354,804 .fcz members    ~348 GiB
structures_index/index-*.parquet  20 shards   row -> tar shard + geometry  ~2 GiB
pipeline/                        587 files   the code that built all of it   3 MB

Row example_id TED_<parent_hash>_<a_start>-<a_end>_<b_start>-<b_end> addresses its structures directly: chain A is member <example_id>_A.fcz of structures/dimers_<shard>.tar, chain B is <example_id>_B.fcz, with shard from structures_index.

import foldcomp, tarfile, pyarrow.parquet as pq
row = pq.read_table("structures_index/index-00000-of-00020.parquet").slice(0, 1).to_pylist()[0]
with tarfile.open(f"structures/dimers_{row['shard']}.tar") as tf:
    _, pdb_a = foldcomp.decompress(tf.extractfile(f"{row['example_id']}_A.fcz").read())

Columns

group	columns
identity	example_id, protein_hash
sequence	target_seq, binder_seq, parent_seq_len
domain spans	a_start, a_end, b_start, b_end, n_res_a, n_res_b, n_domains_in_parent
interface	interface_length, mean_interface_plddt, iface_sig (64-d)
parent quality	parent_ptm, parent_mean_plddt, parent_pae_mean
clustering	cluster_a, cluster_b, pair_key_size, pair_key_rank, desc_dist_to_rep
pseudo-ipSAE labels	pipsae, pipsae_sym, pilis, pilia, pae_block_mean, pae_block_min, pae_ok
all-atom	sc_atom_frac, n_sidechain_atoms

interface_length uses a CA-only 10 Å interface. iface_sig and the pseudo-ipSAE labels use CB-else-CA at 8 Å. The two definitions are deliberately different and must not be mixed.

The sampling structure — read this before you train on it

The corpus is redundant on purpose. pair_key = sorted(cluster_a, cluster_b) is the sampling unit:

60,177,402  pairs
 6,611,707  pair keys   66.8% are true singletons (4,416,572 keys hold 1 pair)
                        the other 33.2% hold 55.8M pairs at 25.4 each
                        p50 size 1, p90 7, p99 112, max 93,077

Ship-everything means w ∝ 1/size^α is a train-time choice, not a build-time one. α=0 (row-uniform) draws 62% of its samples from 1.1% of the keys; α=1 (cluster-uniform) is maximum diversity. An earlier release deduplicated to 2 rows per key and thereby hard-coded α≈1 irreversibly; pair_key_rank >= 0 reproduces that 8,805,444-row subset exactly as a set.

Cluster-uniform with N = len(dataset) covers 99.99% of keys per epoch; 99% coverage needs 30,448,035 draws.

Quality — measured, and the number you have probably seen is the wrong one

Folded as 2-chain complexes with ESMFold2 under a 3-gate of i_PAE < 10 & complex_pLDDT > 0.8 & i_pTM > 0.5:

set	3-gate pass
strict pseudo-dimers, pre-dedup, row-weighted (n=1,000)	0.3620
length-matched cross-parent shuffled control	0.0060
real PINDER, same gate	0.2120
full corpus, row-uniform	0.3897 [0.2367, 0.5432]
full corpus, cluster-uniform	0.1981 [0.1776, 0.2190]

The widely-quoted 36.20% is row-weighted and pre-dedup. Under cluster-uniform sampling — which is what you use it for — expect ~0.20, because 93% of rows sit in multi-member keys that pass at ~2x the singleton rate, and cluster-uniform discards exactly that weighting. Larger keys really are better folds (ρ(gate, log size) = +0.161, +0.101 after controlling for length, interface size and interface pLDDT).

The shuffled control is what makes this a result rather than "ESMFold2 likes two folded things".

ESMFold2 was never a filter. These numbers characterise the corpus; no row was selected on them. Build-time filters are parent pTM/pLDDT, domain geometry, interface size and interface pLDDT only.

pipsae (pseudo-ipSAE) over all 60,177,402 rows: p50 0.7581 / mean 0.7045, against 0.7576 / 0.7031 on an independent 10k prototype. It is the useful quality lever where the free columns saturate: on the older dedup'd subset, S_iface ≥ p90 gave 0.3741 (no better than ≥ p75) while pipsae ≥ p90 gave 0.5238.

Structure fidelity

Stored as foldcomp, not PDB text: 9.40 TiB → 0.34 TiB (35.8x).

• Backbone RMSD 0.0575 Å, measured at scale (4,000 chains / 561,194 residues) against the original Atlas coordinates. Per-chain mean 0.0551, p99 0.0835, max 0.1142.
• Sidechain RMSD 0.139 Å (n=40, name-matched).
• Per-atom tail: p99 0.347 Å, p99.9 0.602 Å, max 1.741 Å — about 4 atoms per dimer past 0.5 Å. A 0.5 Å per-atom gate fails nearly every row on a correct corpus; use per-chain RMSD.

Two traps if you re-measure:

1. Match atoms by (chain, resSeq, atom_name), never by index. foldcomp reorders atoms within a residue; index-matching gives a phantom 0.816 Å sidechain RMSD. Backbone is 0.053 Å either way, which is the tell.
2. Atom order here is N, CA, C, O, CB (chainpack order), not atom37's N, CA, C, CB, O. Reading one as the other swaps O and CB on every residue and still produces a loadable file.

foldcomp.compress() writes 2 bytes of uninitialised memory at offsets 14/15, so .fcz bytes differ across processes (6.76% of members) while decompressing identically (0.000000 Å). Those two offsets are zeroed here, so byte checksums are meaningful.

Known limitations

• Discontinuous domains are dropped. Keeping them would be +64% yield, but concatenating segments invents a peptide bond that a sequence model would learn. Needs a chain-break token first.
• A pseudo-dimer is not a real complex. The 3-gate's blind spot: across passing dimers ipSAE falls 0.97 → 0.35 while pLDDT / i_pTM / i_PAE barely move. The bottom of that ranking is "two well-folded domains sitting side by side with no interface". Filter on interface area or pipsae.
• Larger-domain = target, smaller = binder, by construction. Other Teddymer-style sets do not use that convention.
• Sequences and structures are ESMFold predictions of metagenomic ORFs, not experimental structures.
• Backbone-only tensors are not shipped; regenerate them from the tars with the published extract_bb.py (~200 CPU-hours, ~1.6 h wall at 64-wide). They were 523 GiB of a strictly lossier view of structures/ (5 of ~35 atoms, fp16 on top of foldcomp).

Provenance

component	source
sequences + structures	ESM Metagenomic Atlas (Meta AI / EvolutionaryScale)
domain boundaries	Chainsaw / TED (Encyclopedia of Domains)
structure clustering	foldseek, TM ≥ 0.7, coverage 0.8, set-cover
compression	foldcomp
quality characterisation	ESMFold2

This dataset is released under the MIT License. That covers the derived artifacts here — the pair table, the labels, the clustering, and the packaging. It does not restate the terms of the upstream components above; please honour their attribution requirements as well as this one.

If you use this data, cite the ESM Metagenomic Atlas and the TED / Chainsaw domain assignments alongside this dataset.