ESM-3 troubleshooting

When ESM-3 fails or produces embeddings that don't behave as expected, this page covers the common failure modes — from sequence validation and out-of-memory errors to long runtimes, low-quality embeddings, unexpected similarity scores, and attention-map issues.

Common issues and fixes

ESM-3 uses a large transformer model to generate protein embeddings. Understanding common issues helps you prepare inputs correctly and interpret results appropriately.

Sequence validation failures

• Ensure only the 20 standard amino acids are present (ACDEFGHIKLMNPQRSTVWY); remove ambiguous codes B, J, O, U, X, Z
• Remove whitespace, numbers, and FASTA comments from within the sequence body
• Check that sequence length is within the model context window (up to 2,048 tokens)
• Verify FASTA headers are unique when submitting multiple sequences

Out-of-memory errors

• Long sequences (>1,000 residues) with the large model variant require significant GPU memory
• Try switching to the small or medium model variant for very long sequences
• Split multi-domain proteins into individual domains and embed them separately
• Reduce batch size if submitting multiple sequences; process smaller batches sequentially
• Attention map extraction (return_attention: true) roughly doubles memory usage; disable if not needed

Long runtimes

• The large model variant can take several minutes per sequence; the small model is 5-10x faster
• Batch processing is more efficient than sequential single-sequence submissions
• Check cluster capacity and retry during lower-traffic periods if queue times are long
• For time-sensitive applications, use the small model for initial screening and the large model only for final analysis

Low-quality embeddings

• Short sequences (<30 residues): The model has limited context for very short peptides. Consider padding with flanking residues if the biological context is known
• Repetitive sequences: Low-complexity regions (poly-Ala, Gly-rich linkers) produce less informative embeddings. This is expected and reflects the low information content of these regions
• Non-natural sequences: Heavily engineered or synthetic proteins may fall outside the training distribution. Embeddings are still computed but may be less reliable for similarity comparison with natural proteins

Unexpected similarity scores

• Ensure you are comparing embeddings from the same model variant; embeddings from different models live in different vector spaces
• Very high similarity (>0.95) between unrelated proteins may indicate a pooling issue — try pooling: "cls" instead of mean
• Low similarity between known homologs may occur for proteins with extensive disordered regions

Attention map issues

• If attention maps are all zeros, ensure return_attention was set to true before submission
• Very uniform attention patterns (high entropy) may occur for short or repetitive sequences
• Contact prediction from attention is approximate; for structural contacts, use ESMFold directly
• Multi-head attention averaging can obscure individual head patterns; try examining specific heads if contact prediction seems poor

Batch processing errors

• If some sequences in a batch fail while others succeed, check the failed sequences for length violations or invalid characters
• Extremely heterogeneous batch lengths (10-residue peptide mixed with 2,000-residue protein) may cause padding inefficiency
• Split batches by approximate length for more efficient processing

Still stuck?

• Provide the job ID, model variant, sequence length, and any error messages
• Note whether the issue is at submission, during computation, or in the output
• Share the FASTA header and first 50 residues of the problematic sequence
• Specify whether you need per-residue or per-sequence embeddings

Read next