ESM-3 on the platform

ESM-3 is a protein language model for design and analysis, not folding: it produces embedding vectors that encode sequence, structure, and function, and it can generate novel proteins. Reach for it when you want alignment-free similarity, zero-shot function prediction, clustering, or ML features — the tasks that come before or after structure prediction rather than the fold itself. If you need the 3D coordinates of a sequence, predict them with ESMFold or fetch them with AlphaFold instead. See what you can achieve for the full set.

What ESM-3 does

ESM-3 is a multimodal protein language model, a generative model that reasons over the sequence, structure, and function of proteins simultaneously. It produces high-dimensional embedding vectors that encode deep evolutionary and functional relationships learned from training on billions of natural protein sequences.

Citations

Hayes T, Rao R, Akin H, Sofroniew NJ, Oktay D, Lin Z, Verkuil R, Tran VQ, Deaton J, Wiggert M, et al. Simulating 500 million years of evolution with a language model. Science. 2025 Feb 21;387(6736):850-858. doi: 10.1126/science.ads0018

Key achievements

• Multimodal architecture: Jointly models sequence, structure, and function tracks in a single transformer, enabling cross-modal reasoning
• Scale: Trained on billions of protein sequences with up to 98 billion parameters in the largest variant
• Generative capability: Can generate novel proteins by conditioning on any combination of sequence, structure, and function inputs
• Function prediction: Zero-shot function annotation from embeddings alone, without requiring homology search

Core capabilities

• Sequence embeddings: Produce per-residue (L x D) and per-sequence (1 x D) embedding vectors for downstream ML tasks
• Zero-shot function prediction: Predict protein function from embeddings without fine-tuning, using learned representations of functional annotations
• Similarity search: Compute cosine similarity between embedding vectors for fast, alignment-free protein comparison
• Evolutionary analysis: Embedding space captures evolutionary relationships, enabling clustering and phylogenetic inference
• Contact prediction: Attention maps encode predicted residue-residue contacts for structural insight
• Embedding-based clustering: Group proteins by functional or structural similarity using embedding distance metrics

Technical approach

ESM-3 uses a transformer architecture with discrete tokenization of three modalities:

1. Sequence track: Amino acid tokens processed through the transformer encoder
2. Structure track: Structural tokens derived from local structural neighborhoods (VQ-VAE encoded)
3. Function track: Functional annotation tokens from InterPro, GO terms, and other annotation databases

The model is trained with a generative masked language modeling objective across all three tracks, learning to predict masked tokens conditioned on the unmasked context. This enables:

• Embedding extraction: Use internal representations as feature vectors
• Conditional generation: Generate sequence given structure, or function given sequence
• Cross-modal transfer: Structure information improves function prediction, and vice versa

When to use ESM-3

Primary use cases

Protein similarity and clustering

• Alignment-free comparison: Compare proteins without sequence alignment; embeddings capture remote homology that BLAST misses
• Large-scale clustering: Cluster millions of sequences by embedding similarity in minutes
• Functional grouping: Group proteins by predicted function rather than sequence identity

Function prediction and annotation

• Novel protein annotation: Predict function for proteins without detectable homologs
• Gene ontology prediction: Infer GO terms from embedding representations
• Enzyme classification: Classify proteins into functional categories using embedding-based classifiers

Feature extraction for machine learning

• Transfer learning: Use ESM-3 embeddings as input features for custom ML models (activity prediction, localization, stability)
• Representation learning: Pre-computed embeddings eliminate the need to train protein representations from scratch
• Multi-task models: Combine embeddings with other molecular descriptors for comprehensive property prediction

Evolutionary and structural analysis

• Evolutionary distance: Embedding distances correlate with evolutionary divergence
• Contact prediction: Attention maps provide residue-residue contact predictions
• Variant effect prediction: Compare wild-type and mutant embeddings to estimate functional impact

Webapp integration and workflow

Input methods in .bloom*

Direct sequence entry:

• Paste amino acid sequences in raw or FASTA format
• Real-time validation with length and composition checks
• Support for batch submission of multiple sequences

UniProt integration:

• Direct UniProt accession lookup (e.g., "P12345")
• Automatic sequence fetching with metadata
• Isoform selection for proteins with multiple variants

From previous jobs:

• Import sequences from BLAST search results
• Chain from ESMFold, fragment extraction, or other sequence analysis jobs
• Provenance tracking maintains connection to original data source

Integration with .bloom* ecosystem

Upstream data sources

• UniProt: Sequence fetch for known proteins
• BLAST: Hit sequences from homology searches
• ESMFold: Predicted structures (sequence extraction)

Downstream applications

• DiffDock: Feed embedding-based target characterization into docking workflows
• P2Rank: Pocket prediction on embedding-informed targets
• DeepChem: Use ESM-3 embeddings as protein features for binding prediction models
• Custom ML: Build protein property classifiers using embedding vectors

Best paired with

• ESMFold: Use ESM-3 for embeddings and ESMFold for structure prediction; complementary tools from the same model family
• BLAST: Combine alignment-based and embedding-based similarity for comprehensive homology analysis
• DeepChem: Use embeddings as features in molecular property prediction pipelines

Scope and limitations

Supported inputs

Sequence requirements

• Standard amino acids: 20 canonical amino acids (ACDEFGHIKLMNPQRSTVWY)
• Length range: 10-2,048 tokens (shorter sequences may lack context for high-quality embeddings)
• Single chain: Each sequence is embedded independently
• Batch support: Multiple sequences can be submitted in a single job

Input formats

• Raw amino acid sequence
• FASTA format (single or multi-sequence)
• UniProt accession codes

Embedding limitations

Accuracy considerations

• Short sequences: Peptides under 30 residues may produce lower-quality embeddings due to insufficient context
• Non-natural sequences: Engineered or synthetic sequences not in the training distribution may have less meaningful embeddings
• Post-translational modifications: PTMs are not represented; glycosylated or phosphorylated proteins are embedded as their unmodified sequences

Memory and performance

• Sequence length: Memory scales quadratically with sequence length for attention computation
• Model size: Larger model variants produce better embeddings but require more GPU memory
• Batch size: Larger batches are more efficient but consume more memory

Performance characteristics

Speed and scaling

• Typical runtime: 5-30 seconds per sequence depending on length and model size
• Batch processing: Efficient batched inference for multiple sequences
• GPU acceleration: Required for reasonable performance; CPU inference is impractical for most use cases

Deterministic behavior

• Reproducible: Same sequence and model always produce identical embeddings
• Version sensitivity: Different model versions or sizes produce different embedding spaces; do not mix embeddings from different models

Chat usage

Example prompts

• "Generate protein embeddings for this sequence using ESM-3"
• "Run ESM-3 on my BLAST hits to compute similarity embeddings"
• "Compare these two proteins using ESM-3 embeddings"
• "Predict the function of this novel protein with ESM-3"
• "Cluster my protein dataset using ESM-3 embeddings"

Workflow chaining

• Feeds into: DeepChem (embeddings as features), custom ML models, clustering pipelines
• Accepts from: UniProt (sequences), BLAST (hit sequences), ESMFold (sequence extraction)

Read next