---
title: ESM-3 on the platform
description: Embed a sequence with ESM-3 for alignment-free similarity, zero-shot function prediction and clustering — per-residue and per-sequence vectors, not a fold.
tags:
  - esm3
---

ESM-3 is a protein language model for design and analysis, not folding: it produces embedding vectors that encode sequence, structure, and function, and it can generate novel proteins. Reach for it when you want alignment-free similarity, zero-shot function prediction, clustering, or ML features — the tasks that come before or after structure prediction rather than the fold itself. If you need the 3D coordinates of a sequence, [predict them with ESMFold](/docs/esmfold) or [fetch them with AlphaFold](/docs/alphafold) instead. See [what you can achieve](/docs/capabilities#structure) for the full set.

## What ESM-3 does

ESM-3 is a multimodal protein language model, a generative model that reasons over the **sequence**, **structure**, and **function** of proteins simultaneously. It produces high-dimensional embedding vectors that encode deep evolutionary and functional relationships learned from training on billions of natural protein sequences.

## Citations

> Hayes T, Rao R, Akin H, Sofroniew NJ, Oktay D, Lin Z, Verkuil R, Tran VQ, Deaton J, Wiggert M, et al. Simulating 500 million years of evolution with a language model. *Science*. 2025 Feb 21;387(6736):850-858. doi: [10.1126/science.ads0018](https://doi.org/10.1126/science.ads0018)

## Key achievements

- **Multimodal architecture**: Jointly models sequence, structure, and function tracks in a single transformer, enabling cross-modal reasoning
- **Scale**: Trained on billions of protein sequences with up to 98 billion parameters in the largest variant
- **Generative capability**: Can generate novel proteins by conditioning on any combination of sequence, structure, and function inputs
- **Function prediction**: Zero-shot function annotation from embeddings alone, without requiring homology search

## Core capabilities

- **Sequence embeddings**: Produce per-residue (L x D) and per-sequence (1 x D) embedding vectors for downstream ML tasks
- **Zero-shot function prediction**: Predict protein function from embeddings without fine-tuning, using learned representations of functional annotations
- **Similarity search**: Compute cosine similarity between embedding vectors for fast, alignment-free protein comparison
- **Evolutionary analysis**: Embedding space captures evolutionary relationships, enabling clustering and phylogenetic inference
- **Contact prediction**: Attention maps encode predicted residue-residue contacts for structural insight
- **Embedding-based clustering**: Group proteins by functional or structural similarity using embedding distance metrics

## Technical approach

ESM-3 uses a **transformer architecture** with discrete tokenization of three modalities:

1. **Sequence track**: Amino acid tokens processed through the transformer encoder
2. **Structure track**: Structural tokens derived from local structural neighborhoods (VQ-VAE encoded)
3. **Function track**: Functional annotation tokens from InterPro, GO terms, and other annotation databases

The model is trained with a generative masked language modeling objective across all three tracks, learning to predict masked tokens conditioned on the unmasked context. This enables:
- **Embedding extraction**: Use internal representations as feature vectors
- **Conditional generation**: Generate sequence given structure, or function given sequence
- **Cross-modal transfer**: Structure information improves function prediction, and vice versa

## When to use ESM-3

## Primary use cases

### Protein similarity and clustering

- **Alignment-free comparison**: Compare proteins without sequence alignment; embeddings capture remote homology that BLAST misses
- **Large-scale clustering**: Cluster millions of sequences by embedding similarity in minutes
- **Functional grouping**: Group proteins by predicted function rather than sequence identity

### Function prediction and annotation

- **Novel protein annotation**: Predict function for proteins without detectable homologs
- **Gene ontology prediction**: Infer GO terms from embedding representations
- **Enzyme classification**: Classify proteins into functional categories using embedding-based classifiers

### Feature extraction for machine learning

- **Transfer learning**: Use ESM-3 embeddings as input features for custom ML models (activity prediction, localization, stability)
- **Representation learning**: Pre-computed embeddings eliminate the need to train protein representations from scratch
- **Multi-task models**: Combine embeddings with other molecular descriptors for comprehensive property prediction

### Evolutionary and structural analysis

- **Evolutionary distance**: Embedding distances correlate with evolutionary divergence
- **Contact prediction**: Attention maps provide residue-residue contact predictions
- **Variant effect prediction**: Compare wild-type and mutant embeddings to estimate functional impact

## Webapp integration and workflow

### Input methods in .bloom*

**Direct sequence entry:**
- Paste amino acid sequences in raw or FASTA format
- Real-time validation with length and composition checks
- Support for batch submission of multiple sequences

**UniProt integration:**
- Direct UniProt accession lookup (e.g., "P12345")
- Automatic sequence fetching with metadata
- Isoform selection for proteins with multiple variants

**From previous jobs:**
- Import sequences from BLAST search results
- Chain from ESMFold, fragment extraction, or other sequence analysis jobs
- Provenance tracking maintains connection to original data source

## Integration with .bloom* ecosystem

### Upstream data sources

- **UniProt**: Sequence fetch for known proteins
- **BLAST**: Hit sequences from homology searches
- **ESMFold**: Predicted structures (sequence extraction)

### Downstream applications

- **DiffDock**: Feed embedding-based target characterization into docking workflows
- **P2Rank**: Pocket prediction on embedding-informed targets
- **DeepChem**: Use ESM-3 embeddings as protein features for binding prediction models
- **Custom ML**: Build protein property classifiers using embedding vectors

## Best paired with

- **ESMFold**: Use ESM-3 for embeddings and ESMFold for structure prediction; complementary tools from the same model family
- **BLAST**: Combine alignment-based and embedding-based similarity for comprehensive homology analysis
- **DeepChem**: Use embeddings as features in molecular property prediction pipelines

## Scope and limitations

## Supported inputs

### Sequence requirements
- **Standard amino acids**: 20 canonical amino acids (ACDEFGHIKLMNPQRSTVWY)
- **Length range**: 10-2,048 tokens (shorter sequences may lack context for high-quality embeddings)
- **Single chain**: Each sequence is embedded independently
- **Batch support**: Multiple sequences can be submitted in a single job

### Input formats
- Raw amino acid sequence
- FASTA format (single or multi-sequence)
- UniProt accession codes

## Embedding limitations

### Accuracy considerations
- **Short sequences**: Peptides under 30 residues may produce lower-quality embeddings due to insufficient context
- **Non-natural sequences**: Engineered or synthetic sequences not in the training distribution may have less meaningful embeddings
- **Post-translational modifications**: PTMs are not represented; glycosylated or phosphorylated proteins are embedded as their unmodified sequences

### Memory and performance
- **Sequence length**: Memory scales quadratically with sequence length for attention computation
- **Model size**: Larger model variants produce better embeddings but require more GPU memory
- **Batch size**: Larger batches are more efficient but consume more memory

## Performance characteristics

### Speed and scaling
- **Typical runtime**: 5-30 seconds per sequence depending on length and model size
- **Batch processing**: Efficient batched inference for multiple sequences
- **GPU acceleration**: Required for reasonable performance; CPU inference is impractical for most use cases

### Deterministic behavior
- **Reproducible**: Same sequence and model always produce identical embeddings
- **Version sensitivity**: Different model versions or sizes produce different embedding spaces; do not mix embeddings from different models

## Chat usage

### Example prompts

- "Generate protein embeddings for this sequence using ESM-3"
- "Run ESM-3 on my BLAST hits to compute similarity embeddings"
- "Compare these two proteins using ESM-3 embeddings"
- "Predict the function of this novel protein with ESM-3"
- "Cluster my protein dataset using ESM-3 embeddings"

### Workflow chaining

- **Feeds into**: DeepChem (embeddings as features), custom ML models, clustering pipelines
- **Accepts from**: UniProt (sequences), BLAST (hit sequences), ESMFold (sequence extraction)

## Read next