Rule Set 3 vs Rule Set 2 vs DeepSpCas9: Which sgRNA Score to Trust

If you are picking one SpCas9 on-target model today, use Rule Set 3 (DeWeirdt et al., Nature Communications 2022): it is the only one of the three that conditions its prediction on the tracrRNA variant you actually use, and it was validated against a new dataset of sgRNAs tiling essential and non-essential genes where it improved substantially on prior prediction models. Rule Set 2 (Doench et al., Nature Biotechnology 2016) is its direct predecessor and still the most widely embedded score in older design tools. DeepSpCas9 (Kim et al., Science Advances 2019) is trained on a different measurement — indel frequency at 12,832 target sequences in a human cell library — and is the better reference when your read-out is editing at a single locus rather than depletion in a pooled screen. All three answer the same narrow question: given this 20-nt spacer and its immediate context, how active is it likely to be? None of them answers whether the guide is unique in your genome.

The three models at a glance#

Two things in that table do most of the work. First, the models are not three attempts at one number; they are three regressions onto three different experimental read-outs, which is why they disagree and why "which is best" is badly posed without naming your read-out. Second, every one of them is an SpCas9 model. A score produced for a 20-nt spacer with an NGG PAM does not transfer to SaCas9 or Cas12a, and a tool that silently rescales it for another nuclease is reporting a category error.

What each model is actually trained on#

Rule Set 2 (Doench 2016)#

Rule Set 2 came out of a programme that built human and mouse genome-wide libraries, ran positive and negative selection screens with them, and folded the resulting large-scale empirical data back into the computational design rules (Doench et al., 2016). The same paper profiled the off-target activity of thousands of sgRNAs and produced a separate metric for predicting off-target sites — the CFD score. That pairing matters for how the two numbers are often misused: they were published together, but they are different models answering different questions, and the on-target one knows nothing about the genome.

Rule Set 3 (DeWeirdt 2022)#

Rule Set 3's starting observation is a gap in every model before it: multiple tracrRNA variants are commonly used for screening, and no existing model accounted for that feature when nominating sgRNAs (DeWeirdt et al., 2022). The authors built an on-target model that makes optimal predictions for multiple tracrRNA variants, validated it on a new dataset of sgRNAs tiling essential and non-essential genes, and reported substantial improvement over prior prediction models. Analysing the activity differences between tracrRNA variants, they showed that Pol III transcription termination is a strong determinant of sgRNA activity.

The practical consequence is a parameter, not a caveat. The reference implementation gpp-rnd/rs3 exposes the tracrRNA as a model input (sequence_tracr), with the two variants it was trained across selectable by name. If you score a library under one tracrRNA and clone it into a vector carrying the other, you have changed an input the model reads — the ranking you acted on was computed for a different construct. This is the single most common way a Rule Set 3 score gets quietly invalidated, and it is invisible if the design tool hides the parameter.

DeepSpCas9 (Kim 2019)#

DeepSpCas9 is trained on a directly measured editing read-out rather than a selection phenotype: SpCas9 activity at 12,832 target sequences, assayed with a human cell library containing paired sgRNA-encoding and target sequences, with deep learning applied to the resulting indel frequencies (Kim et al., 2019). Tested against independently generated datasets, including ones published by other groups, it showed high generalization performance. Because its label is indel frequency, it is the most natural reference when you intend to validate with amplicon sequencing at one site.

Do sgRNA scores predict knockout efficiency?#

Partly, and the honest version of the answer depends on which measurement you call "efficiency".

• Indel frequency by amplicon NGS. This is the closest thing to a direct read-out of cutting at your locus, and it is the label DeepSpCas9 was trained on (Kim et al., 2019). An on-target score is a prior over this quantity for a guide you have not yet tested; it is not a substitute for measuring it.
• Depletion in a pooled screen. This is the signal behind Rule Set 2 and Rule Set 3, and it is a composite: cutting efficiency, frameshift outcome, protein half-life, and how essential the gene is in your cell line all enter it. A guide can cut well and deplete weakly because the gene is not required under your selection.
• Protein loss by western blot. Neither a score nor an indel rate predicts this cleanly. An in-frame indel, a re-initiated start, or a truncation that retains function all break the chain between a cut and a null allele.

So the usable rule is comparative, not absolute. These models rank candidate spacers inside a region against each other. They do not tell you that a given guide will reach a particular editing percentage in your cell line, under your delivery method, at your chromatin state — none of which is an input to any of the three models. Treat the score as the order in which to test guides, and keep the measurement.

A corollary for screens: if your question is which genes scored, the model that designed the library is no longer the relevant statistic. Hit calling is a separate inference over counts, which is what MAGeCK does — prioritising sgRNAs, genes and pathways from genome-scale knockout screen counts (Li et al., Genome Biology 2014; liulab-dfci/MAGeCK).

Why an on-target score is not off-target safety#

This is the most consequential confusion in guide design, and it is structural rather than a matter of degree. An on-target model takes a spacer and its immediate context and returns an activity estimate. Nothing in that computation looks at a genome, so nothing in its output can speak to whether the spacer has near-matches elsewhere. A guide can carry the best on-target score in a region and still have a perfect 20-nt match in a second locus.

Off-target assessment is a different class of computation with its own tools:

• Genome-wide in silico search. Cas-OFFinder searches a given genome, or user-defined sequences, for potential off-target sites; unlike earlier algorithms it is not limited by the number of mismatches and allows variation in the PAM recognised by Cas9 (Bae et al., Bioinformatics 2014). This is the external tool to reach for when you need a genome-wide candidate list. It is not offered on .bloom*.
• Position-dependent mismatch tolerance. Which near-matches matter is not simply a mismatch count; Hsu et al., Nature Biotechnology 2013 mapped the DNA targeting specificity of RNA-guided Cas9 nucleases, and the CFD metric published with Rule Set 2 encodes that kind of positional weighting (Doench et al., 2016).
• Empirical measurement. GUIDE-seq captures double-stranded oligodeoxynucleotides into Cas9-induced double-strand breaks to profile off-target cleavage genome-wide. Applied to 13 RNA-guided nucleases in two human cell lines, it found wide variability in off-target activity — and reported that the majority of the sites it identified were not detected by existing computational methods or by ChIP-seq (Tsai et al., Nature Biotechnology 2015).

Take that last finding seriously when reading any in silico off-target list: the computational layer is a triage step with a known false-negative rate against measurement, not a safety certificate. For any therapeutic or regulatory claim, the measurement is the evidence.

When to use which#

A reasonable default for a knockout experiment: rank with Rule Set 3 under your real tracrRNA, take the top handful rather than the single top guide, run a genome-wide off-target search on those candidates with an external tool, and test two or three at the bench with amplicon sequencing. The score chose the order; the assay chose the guide.

Limitations worth stating plainly#

• Context, not chromatin. All three models read sequence. Chromatin accessibility, methylation, copy number and delivery efficiency are not inputs, and they routinely dominate observed editing differences between loci.
• Scales are not comparable. Rule Set 3 scores are on the model's own scale. They are meaningful as a ranking among candidates, and a threshold borrowed from a 0–1 score published for a different model does not transfer.
• A score is not a measurement. An unvalidated guide with a high score is still an unvalidated guide.
• Screen phenotype ≠ cutting. Weak depletion in a screen can mean the gene is not essential in that line, not that the guide failed.
• Off-target lists are incomplete by construction. GUIDE-seq found most of its sites outside existing computational predictions (Tsai et al., 2015).

How to run it on .bloom*#

crispr_design_guides runs Rule Set 3 — the rs3 package (Apache-2.0, gpp-rnd/rs3), the published model rather than a composition heuristic. It can be run without an account. What it does, precisely:

1. You supply the sequence. target_sequence is required. The tool holds no genome index and does not resolve a gene symbol to its sequence, so fetch the sequence first with a sequence-database lookup; target_gene is a label on the output, not a query. The CRISPR quickstart walks the sequence-first order, and genomics lookups are where the sequence comes from.
2. It scans both strands for SpCas9 PAM sites, extracts the 20-nt protospacers, and scores each one with Rule Set 3 over the 30-nt context the model expects — 4 nt upstream, the 20-nt spacer, the 3-nt PAM, 3 nt downstream.
3. Guides too close to either end come back unscored. A protospacer whose 30-nt context runs off the end of your sequence returns a null score with a stated reason, and sorts after the scored guides. It is never scored off a padded sequence. The fix is to submit more flanking sequence.
4. Set the tracrRNA you are actually using. The tracr parameter takes Hsu2013 (the default) or Chen2013 and is passed to Rule Set 3 as its sequence_tracr input. This is the parameter most worth getting right, for the reason the paper gives.
5. SpCas9 only. cas_type accepts SpCas9. Another nuclease is refused rather than scored with the wrong model.
6. Read the ranking, not a threshold. Scores are on Rule Set 3's own scale and rank guides relative to each other within your submitted sequence. GC content is reported as bookkeeping and does not enter the score.

⚠ crispr_design_guides consults no reference genome. Every candidate comes from the sequence you submitted, so the result makes no genome-wide off-target claim and no uniqueness claim. For a genome-wide search, use Cas-OFFinder — an external tool, not offered on .bloom*.

Two neighbouring tools, with the same honesty constraint:

• crispr_offtarget scans for near-matches to a guide, up to 4 mismatches on both strands, inside the sequence you supply plus an optional user-supplied FASTA. It ranks hits by a mismatch-count and mismatch-position heuristic, not by the published CFD matrix. It is the right tool for a plasmid, amplicon or transgene; supplying a genome build makes the job refuse rather than imply genome-wide coverage. "No hits" means no near-match in what you submitted.
• crispr_screen_analyze runs MAGeCK itself (mageck test, mageck 0.5.9.5) on a guide-level count table and returns its gene-level and sgRNA-level statistics — RRA score, p-value, FDR, log fold change, rank and good-sgRNA counts. control_samples and treatment_samples name columns in your table and are never inferred. If the binary is absent the job fails rather than substituting an approximation.

Design guides for your target sequence, with the tracrRNA your vector actually carries, on the .bloom* platform.

Key takeaways#

• Rule Set 3 is the default on-target model to use for SpCas9, and the one crispr_design_guides runs. Its distinguishing feature is that the tracrRNA variant is a model input (DeWeirdt et al., 2022).
• Rule Set 2 is its predecessor; keep it for reproducing older rankings (Doench et al., 2016).
• DeepSpCas9 is trained on indel frequency at 12,832 target sequences and is the better cross-check when amplicon NGS is your read-out (Kim et al., 2019). It is not offered on .bloom*.
• Scores rank candidates; they do not promise an editing percentage, and screen depletion, indel frequency and protein loss are three different measurements.
• On-target score is not off-target safety. Genome-wide search is Cas-OFFinder; the measurement is GUIDE-seq, which found most of its sites outside existing computational predictions.
• Related reading: ESMFold vs AlphaFold2 for the same question asked of structure prediction — what a model's training label does and does not license you to claim.

References#

1. DeWeirdt, P.C., McGee, A.V., Zheng, F. et al. "Accounting for small variations in the tracrRNA sequence improves sgRNA activity predictions for CRISPR screening." Nature Communications (2022). doi:10.1038/s41467-022-33024-2
2. Doench, J.G., Fusi, N., Sullender, M. et al. "Optimized sgRNA design to maximize activity and minimize off-target effects of CRISPR-Cas9." Nature Biotechnology (2016). doi:10.1038/nbt.3437
3. Kim, H.K., Kim, Y., Lee, S. et al. "SpCas9 activity prediction by DeepSpCas9, a deep learning-based model with high generalization performance." Science Advances (2019). doi:10.1126/sciadv.aax9249
4. Bae, S., Park, J., Kim, J.-S. "Cas-OFFinder: a fast and versatile algorithm that searches for potential off-target sites of Cas9 RNA-guided endonucleases." Bioinformatics (2014). doi:10.1093/bioinformatics/btu048
5. Tsai, S.Q., Zheng, Z., Nguyen, N.T. et al. "GUIDE-seq enables genome-wide profiling of off-target cleavage by CRISPR-Cas nucleases." Nature Biotechnology (2015; published online 2014). doi:10.1038/nbt.3117
6. Hsu, P.D., Scott, D.A., Weinstein, J.A. et al. "DNA targeting specificity of RNA-guided Cas9 nucleases." Nature Biotechnology (2013). doi:10.1038/nbt.2647
7. Li, W., Xu, H., Xiao, T. et al. "MAGeCK enables robust identification of essential genes from genome-scale CRISPR/Cas9 knockout screens." Genome Biology (2014). doi:10.1186/s13059-014-0554-4
8. Rule Set 3 reference implementation: gpp-rnd/rs3 (Apache-2.0). MAGeCK source: liulab-dfci/MAGeCK.