Run BLAST across many query files in one job and package the results for storage.

Run BLAST across many query files in one run and package the results for storage. The concurrency setting (-P in the runner) is what most affects throughput — tune it to the vCPUs available so parallel searches don't oversubscribe the machine.

Where these commands run

You cannot open a shell on the platform yourself. The commands on this page are BLAST+ command lines: ask Ivy in chat to run them in the chat's code workspace, which ships BLAST+, or run them on your own machine. For a hosted search with no commands, see the BLAST quickstart.

Batch runner script (bash)

#!/usr/bin/env bash
set -euo pipefail

DB="inputs/db/genomes_v1"
OUT_DIR="outputs/batch"
mkdir -p "$OUT_DIR"

find inputs/sequences -name "*.fasta" -print0 | xargs -0 -n1 -P4 -I{} bash -c '
  file="$1"
  base=$(basename "$file" .fasta)
  blastn \
    -query "$file" \
    -db '"'"'$DB'"'"' \
    -out '"'"'$OUT_DIR'"'"'/"$base".tsv \
    -outfmt "6 qseqid sseqid pident length evalue bitscore" \
    -max_target_seqs 5
' _ {}

Adjust -P4 to control concurrency.
Emit a per-file TSV to keep failures isolated.

Capture run metadata

Write a SUMMARY.json next to outputs:

python - <<'PY'
import json, os, subprocess, hashlib

def md5(path: str) -> str:
    h = hashlib.md5()
    with open(path, "rb") as f:
        h.update(f.read())
    return h.hexdigest()

summary = {
    "blast_version": subprocess.check_output(["blastn", "-version"], text=True).strip(),
    "database": "inputs/db/genomes_v1",
    "database_checksums": [md5(p) for p in sorted(os.listdir("inputs/db")) if p.startswith("genomes_v1")],
    "outputs": sorted(os.listdir("outputs/batch")),
}

with open("outputs/batch/SUMMARY.json", "w", encoding="utf-8") as handle:
    json.dump(summary, handle, indent=2)
PY

Export results for S3

tar -czf outputs/blast_results.tar.gz -C outputs/batch .

Upload the tarball; keep SUMMARY.json to describe the run.

Batch jobs & exports

Batch runner script (bash)

Capture run metadata

Export results for S3

On this page