Skip to main content
    Ivy Biosciences logo
    Ivy Biosciences.bloom* docs
    login.bloom*
    Ivy Biosciences Docs/Call the platform/Cli/Batch Processing

    Batch Processing

    Submit, monitor, and manage batch job collections with parallel execution for high-throughput screening.

    Open in Claude(opens in a new tab)Open in ChatGPT(opens in a new tab)3 min read
    batchparallelscreeningautomation

    Difficulty

    intermediate

    Prerequisites

    Authenticated CLI (ivybloom auth login), Familiarity with job submission (ivybloom run)

    The ivybloom CLI is the scriptable surface for the platform, and batch processing is how you submit, monitor, and manage whole collections of jobs with parallel execution. Reach for batch instead of individual run commands when a script needs high-throughput screening — running many jobs at once and managing them as a single unit you can pause, resume, or cancel. It's the workhorse for reproducible, large-scale pipelines; for the full command surface, see the CLI overview.

    Batch Processing

    Submit, monitor, and manage batch job collections with parallel execution for high-throughput screening.

    batch is a hidden command: ivybloom batch runs it, but ivybloom --help does not list it. Run ivybloom batch --help to see its subcommands.

    When to Use Batch

    Use batch instead of individual run commands when you need to:

    • Screen a compound library through ADMET prediction
    • Dock multiple ligands against a protein target
    • Fold a family of related protein sequences
    • Run any tool across many inputs with parallel execution

    A single batch submit replaces dozens of manual run invocations with built-in parallelism, checkpointing, and result aggregation.

    Batch File Format

    Batch jobs are defined in YAML or JSON files. Here is the full schema:

    name: "ADMET Screen"
description: "Screen drug candidates for ADMET properties"
tool: admet                          # Default tool for all jobs
variables:                           # ${var} substitution
  target_ph: "7.4"
  project: "proj_abc123"
defaults:                            # Shared params for all jobs
  timeout: 3600
jobs:
  - name: "aspirin"
    parameters: { smiles: "CC(=O)Oc1ccccc1C(=O)O" }
  - name: "ibuprofen"
    parameters: { smiles: "CC(C)Cc1ccc(cc1)C(C)C(=O)O" }
  - name: "caffeine"
    parameters: { smiles: "Cn1c(=O)c2c(ncn2C)n(C)c1=O" }
options:
  parallel: 4                        # Max concurrent workers (default: 3)
  stop_on_error: false               # Continue on individual failures
  retry_failed: 2                    # Retry failed jobs up to N times
  checkpoint: true                   # Enable checkpointing for resume
  output_dir: "./results"            # Output directory for artifacts

    Variable Substitution

    Use ${var} syntax in parameters to reference variables defined at the top level:

    variables:
  receptor: "1ABC"
  ph: "7.4"
jobs:
  - name: "dock-aspirin"
    parameters:
      receptor_pdb: "${receptor}"
      ph: "${ph}"
      ligand_smiles: "CC(=O)Oc1ccccc1C(=O)O"

    Variables can be overridden at submit time with the --var flag.

    Per-Job Tool Overrides

    Each job can override the default tool:

    tool: admet
jobs:
  - name: "fold-first"
    tool: esmfold                    # Override default tool
    parameters: { protein_sequence: "MKLVFF..." }
  - name: "admet-aspirin"
    parameters: { smiles: "CC(=O)Oc1ccccc1C(=O)O" }

    Submitting Batches

    # Basic submission
ivybloom batch submit screen.yaml

# With parallelism override
ivybloom batch submit screen.yaml --parallel 8

# Wait for all jobs to complete
ivybloom batch submit screen.yaml --wait

# Dry-run: validate the batch file without submitting
ivybloom batch submit screen.yaml --dry-run

# Override variables at submit time
ivybloom batch submit screen.yaml --var target_ph=8.0 --var receptor=2XYZ
    --parallelintdefault: 3

    Maximum number of concurrent jobs (-P shorthand).

    --waitbool

    Block until all jobs complete (-w shorthand).

    --dry-runbool

    Validate the batch file and parameters without submitting (-d shorthand).

    --varstring

    Override a variable: --var key=value. Can be repeated (-v shorthand).

    Monitoring

    Check Status

    ivybloom batch status BATCH_ID

    Real-Time Monitor

    The monitor subcommand provides a live TUI dashboard showing job progress:

    # Real-time dashboard (default: poll every 3s)
ivybloom batch monitor BATCH_ID

# Custom poll interval
ivybloom batch monitor BATCH_ID --interval 5

# JSON stream output (for piping)
ivybloom batch monitor BATCH_ID --json
    --intervalintdefault: 3

    Poll interval in seconds.

    --jsonbool

    Output JSON stream instead of TUI dashboard.

    Listing Batches

    ivybloom batch list
ivybloom batch list --format json --limit 20

    Execution Control

    Pausing

    # Pause execution (running jobs continue, no new jobs start)
ivybloom batch pause BATCH_ID

# Pause and cancel all pending jobs
ivybloom batch pause BATCH_ID --cancel-pending

    Resuming

    # Resume a paused batch
ivybloom batch resume BATCH_ID

# Re-run all jobs (not just pending/failed)
ivybloom batch resume BATCH_ID --force-all

# Resume from a specific job index
ivybloom batch resume BATCH_ID --from-index 5

# Override variables on resume
ivybloom batch resume BATCH_ID --var target_ph=8.0
    --force-allbool

    Re-run all jobs, not just pending or failed ones.

    --from-indexint

    Resume from a specific job index (0-based).

    --varstring

    Override variables on resume.

    Cancelling

    # Cancel specific jobs within a batch
ivybloom batch cancel JOB_ID1 JOB_ID2

    Collecting Results

    Individual Job Results

    # Download results for specific jobs
ivybloom batch results JOB_ID1 JOB_ID2 --format json
ivybloom batch results JOB_ID1 --output-dir ./out/
    --formatstringdefault: table

    Output format: table, json, csv, yaml (-f shorthand).

    --output-dirstring

    Directory to save result files (-o shorthand).

    Aggregate Batch Results

    The collect command aggregates results from all jobs in a batch:

    # Collect all results as JSON
ivybloom batch collect BATCH_ID --output results.json

# CSV format, completed jobs only
ivybloom batch collect BATCH_ID --format csv --completed-only

# Filter by confidence threshold
ivybloom batch collect BATCH_ID --confidence 0.8

# Include in-progress partial results
ivybloom batch collect BATCH_ID --partial
    {
  "batch_id": "batch_abc123",
  "total": 3,
  "completed": 2,
  "failed": 0,
  "results": [
    {"job": "aspirin", "status": "completed", "score": 0.92},
    {"job": "ibuprofen", "status": "completed", "score": 0.87},
    {"job": "caffeine", "status": "running"}
  ]
}
    --outputstring

    Output file path (-o shorthand).

    --formatstringdefault: json

    Output format: json, csv (-f shorthand).

    --completed-onlybool

    Only include completed jobs.

    --confidencefloat

    Minimum confidence threshold for results (-c shorthand).

    --partialbool

    Include partial results from in-progress jobs.

    Batch Execution Flow

    Submitted jobs run in a pool of N parallel slots; monitor, pause/resume and collect operate on the batch.

    CI/CD Integration

    For CI/CD automation, add --agent for structured JSON output. See Agent Mode for the full envelope schema.

    GitHub Actions Example

    - name: Run ADMET screen
  run: |
    BATCH_ID=$(ivybloom batch submit screen.yaml --wait --agent | jq -r '.data.batch_id')
    ivybloom batch collect $BATCH_ID --format csv --output results.csv
  env:
    IVYBLOOM_API_KEY: ${{ secrets.IVYBLOOM_API_KEY }}

    Exit Codes

    CodeMeaning
    0All jobs completed successfully
    1One or more jobs failed

    Examples

    Screen 100 Molecules Through ADMET

    # admet-screen.yaml
name: "ADMET Library Screen"
tool: admet
options:
  parallel: 8
  checkpoint: true
jobs:
  - name: "mol-001"
    parameters: { smiles: "CC(=O)Oc1ccccc1C(=O)O" }
  - name: "mol-002"
    parameters: { smiles: "CC(C)Cc1ccc(cc1)C(C)C(=O)O" }
  # ... up to 100 entries
    ivybloom batch submit admet-screen.yaml --wait
ivybloom batch collect BATCH_ID --format csv --output admet-results.csv

    Dock a Ligand Library Against a Target

    # docking-screen.yaml
name: "DiffDock Library Screen"
tool: diffdock
defaults:
  receptor_pdb_id: "1ABC"
options:
  parallel: 4
jobs:
  - name: "aspirin"
    parameters: { ligand_smiles: "CC(=O)Oc1ccccc1C(=O)O" }
  - name: "ibuprofen"
    parameters: { ligand_smiles: "CC(C)Cc1ccc(cc1)C(C)C(=O)O" }

    Fold a Protein Family

    # family-fold.yaml
name: "Kinase Family Fold"
tool: esmfold
options:
  parallel: 3
jobs:
  - name: "EGFR"
    parameters: { protein_sequence: "MRPSGTAGAALLALLAALCPA..." }
  - name: "BRAF"
    parameters: { protein_sequence: "MAALSGGGGGGAEPGQALFNG..." }
  - name: "ABL1"
    parameters: { protein_sequence: "MLEICLKLVGCKSKKGLSSSS..." }
    • Job Management — Monitor individual jobs within a batch
    • Compounds — Build compound libraries for batch screening
    • Recipes — End-to-end workflow tutorials using batch
    • Agent Mode — Automate batch workflows in CI/CD

    Was this page helpful?

    CLI API Keys & Authentication

    Manage authentication and API keys for CLI automation and integrations.

    Billing & Credits

    Track usage, manage subscriptions, and monitor credit spending from the command line.

    On this page

    On this page

    Loading...