---
title: Batch processing patterns
description: Run Biotite over many sequence or structure files — a timestamped run folder, xargs parallelism, per-file JSON outputs, retries that skip finished files.
---

Batch processing lets you run Biotite over many sequence or structure files in one reproducible run. A clean run layout, per-file outputs, and frequent checkpointing are what most affect whether a large batch finishes cleanly and stays easy to audit.

> **Where this code runs**

> You cannot open a shell on the platform yourself. The scripts on this page are plain Biotite Python and shell: ask Ivy in chat to run them in the chat's [code workspace](/docs/sandbox), which ships Biotite, or run them on your own machine.

## Recommended run layout

```
biotite-job/
├─ inputs/
│  ├─ sequences/
│  └─ structures/
├─ outputs/
│  └─ run-<timestamp>/
├─ scripts/
│  ├─ parse_sequences.py
│  └─ process_structures.py
└─ logs/
   └─ run.log
```

Use a unique `run-<timestamp>/` for each submission to avoid overwriting
results.

## Streaming batches

```bash
find inputs/sequences -name "*.fasta" -print0 |
  xargs -0 -n1 -P4 -I{} python scripts/parse_sequences.py \
    "{}" "outputs/run-$(date +%s)/$(basename "{}").json"
```

- Tune `-P` for concurrency.
- Keep per-file outputs to isolate failures.

## Minimal sequence parser (example)

```python
from pathlib import Path
from biotite.sequence.io import load_sequence
import json
import sys

def main(in_path: str, out_path: str) -> None:
    record = load_sequence(in_path, format=None)
    seq = record.get_sequence()
    data = {
        "id": list(record.keys())[0],
        "length": len(seq),
        "gc": seq.get_gc_content(),
    }
    Path(out_path).parent.mkdir(parents=True, exist_ok=True)
    with open(out_path, "w", encoding="utf-8") as handle:
        json.dump(data, handle, indent=2)

if __name__ == "__main__":
    main(sys.argv[1], sys.argv[2])
```

## Checkpointing and retries

- Write partial outputs frequently; avoid holding results in memory.
- Keep a `logs/run.log` with start/end timestamps per file.
- On retry, skip outputs that already exist unless a `--force` flag is passed.

## Packaging outputs

```bash
tar -czf outputs/biotite_batch.tar.gz -C outputs/run-$(date +%s) .
```

Include a `SUMMARY.json` (counts, skipped files, errors) for downstream
auditing.