BioBERT limits & performance

This page helps you plan around BioBERT's input and throughput constraints — the 512-token segment limit, batch and file size ceilings, and how task type, document length, and entity density drive processing time. Document length and batch size most affect how long a job takes.

Text limits

BioBERT uses a BERT-based architecture with specific input constraints:

• Token limit: 512 tokens per segment (WordPiece tokenization)
• Character limit: Approximately 2,000-3,000 characters per segment depending on vocabulary coverage
• Long documents: Automatically split with 64-token overlap to preserve context across boundaries
• Batch size: Up to 1,000 documents per job (abstracts or paragraphs)
• File size: 50MB maximum upload per file
• Total job size: 10,000 documents maximum across all input files

Token counting

BioBERT uses WordPiece tokenization where:

• Common biomedical terms may be single tokens (e.g., "cancer" → 1 token)
• Rare terms get split (e.g., "pembrolizumab" → 4 tokens)
• Average PubMed abstract: ~200-300 tokens

Throughput

Processing times depend on task complexity and hardware:

Factors affecting speed

• Document length: Longer documents = more segments = longer processing
• Entity density: Text with many entities takes longer for NER
• Relationship complexity: Multiple entity pairs increase RE time
• Queue depth: Wait times vary with platform load

GPU acceleration

BioBERT inference leverages GPU acceleration when available:

• GPU inference: 5-10× faster than CPU for large batches
• Batch optimization: Larger batches benefit more from GPU parallelism
• Memory requirements: ~4GB GPU memory for standard inference
• CPU fallback: Automatic fallback if GPU unavailable (slower but functional)

Quotas and rate limits

• API rate limit: 60 requests/minute per user
• Concurrent jobs: Maximum 5 simultaneous BioBERT jobs
• Daily quota: Varies by workspace tier
• Polling frequency: Minimum 5-second intervals recommended

Optimization tips

• Use batch processing: Submit multiple documents in one job rather than individual requests
• Pre-filter text: Remove irrelevant sections (references, acknowledgments) to reduce processing
• Set appropriate confidence thresholds: Higher thresholds (>0.8) reduce noise; lower thresholds (<0.6) increase recall
• Choose relevant entity types: Only request entity types you need to reduce output size
• Segment long documents: Pre-split very long documents at natural boundaries (paragraphs, sections)
• Cache results: Store processed documents to avoid redundant API calls

Memory considerations

• Input buffering: Large files are streamed, not fully loaded
• Output size: Dense annotation can produce outputs larger than inputs
• Download timeouts: Large result sets may need chunked downloads