Transformers on the platform

Transformers mines biomedical text for structured knowledge — extracting drugs, genes, and diseases as named entities, and the relationships between them, using Hugging Face models pre-trained on the life-science literature. Reach for it when you need to turn unstructured papers or abstracts into entities and relations, whether for target identification, drug repurposing, or knowledge-graph construction. It pairs naturally with literature retrieval from genomics database links and PubMed, extracting compound–target–disease relationships from the text those searches return. See what you can achieve for the full set.

What Transformers does

The Transformers tool provides access to domain-specific Hugging Face transformer models for biomedical text mining and natural language processing. These models are pre-trained on biomedical literature (PubMed, PMC) and chemical databases, giving them superior performance on life science text compared to general-purpose language models.

Citations

Lee J, Yoon W, Kim S, Kim D, Kim S, So CH, Kang J. BioBERT: a pre-trained biomedical language representation model for biomedical text mining. Bioinformatics. 2020;36(4):1234-1240. doi: 10.1093/bioinformatics/btz682

Gu Y, Tinn R, Cheng H, Lucas M, Usuyama N, Liu X, Naumann T, Gao J, Poon H. Domain-specific language model pretraining for biomedical natural language processing. ACM Trans Comput Healthcare. 2022;3(1):1-23. doi: 10.1145/3458754

Key achievements

• Domain specificity: Pre-trained on millions of biomedical abstracts and full-text articles, capturing terminology, abbreviations, and relationships specific to life sciences
• Entity recognition: Achieves >90% F1 score on biomedical NER benchmarks for drug, gene, and disease entity extraction
• Multi-task support: Single model platform supports NER, relation extraction, text classification, and embedding generation
• Chemical awareness: ChemBERTa models understand chemical nomenclature, IUPAC names, and structure-text relationships

Core capabilities

The platform's transformers_inference job supports six tasks: embeddings, ner, classification, qa (question answering), summarization, and zero_shot classification. There is no separate relation-extraction task — infer relationships from co-occurring entities or route the text through the orchestrator LLM instead.

• Named entity recognition (NER): Extract drugs, genes, proteins, diseases, chemicals, mutations, and cell lines from scientific text
• Text classification: Categorize documents by topic, sentiment, or relevance (e.g., classify abstracts as relevant to a therapeutic area), including zero-shot classification against labels you supply
• Text embeddings: Generate dense vector representations for document similarity, semantic search, and clustering
• Question answering: Extract answers to biomedical questions from document context
• Summarization: Condense long documents into a shorter summary

Available models

model_name takes any Hugging Face model ID (default dmis-lab/biobert-base-cased-v1.2) — it is a free-text field, not a fixed list. Commonly used biomedical IDs:

Technical approach

The models use BERT-style transformer architectures with domain-specific pre-training:

1. Pre-training: Models are trained on large biomedical corpora using masked language modeling (MLM), learning contextual word representations
2. Task adaptation: Pre-trained models are fine-tuned on labeled datasets for specific tasks (NER, RE, classification)
3. Inference: Input text is tokenized, processed through the transformer, and decoded into task-specific outputs (entity spans, relation triples, class labels, embeddings)

When to use Transformers

Primary use cases

Literature mining

• Entity extraction: Extract all mentioned drugs, genes, and diseases from a set of research papers
• Trend analysis: Identify emerging therapeutic targets or drug candidates across recent publications
• Knowledge graph construction: Build structured knowledge from unstructured text by extracting entities and relationships

Drug discovery intelligence

• Target identification: Mine literature for gene-disease associations to identify novel therapeutic targets
• Drug repurposing: Find literature evidence for new indications of existing drugs
• Safety signal detection: Extract adverse event mentions from clinical literature and case reports
• Competitor analysis: Track publications mentioning specific drugs, targets, or therapeutic areas

Data enrichment

• Annotation automation: Automatically annotate datasets with standardized entity identifiers (gene IDs, drug IDs, disease IDs)
• Abstract triage: Classify and prioritize abstracts for systematic review inclusion
• Semantic search: Find documents semantically similar to a query, even without keyword overlap

Webapp integration and workflow

Input methods in .bloom*

Direct text entry:

• Paste abstracts, paragraphs, or full documents
• Support for plain text and basic formatting

PubMed integration:

• Enter PubMed IDs (PMIDs) for automatic abstract retrieval
• Batch processing of multiple PMIDs

From previous jobs:

• Import text from literature search results
• Chain from web search or database query outputs

File upload:

• Upload text files (.txt) with one document per line
• CSV with a text column for batch processing

Integration with .bloom* ecosystem

Upstream data sources

• PubMed search: Retrieve abstracts for NER and relation extraction
• Web search: Gather online scientific content for analysis
• Literature passport: Feed text from comprehensive literature surveys

Downstream applications

• Target Passport: Extracted gene-disease associations feed into target assessment
• Compound Passport: Drug entity mentions contribute to compound profiling
• Knowledge graphs: Extracted relations populate the project knowledge base

Best paired with

• PubMed search: Retrieve relevant literature, then extract entities and relations
• Target Passport: Use extracted gene-disease evidence for target validation
• Compound Passport: Integrate drug mentions and safety signals into compound assessment
• Literature Passport: Comprehensive literature analysis with entity-level detail

Scope and limitations

Supported inputs

Text requirements

• Language: English biomedical text only
• Length: Up to 512 tokens per input (approximately 300-400 words); longer documents are automatically chunked
• Format: Plain text, PubMed abstracts, or formatted paragraphs

Batch processing

• Up to 100 documents per batch job
• Processing time scales linearly with document count and length

NLP limitations

Accuracy considerations

• Entity boundaries: Complex multi-word entities (e.g., "recombinant human erythropoietin") may have boundary errors
• Abbreviations: First occurrence should be defined; models handle common abbreviations but may miss novel ones
• Negation: "not associated with BRCA1" may still extract BRCA1 as an entity; context interpretation is limited
• Co-reference: Pronouns and anaphoric references across sentences may not be resolved

Domain coverage

• Models perform best on biomedical and chemical text; performance degrades on clinical notes, patents, or regulatory documents
• Rare entities (novel genes, unpublished drug names) may not be recognized
• Non-English text is not supported

Chat usage

Example prompts

• "Extract drug and gene entities from this abstract"
• "Find all drug-target interactions mentioned in PMID:12345678"
• "Classify these abstracts as relevant or irrelevant to EGFR inhibitors"
• "Generate embeddings for these PubMed abstracts for clustering"

Workflow chaining

• Feeds into: Target Passport, Compound Passport, Literature Passport, knowledge graphs
• Accepts from: PubMed search, web search, literature databases

Read next