BioBERT

BioBERT lets you mine biomedical text — extracting genes, proteins, diseases, chemicals, and the relationships between them from abstracts and papers. Reach for it when you want structured entities out of literature rather than a list of citations. It is the natural next step after its sibling PubMed: use PubMed to retrieve the papers, then BioBERT to pull out the drug–target–disease relationships. See what you can achieve.

BioBERT integration in .bloom* platform

BioBERT (Bidirectional Encoder Representations from Transformers for Biomedical Text Mining) is a domain-specific language representation model pre-trained on large-scale biomedical corpora. Developed by the DMIS Lab at Korea University, it significantly outperforms general-purpose BERT on biomedical NLP tasks.

Citations

Lee J, Yoon W, Kim S, Kim D, Kim S, So CH, Kang J. BioBERT: a pre-trained biomedical language representation model for biomedical text mining. Bioinformatics. 2020 Feb 15;36(4):1234-1240. doi: 10.1093/bioinformatics/btz682

Training data

BioBERT was pre-trained on massive biomedical corpora:

• 4.5 billion words from PubMed abstracts
• 13.5 billion words from PubMed Central (PMC) full-text articles

Performance improvements over BERT

BioBERT largely outperforms general BERT on key biomedical tasks:

Key capabilities

Biomedical Named Entity Recognition (NER)

Identify and classify biomedical entities in text:

• Genes/proteins: BRCA1, p53, insulin receptor
• Diseases: cancer, diabetes, Alzheimer's disease
• Chemicals/drugs: aspirin, doxorubicin, metformin
• Species: human, mouse, E. coli
• Mutations: c.248T>C, V600E

Relation Extraction (RE)

Extract relationships between entities:

• Drug-disease: Drug X treats disease Y
• Protein-protein: Protein A interacts with protein B
• Gene-disease: Mutation in gene X causes disease Y
• Drug-drug: Drug A interacts with drug B

Question Answering (QA)

Answer biomedical questions from text:

• "What drug treats condition X?"
• "What is the mechanism of action of drug Y?"
• "Which genes are associated with disease Z?"

Why domain-specific pre-training matters

General NLP models often fail on biomedical text because:

• Domain-specific vocabulary: Terms like "transcriptional", "antimicrobial"
• Complex nomenclature: Gene names (BRCA1), mutation notation (c.248T>C)
• Technical language: Scientific writing style and conventions
• Word distribution shift: Different word frequencies vs. general text

BioBERT's biomedical pre-training addresses these challenges.

Available model versions

Webapp integration

Input methods in .bloom*

Text input:

• Paste abstracts or article text directly
• Upload text files or PDFs
• Import from PubMed IDs

Batch processing:

• Process multiple documents in parallel
• CSV input with text column
• Progress tracking for large datasets

Output formats

NER results:

• Annotated text with entity spans
• Entity type classification
• Confidence scores per entity

RE results:

• Entity pairs with relationship types
• Evidence sentences
• Confidence scores

QA results:

• Extracted answers from text
• Supporting evidence passages
• Answer confidence scores

Use cases in drug discovery

Literature mining

• Extract drug-target relationships from publications
• Identify disease-gene associations
• Track competitive intelligence

Knowledge graph construction

• Populate biomedical knowledge bases
• Link entities across publications
• Build drug discovery knowledge graphs

Automated review

• Screen abstracts for relevance
• Extract key findings from papers
• Support systematic literature reviews

Integration with .bloom* ecosystem

Upstream data sources

• PubChem: Compound information for entity linking
• BLAST: Sequence-based gene identification
• Literature databases: PubMed, PMC integration

Downstream applications

• DeepPurpose: Text-mined targets for DTI prediction
• REINVENT: Target information for compound design
• ADMET tools: Literature-based safety signals

Resources

• GitHub (fine-tuning): https://github.com/dmis-lab/biobert
• Pre-trained weights: https://github.com/naver/biobert-pretrained
• Publication: Bioinformatics 2020

Chat Usage

Example Prompts

• "Use BioBERT to search PubMed for EGFR C797S resistance mechanisms"
• "Extract entities from this abstract about Alzheimer's tau aggregation"
• "Run BioBERT entity extraction on the uploaded PDF"

Workflow Chaining

• Feeds into: Literature Passport (entity data), Target Passport (extracted targets), Compound Passport (extracted compounds)
• Accepts from: PubMed search results, free-text queries, uploaded documents

BioBERT Wet Lab Mode

A specialized variant for experimental protocol extraction:

• Input: Experimental text or protocol description
• Output: Structured experimental plan with reagents, steps, and conditions
• Trigger: "Extract a wet lab protocol from this text"

FAQ

Read next