---
title: BioBERT
description: Deprecated. BioBERT literature search is no longer available in chat; use PubMed search instead.
tags:
  - biobert
---

BioBERT lets you mine biomedical text — extracting genes, proteins, diseases, chemicals, and the relationships between them from abstracts and papers. Reach for it when you want structured entities out of literature rather than a list of citations. It is the natural next step after its sibling [PubMed](/docs/pubmed): use PubMed to retrieve the papers, then BioBERT to pull out the drug–target–disease relationships. See [what you can achieve](/docs/capabilities#literature).

> **Not available in chat**

> BioBERT is deprecated on the platform and not registered with the orchestrator, so chat cannot call it. A current LLM performs BioBERT's entity extraction and relevance ranking natively over text retrieved with [PubMed](/docs/pubmed); for curated identifiers an LLM must not invent (NCBI Gene IDs, MeSH terms), use PubTator3. Existing BioBERT job results remain viewable.

## BioBERT integration in .bloom* platform

**BioBERT** (Bidirectional Encoder Representations from Transformers for Biomedical Text Mining) is a domain-specific language representation model pre-trained on large-scale biomedical corpora. Developed by the **DMIS Lab at Korea University**, it significantly outperforms general-purpose BERT on biomedical NLP tasks.

## Citations

> Lee J, Yoon W, Kim S, Kim D, Kim S, So CH, Kang J. BioBERT: a pre-trained biomedical language representation model for biomedical text mining. *Bioinformatics*. 2020 Feb 15;36(4):1234-1240. doi: [10.1093/bioinformatics/btz682](https://doi.org/10.1093/bioinformatics/btz682)

## Training data

BioBERT was pre-trained on massive biomedical corpora:
- **4.5 billion words** from PubMed abstracts
- **13.5 billion words** from PubMed Central (PMC) full-text articles

## Performance improvements over BERT

BioBERT largely outperforms general BERT on key biomedical tasks:

| Task | Improvement |
|------|-------------|
| Named Entity Recognition (NER) | +0.62% F1 |
| Relation Extraction (RE) | +2.80% F1 |
| Question Answering (QA) | +12.24% MRR |

## Key capabilities

### Biomedical Named Entity Recognition (NER)

Identify and classify biomedical entities in text:
- **Genes/proteins**: BRCA1, p53, insulin receptor
- **Diseases**: cancer, diabetes, Alzheimer's disease
- **Chemicals/drugs**: aspirin, doxorubicin, metformin
- **Species**: human, mouse, E. coli
- **Mutations**: c.248T>C, V600E

### Relation Extraction (RE)

Extract relationships between entities:
- **Drug-disease**: Drug X treats disease Y
- **Protein-protein**: Protein A interacts with protein B
- **Gene-disease**: Mutation in gene X causes disease Y
- **Drug-drug**: Drug A interacts with drug B

### Question Answering (QA)

Answer biomedical questions from text:
- "What drug treats condition X?"
- "What is the mechanism of action of drug Y?"
- "Which genes are associated with disease Z?"

## Why domain-specific pre-training matters

General NLP models often fail on biomedical text because:
- **Domain-specific vocabulary**: Terms like "transcriptional", "antimicrobial"
- **Complex nomenclature**: Gene names (BRCA1), mutation notation (c.248T>C)
- **Technical language**: Scientific writing style and conventions
- **Word distribution shift**: Different word frequencies vs. general text

BioBERT's biomedical pre-training addresses these challenges.

## Available model versions

| Model | Base | Additional Training |
|-------|------|---------------------|
| BioBERT-Base v1.1 | BERT-base-Cased | + PubMed 1M |
| BioBERT-Large v1.1 | BERT-large-Cased | + PubMed 1M (custom 30k vocab) |

## Webapp integration

### Input methods in .bloom*

**Text input:**
- Paste abstracts or article text directly
- Upload text files or PDFs
- Import from PubMed IDs

**Batch processing:**
- Process multiple documents in parallel
- CSV input with text column
- Progress tracking for large datasets

### Output formats

**NER results:**
- Annotated text with entity spans
- Entity type classification
- Confidence scores per entity

**RE results:**
- Entity pairs with relationship types
- Evidence sentences
- Confidence scores

**QA results:**
- Extracted answers from text
- Supporting evidence passages
- Answer confidence scores

## Use cases in drug discovery

### Literature mining
- Extract drug-target relationships from publications
- Identify disease-gene associations
- Track competitive intelligence

### Knowledge graph construction
- Populate biomedical knowledge bases
- Link entities across publications
- Build drug discovery knowledge graphs

### Automated review
- Screen abstracts for relevance
- Extract key findings from papers
- Support systematic literature reviews

## Integration with .bloom* ecosystem

### Upstream data sources
- **PubChem**: Compound information for entity linking
- **BLAST**: Sequence-based gene identification
- **Literature databases**: PubMed, PMC integration

### Downstream applications
- **DeepPurpose**: Text-mined targets for DTI prediction
- **REINVENT**: Target information for compound design
- **ADMET tools**: Literature-based safety signals

## Resources

- **GitHub (fine-tuning)**: [https://github.com/dmis-lab/biobert](https://github.com/dmis-lab/biobert)
- **Pre-trained weights**: [https://github.com/naver/biobert-pretrained](https://github.com/naver/biobert-pretrained)
- **Publication**: [Bioinformatics 2020](https://doi.org/10.1093/bioinformatics/btz682)

## Chat Usage

### Example Prompts

- "Use BioBERT to search PubMed for EGFR C797S resistance mechanisms"
- "Extract entities from this abstract about Alzheimer's tau aggregation"
- "Run BioBERT entity extraction on the uploaded PDF"

### Workflow Chaining

- **Feeds into**: Literature Passport (entity data), Target Passport (extracted targets), Compound Passport (extracted compounds)
- **Accepts from**: PubMed search results, free-text queries, uploaded documents

### BioBERT Wet Lab Mode

A specialized variant for experimental protocol extraction:
- Input: Experimental text or protocol description
- Output: Structured experimental plan with reagents, steps, and conditions
- Trigger: "Extract a wet lab protocol from this text"

## FAQ

### What tasks can BioBERT perform?

  BioBERT handles named entity recognition, relation extraction, and question answering for biomedical text.

### What data was BioBERT trained on?

  4.5B words from PubMed abstracts and 13.5B words from PMC full-text articles.

## Read next