Beyond NER: A Comparative Benchmark of Medical BERTs for Multi-Label Adverse Drug Reaction Classification on the PsyTAR Dataset
Students & Supervisors
Student Authors
Supervisors
Abstract
The rapid expansion of user-generated information on social media and health forums has opened new potential for pharmacovigilance by offering firsthand reports of Adverse Drug Reactions (ADRs). But it’s still hard to get useful information from this messy, unstructured material. To address this, we benchmark five pre-trained transformer models—BERT, BioBERT, ClinicalBERT, PubMedBERT, and SciBERT—for multi-label classification on the PsyTAR dataset, which consists of 6,009 sentences. This study aims to determine which language model effectively captures ADR-related semantics in user-generated medicine reviews. We fine-tune each model using advanced optimization strategies, including layer-wise learning rate decay, weighted random sampling to alleviate class imbalance, and Stochastic Weight Averaging (SWA) to improve generalization. The results show that the general-domain BERT and the literature-focused PubMedBERT did better than biomedical models that were unique to a certain field. They got F1-micro scores of 0.763 and 0.755, respectively. These results indicate that generic and literature-based pre-training is more suitable for informal health content compared to models trained exclusively on clinical data, providing practical insights for model selection in pharmacovigilance.
Keywords
Publication Details
- Type of Publication:
- Conference Name: International Conference on Intelligent Data Analysis and Applications (IDAA 2025)
- Date of Conference: 12/12/2025 - 12/12/2025
- Venue: Daffodil International University, Dhaka, Bangladesh
- Organizer: Daffodil International University, Health Informatics Research Lab