Feedback

Evaluation of a natural language processing tool for extracting gender, weight, ethnicity, and race in the US food and drug administration adverse event reporting system

Affiliation
Division of Pharmacovigilance ,Office of Surveillance and Epidemiology ,Center for Drug Evaluation and Research ,US Food and Drug Administration ,Silver Spring ,MD ,United States
Dang, Vivian;
Affiliation
Division of Pharmacovigilance ,Office of Surveillance and Epidemiology ,Center for Drug Evaluation and Research ,US Food and Drug Administration ,Silver Spring ,MD ,United States
Wu, Eileen;
Affiliation
Division of Pharmacovigilance ,Office of Surveillance and Epidemiology ,Center for Drug Evaluation and Research ,US Food and Drug Administration ,Silver Spring ,MD ,United States
Kortepeter, Cindy M.;
Affiliation
Division of Pharmacovigilance ,Office of Surveillance and Epidemiology ,Center for Drug Evaluation and Research ,US Food and Drug Administration ,Silver Spring ,MD ,United States
Phan, Michael;
Affiliation
Office of Biostatistics ,Office of Translational Sciences ,Center for Drug Evaluation and Research ,US Food and Drug Administration ,Silver Spring ,MD ,United States
Zhang, Rongmei;
Affiliation
Office of Biostatistics ,Office of Translational Sciences ,Center for Drug Evaluation and Research ,US Food and Drug Administration ,Silver Spring ,MD ,United States
Ma, Yong;
Affiliation
Division of Pharmacovigilance ,Office of Surveillance and Epidemiology ,Center for Drug Evaluation and Research ,US Food and Drug Administration ,Silver Spring ,MD ,United States
Muñoz, Monica A.

The US Food and Drug Administration Adverse Event Reporting System (FAERS) contains over 24 million individual case safety reports (ICSRs). In this research project, we evaluated a natural language processing (NLP) tool’s ability to extract four demographic variables (gender, weight, ethnicity, race) from ICSR narratives. Specificity of the NLP algorithm was over 94% for all demographics, while sensitivity varied between the demographics: 98.6% (gender), 45.5% (weight), 100% (ethnicity), and 85.3% (race). Among ICSRs missing weight, ethnicity, and race in the structured field, few cases had this information in the narrative (>95% missing); consequently, the positive predictive value (PPV) for these three demographics had wide 95% confidence intervals. After NLP implementation, the total number of ICSRs missing gender was reduced by 33% (i.e., NLP identified 472 thousand reports having a gender value in the narrative that was not in the structured field), while the total number of ICSRs missing weight, ethnicity, or race was reduced by less than 4%. This study demonstrated that the implementation of an NLP tool can provide meaningful improvements in the availability of gender information for pharmacovigilance activities conducted with FAERS data. In contrast, NLP tools targeting the extraction of weight, ethnicity, or race from free-text fields have minimal impact largely because the information was infrequently provided by the reporter. Further gains in completeness of these fields must originate from increases in provision of demographic information from the reporter rather than informatic solutions.

Cite

Citation style:
Could not load citation form.

Access Statistic

Total:
Downloads:
Abtractviews:
Last 12 Month:
Downloads:
Abtractviews:

Rights

License Holder: Copyright © 2022 Dang, Wu, Kortepeter, Phan, Zhang, Ma and Muñoz.

Use and reproduction: