Opportunities and challenges in developing risk prediction models with electronic health records data: a systematic review
Opportunities and challenges in developing risk prediction models with electronic health records data: a systematic review
No Thumbnail Available
Date
2016-05-17
Authors
Benjamin A Goldstein
Ann Marie Navar
Michael J Pencina
John P A Ioannidis
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
Objective: Electronic health records (EHRs) are an increasingly common data source for clinical risk prediction,
presenting both unique analytic opportunities and challenges. We sought to evaluate the current state of EHR
based risk prediction modeling through a systematic review of clinical prediction studies using EHR data.
Methods: We searched PubMed for articles that reported on the use of an EHR to develop a risk prediction
model from 2009 to 2014. Articles were extracted by two reviewers, and we abstracted information on study design, use of EHR data, model building, and performance from each publication and supplementary documentation.
Results: We identified 107 articles from 15 different countries. Studies were generally very large (median sample size ¼ 26 100) and utilized a diverse array of predictors. Most used validation techniques (n ¼ 94 of 107) and
reported model coefficients for reproducibility (n ¼ 83). However, studies did not fully leverage the breadth of
EHR data, as they uncommonly used longitudinal information (n ¼ 37) and employed relatively few predictor
variables (median ¼ 27 variables). Less than half of the studies were multicenter (n ¼ 50) and only 26 performed
validation across sites. Many studies did not fully address biases of EHR data such as missing data or loss to follow-up. Average c-statistics for different outcomes were: mortality (0.84), clinical prediction (0.83), hospitalization (0.71), and service utilization (0.71).
Conclusions: EHR data present both opportunities and challenges for clinical risk prediction. There is room for improvement in designing such studies.