Evaluation of Da2 Pilot Projects and Recommended Actions

MEMORANDUM

TO:         Carol Lovin, EVP and Chief Strategy Officer, Carolinas HealthCare System

FROM:        Michael Dulin, Head of Dickson Advanced Analytics

        [Your name], Director of Analytics Research Projects

DATE:        November 13, 2014

SUBJECT:         Evaluation of DA2 Pilot Projects and Recommended Actions

Dickson Advanced Analytics (DA1) has recently completed four pilot projects, each intended to address key challenges and opportunities facing CHS. Per your request, we have performed an evaluation of three of these pilot projects based on the project descriptions we were provided.  In  this report we share the results of our evaluation and make recommendations on which pilots CHS should implement into regular operations.  We also discuss how, and which, if any, of the pilots we should take in a different direction (and what that direction should be).

Purpose and Scope of the Investigation

We evaluated these three pilot projects:

P1: Mapping Underserved Communities

P2: Reducing Readmission

P3: Advanced Illness Management

The goals of our investigation were to: (1) evaluate the projects based on a common set questions and (2) recommend next step(s) for each pilot project.

Questions Guiding the Evaluation

We used the following questions to guide our investigation of the three projects:

• What was the business objective served by the pilot project?  
• What were the pilot project’s goals? How well do these goals align with strategic priorities for DA2 and CHS?
• What did the pilot actually do? What were the key results of the pilot?
• What were the strengths of the pilot in terms of how it was executed?
• Did we have any concerns about execution of the pilot that made us question the validity of the results? Were there important gaps in the information we received that made our assessment more uncertain?
• If we thought the pilot should be implemented into regular operations of CHS, how should that happen? How hard would it be? What potential pitfalls should we worry about? Would implementation of the pilot be cost effective?
• If we thought the pilot should not yet be implemented into regular operations, why not, and what should we do instead?

Summary of Our Evaluation of the Pilots

The pilots provide a great opportunity to improve the health care services by predicting the requirements and needs of the patients well in advance that helps us to allocate our resources effectively. The pilots also improve the quality of the health care facilities by helping physicians identify the risks associated with patient’s health and address them effectively so that they don’t have to be re-admitted or don’t have to stay for long in the health care facility.  Finally, pilots help us create medical data such as patient records, surgical procedure outcomes and treatment history that can be integrated in powerful ways that would lead to a better patient care, lower health care cost and healthier patients.

Pilot 1: Mapping Underserved Communities

Summary of the Project and Results

The primary aim of the pilot is to reduce the usage of Emergency Departments(ED) by making primary care more accessible. We believe that people are generally underserved by primary care facility which makes them use ED more often than it is necessary. So we created a scalable model using geotagging technique that used both clinical and descriptive data (in form of census data). The model would not only help us identify the areas and localities that are not served effectively by primary care facilities but also help us to anticipate localized future demand for healthcare professionals, and to develop interventions to improve access to primary care facilities.

Evaluation of the Project

Strengths of pilot:

• Both clinical data (available to the facility) and the census data (available to the public) required in this model are easily accessible.
• The pilot is easy to interpret, is relatively inexpensive and processes at a fast pace.
• The model is scalable and can be used in future anticipate localized future demand for healthcare professionals, to develop interventions to improve access to primary care facilities, to provide data to support policy decisions regarding healthcare initiatives, and to measure the impact of interventions designed to improve healthcare access.

Concerns about execution of the pilot:

• Insufficient data about the address of the large number of patients (about 180,000) can result in biased results or may lead to missing out on deserved area.
• Dependence on Census data makes it a laggard trend follower, as census data is published once in 10 years.
• Scope is limited to only identifying the areas that lacks the access of healthcare.
• Lot of data so we need some data protection.

Factors affecting the difficulty and cost effectiveness of implementing the pilot:

• Easy availability of data and use of geotagging makes it easy to implement.
• Use of clinical and census data means no cost of obtaining data from external sources.  
• Large data means large storage cost.

Pilot 2: Reducing Readmissions

Summary of the Project and Results

The primary aim of the pilot is to reduce the number of revisits made by the patients to the CHS facility by creating an algorithm that calculated readmission risk score, which helps us predict whether the admitted patient would need a revisit. The model uses the internal data from the CHS facilities and shortlist certain demographic and medical variables for prediction. The results from the model can be segmented based on the demographic and the risk profile (% of readmission risk) of the patient which help discharge care manager manage patients effectively and provide them quality and timely medication. The model had an accuracy rate of 79% in predicting a patient’s risk of readmission within 30 days of discharge.