Evaluating the Impact of Meducation®, a SMART Application to Medication Adherence: The Need to Evaluate EHR Apps Post-Implementation

Abstract

Background: Medication adherence is a complex behavior influenced not only by the patient but also by clinicians and structural factors. At a system level, many organizations are attempting to incorporate technology as a solution to improve medication adherence behaviors. This preliminary study evaluated Meducation®, a Substitutable Medical Applications and Reusable Technology (SMART) app that was implemented within a health care organization's electronic health record (EHR). Meducation® provides a patient-friendly medication resource printed by a bedside nurse at discharge.

Method: A retrospective study determined the impact of Meducation® on patient medication adherence based on the proportion of days covered.

Results: A total of 100 participants were included in the study. These patients were discharged from a medical-surgical unit of an urban medical center in northern California. The Mann-Whitney U test showed that there was no statistically significant difference in medication adherence between patients that received standard medication handouts (n = 50) versus Meducation® (n = 50; p = .691).

Conclusion: Although non-significant, this study reveals the importance of measuring the outcomes of technology solutions implemented in health care organizations. Considering the variability among patients and clinical culture, it is vital to deliberately determine if promising software performs as expected upon implementation in each unique environment.

Introduction

Many of the current proprietary electronic health record (EHR) platforms that are being used in the United States tend to limit what can be designed and optimized. Third-party app development such as Substitutable Medical Applications, Reusable Technologies (SMART) was enabled to expand the EHR ecosystem and overcome the limitations of existing EHR architecture. SMART apps are open-based, standard platforms that use EHRs as containers and have an app programming interface (API) that is vendor agnostic (Mandl et al., 2019; Mandl et al., 2012). The use of SMART apps is poised to be rapidly adapted, and so there is a critical need to evaluate the actual (not just promised) effectiveness of these apps in clinical settings.

The Agency for Healthcare Research and Quality (2017) reported only 12% of Americans have the necessary health literacy skills to manage the demands of health care, which include understanding and comprehending often-complicated medication instructions. The literature suggests numerous factors affect a patient's adherence to treatment and medications. A study conducted in 2014 found an association between lower numeracy and health literacy skills and post-discharge medication errors, which approximately half of the patients made (Mixon et al., 2014).

Medication adherence is a complex behavior influenced not only by the patient but also by clinicians and structural factors. At a system level, many organizations are attempting to incorporate technology, such as SMART apps, as a solution to improve medication adherence behaviors. Meducation® is a SMART app (see Figure 1) accessible to nurses within the EHR's discharge clinical workflow. Meducation® delivers medication instructions that primarily target individuals considered to be high risk due to low health literacy, impaired vision, or language barriers. The app enables nurses to dynamically create fully personalized medication patient instructions in more than 20 languages, written at a 5th- to 8th-grade reading level with large font sizes, pictograms, and videos designed to make taking medications more intuitive and straightforward (First Databank, 2017). Meducation® was implemented in 2017 at the participating site as a potential solution to help improve patients' medication adherence.

Figure 1: Meducation® Screenshot

Purpose

The purpose of this study was to examine the impact of Meducation® on medication adherence among discharged patients at one site. We explored the relationship between the use of Meducation® and a standard medication handout (see Figure 2) among patients discharged in an acute care hospital. We also examined the relationship between gender, health literacy, language, and polypharmacy to medication adherence.

Figure 2: Standard Medication Handout

Methods

Design

The design was a retrospective study that focused on quantifying the impact of the discharge medication instructions on medication adherence by comparing two groups of patients: those who received standard discharge medication instructions (i.e., did not receive Meducation®) and those who received Meducation®.

Sample and Setting

We reviewed patient charts from an urban hospital's medical/surgical unit in San Francisco, California that met the following criteria: inpatient status by the time of discharge, at least 18 years old, had at least one new medication prescribed upon discharge, discharged to home (specifically, not skilled nursing facility), had Meducation® or standard medication education (not Meducation®) as part of their discharge documentation, and had “External Rx History” (prescription dispensing data history) available in Cerner® PowerChart® (Kansas City, MO) with at least two dispensing dates available post-discharge.

Sample Size

We computed power analysis using G*Power 3.1 (α = 0.05, effect size = 0.5, and power = .8). The sample size was calculated using at least 10 observations per variable, which rendered an optimal sample size of 100 participants. We reviewed a total of 120 patient charts to account for possible missing data.

Measures

Medication adherence

There are generally two types of medication adherence measurements: direct and indirect measurements (Kreysm, 2016). Direct measurements are considered to be more accurate; however, they can be costly and resource-intensive. Alternatively, indirect measurements are less complex and less expensive to implement. Indirect measures include patient self-report, pill counting, clinical response, surveys, refill data, and electronic drug monitoring (Kreysm, 2016).

Although refill data are imperfect in tracking precise dosages, they are about as accurate as self-report or survey-based data (Kreysm, 2016). Refill data do not require patient feedback or communication, and they allow the measurement of a large number of patients over an extended period.

This study used the proportion of days covered (PDC), which provides a more conservative estimate of medication adherence compared to other existing adherence measurements. PDC is considered the preferred method of measuring medication adherence by the Pharmacy Quality Assurance (Pharmacy Quality Solutions, n.d.). The PDC is the total number of days with possession of medication in a period.

We calculated the PDC for each participant in 2018 to measure medication adherence; the formula is the number of days a participant was "covered" by the medication in a period divided by the total number of days in a period (Pillittere-Dugan et al., 2009). The PDC score can range from 0 to 1 (0% to 100% medication adherence) A PDC score of ≥ 0.8 (80%), is considered patient adherence (Pharmacy Quality Solutions, n.d.). This information was acquired using the external Rx history function within the EHR platform, which displays and aggregates dispensed medication histories provided by pharmacy benefit managers and pharmacies. The external Rx history screen contains the drug name, dose, rate, route, last fill, quantity, and other sig. information (pharmacy abbreviations).

Demographic characteristics

We also gathered demographic data on each participant, including age, gender (male/female), health literacy status (yes/no/others), language (English/others), along with polypharmacy (yes/no/unable to obtain), and readmission risk score (range from 0 to 12 with 0-3 as low risk, 4-6 moderate risk, 7-12 as high risk). The readmission score is calculated by capturing 15 questions spread across the admission history form in the EHR. The questions are related to patient’s assistance with medications, functional status, fall history, treatment plan, disease state (end-stage condition, mental illness, incontinence, pressure ulcers, chronic conditions), health literacy status, repeat hospitalizations, oxygen management, cognitive deficit, and living status. The presence of each criterion adds one point to the readmission risk score. We abstracted these data in the patients’ admission history form. The admission history form is completed by a nurse in the EHR when a patient is admitted to a nursing unit. The form contains the complete history and assessment of the patient to identify relevant nursing diagnoses or problems.

Data Collection

After receiving the institution’s IRB approval, we gathered retrospective data via chart audits spanning January 1, 2018, through December 31, 2018. We generated a report on patients who were discharged in 2018 with or without Meducation® given. De-identified patient data from the report were exported to an Excel sheet.

We divided the patients into two groups. Participants in the first group were patients who received Meducation® at discharge. The second group consisted of patients who did not receive Meducation® (i.e., were provided with the standard medication handout given at discharge). We assigned consecutive numbers to each of the 918 records for those who received Meducation®. We then used a random number generator to select the 60 patients that would constitute the intervention group. We followed the same procedure for the 1,750 patient records who did not receive Meducation®, rendering the 60 patients that would constitute the control group.

Next, we conducted chart data abstraction for the randomly selected participants, including demographic and clinical data such as age, gender, polypharmacy status, health literacy, and readmission risk score. We then calculated the PDC for each medication that was prescribed at discharged using the PDC Calculator Prime v3.3 (Santa Clara Valley Health and Hospital System, n.d.). We used the data from the external Rx history function in the EHR to calculate the PDC, including the dose, date of prescription, and the quantity of pills. The PDC Calculator Prime v3.3 calculated the coverage gap, number of days covered by medications, number of days in treatment period, and the PDC. After calculating the PDC for each medication that was prescribed at discharge for each patient, we calculated the mean PDC. The measurement period was from the patient’s initial prescription date post-discharge to the end of 2018.

Statistical Analysis

We used IBM SPSS (v25) for the statistical analyses, including descriptive statistics such as mean, standard deviation, and percentage. Given the non-normal distribution of the PDC data (left-skewed), we used non-parametric tests to compare the groups on mean medication adherence. We analyzed demographic characteristics to medication adherence using Mann Whitney U or Kruskal-Wallis H. Statistical significance was set to the traditional two-sided a of 0.05.

Missing Data

Although we abstracted a total of 120 charts, there were missing data within each variable. We used listwise deletion, the default on most statistical software, to address missing data. Since we accounted for the missing data during power analysis, our finalized sample was sufficient to perform the analyses.

Results

Participant characteristics

The sample consisted of 51% males and 49% females, with a mean age of 68.32 years old (SD = 16.15). The majority reported that they had no health literacy deficiency (80%) and took less than seven home medications (80%). Most were English speakers (86%), followed by Cantonese speakers (8%); the remaining 6% spoke other languages. The mean readmission risk score was 5.44 (SD = 2.23), indicating moderate readmission risk.

Group difference in patient’s medication adherence score

We assessed for differences in patients’ medication adherence score (using PDC) between participants who received Meducation® (intervention group) versus those who received a standard medication handout (control group). Initially, we used the t test to compare the two groups. However, we detected that the data in both groups were substantially negatively skewed; as such, we opted for the Mann-Whitney U test, the nonparametric alternative to the t test. The results (see Table 1) revealed that there was no statistically significant difference in PDC across the two groups (p = .691).

Table 1: Participant/Group Characteristics

Next, we evaluated if there were any statistically significant differences between gender, health literacy status, language, and polypharmacy to medication adherence. We used Mann-Whitney U to analyze gender and language to the medication adherence score. We used Kruskal-Wallis H test to assess the differences in medication adherence scores to language, health literacy status, and polypharmacy.

We found no statistically significant differences between these characteristics to medication adherence. For instance, males who were given Meducation® versus males who received a standard medication handout did not result in any statistically significant difference (p = 0.992; Table 1).

Discussion

Many organizations have been exploring various techniques to improve medication adherence, including post-discharge follow-up calls, hospital-community partnership transitional programs, teach-back, motivational interviewing, financial incentives, and technology-centered medication tools such as SMART apps (Hyrkas & Wiggns, 2014; Patel et al., 2017; Starring et al., 2010). This retrospective study at one facility showed that there was no statistically significant difference between patients that received a SMART app-generated handout versus a standard medication handout to medication adherence. Meducation summarizes patients’ medication regimens into a single page that often includes pictograms of usage and a simplified calendar format (see Figure 1). The standard medication handout (see Figure 2), on the other hand, lists the medications that patients need to take with the dosing instructions after the medication. There are no grids or calendar format or pictograms included.

The mean PDC for the use of Meducation® was only 0.4% higher than the standard medication handout. While there is limited evidence examining the impact of SMART apps such as Meducation® to medication adherence, one study among patients with cardiovascular disease found that Meducation® positively affected medication adherence and clinical outcomes after six months (Zullig et al., 2014).  

We also did not find a statistically significant association between health literacy to medication adherence. Mosher et al. (2012) found that health literacy was not associated with self-reported medication adherence among elderly veterans. Similarly, in a study conducted by Huang et al. (2018), the authors found that health literacy was not significantly associated with medication adherence with patients diagnosed with type 2 diabetes. Likewise, in a systematic review, although the authors found a weak but significant relationship between health literacy and medication adherence, the authors suggested that the relationship may be mediated with other adherence determinants (Zhang et al., 2014).

Previous studies have shown conflicting results on the impact of gender to medication adherence (Chen et al., 2014; Manteuffel et al., 2014); our study did not detect a statistically significant difference between gender to medication adherence.

There are potentially different reasons as to why we did not observe any statistically significant differences in medication adherence between those who did and did not get Meducation®. This might be due to random variation in the data or due to a smaller effect size. However, non-significant results do not imply there is no clinical significance. Medication adherence is a complex series of decision-making on the part of the patient. Different factors need to align for optimal medication adherence. Vrijens et al. (2012) outlined three distinct phases for optimal adherence: 1) initiation phase during which there is clear communication of the medications to patients and family members, and patients fill the prescription, 2) implementation phase during which patients take the medications as prescribed, and 3) discontinuation phase during which patients sustain the medications and terminate when indicated. A miss in one of these phases may result in poorer medication adherence. This study emphasizes the critical importance of the initiation phase. Medication instruction at discharge is a multifaceted process, and simply supplying an instruction, regardless of content, may be insufficient. Medication instruction should be patient-centered, with dedicated time given to this process regardless of the type of medication handout. Improvement of patients’ understanding of their medications requires several interventions, including the use of “teach-back” to verify understanding and comprehension (Schillinger et al., 2003).

With the potential growth of SMART apps, there is a critical need to evaluate these apps to make sure they are effective and deliver the intended outcomes. Inadequate evaluation and the blind adoption of technology interventions could lead to wasted human and financial resources and potentially unintended consequences.

Limitations

Due to the preliminary nature of this single-site study at a mid-sized urban medical center, results from this study may have limited external validity. Our study population was also limited to patients discharged in a general medical/surgical unit. The use of the refill database to quantify medication adherence, although considered an acceptable estimate, assumes that refilling equates to patients’ medication adherence behavior. Future research should include focusing on specific medications or disease conditions to examine if there are any differences. Expanding to multiple sites and units should also be considered to solidify external validity further.

Conclusion

It is known that adherence to medications is a complex phenomenon and is dependent upon various factors. As poor medication adherence is often a contributor to poorer health outcomes, health care organizations and individual providers continue to innovate and try to use technology-based interventions such as SMART apps to aid patients.

As we move toward a more digital era of health care in settings involving a variety of patients, institutional cultures, and clinical practices, our findings suggest the need for two tiers of quality control: (1) verify that the new technology is operating reliably, and (2) determine the extent to which the new resource provides measurable benefit(s) to the intended population(s) in each setting.

The views and opinions expressed in this blog or by commenters are those of the author and do not necessarily reflect the official policy or position of HIMSS or its affiliates.