Compliance with reporting standards in Mobile App interventions for ART Adherence among PLHIV

Abstract

Introduction

In recent decades, there has been a proliferation of mobile health (mHealth) interventions to address public health challenges such as HIV/AIDS. Hence, there is a need for standardizing the report of mHealth interventions and frameworks to enable effective knowledge sharing and promote developments. This study aims to review publications on mobile applications used for antiretroviral therapy (ART) adherence among people living with HIV (PLHIV) to evaluate their compliance with the standard reporting guideline by the WHO.

Method

A comprehensive search of published literature was conducted on PubMed, PubMed Central, and MEDLINE databases. We selected randomized controlled trials reporting mobile applications used to improve ART adherence among PLHIV. Only studies published in the last 10 years and the English language were included. Each selected study was reviewed by two independent reviewers against the standard 16-item checklist developed by the WHO.

Results

A total of 16 studies were included in the review. Most of the studies were conducted in the United States of America (n = 7). Only 4 (25%) of the studies reported more than 70% (11/16) of the items on the standard reporting checklist by WHO. More than 80% of the studies reported the intervention content (n = 15) and intervention delivery (n = 13). The least reported items were; interoperability/Health Information Systems (HIS) context (n = 2), infrastructure (population level such as electricity, internet connectivity, etc.) (n = 4), and cost assessment (n = 4). However, these are important factors that ensure the sustainability and usability of mHealth intervention, especially in low- and middle-income countries.

Conclusion

Most mHealth interventions promoting ART adherence did not comply with the standard reporting guideline. The lack of standardization of mHealth interventions may be responsible for increased siloed mobile applications. Hence, there is a need for global adoption of the checklist by Ministries of Health, international organizations, journals, and relevant authorities.

Introduction

The ongoing rise in the number of people living with HIV (PLHIV) poses a significant health and economic challenge. According to the Joint United Nations Programme on HIV/AIDS (UNAIDS) (2024), there were approximately 39 million individuals worldwide living with HIV in 2022 [1]. Out of this total, 37.5 million were adults, 1.5 million were children under 15 years old and 53% were females [1]. As of December 2022, 29.8 million people, constituting 76% (with a range of 65–89%) of all PLHIV, were receiving antiretroviral therapy, a significant increase from the 7.7 million reported in 2010 [1].

In recent decades, there has been a proliferation of mobile health (mHealth) interventions to address public health challenges such as HIV/AIDS. Numerous authors and institutions have extensively published the clinical experiences and outcomes of PLHIV primarily treated with antiretroviral therapy [2]. With the growing recognition of the effectiveness of mobile health (mHealth) interventions in healthcare delivery and outcomes, there has been an increase in the adoption of mHealth interventions in recent decade [3]. According to the World Bank, over 500 mHealth projects were launched in 2011 alone [4]. However, the quality, completeness, and objectivity of the current evidence base for reporting these mHealth interventions vary, making it challenging to compare intervention strategies [5]. Consequently, there is a pressing need for the standardization of reporting for mHealth interventions and the establishment of frameworks to enhance future research quality, making it easier to evaluate new findings and identify key gaps in the evidence.

In 2016, the World Health Organization (WHO) mHealth Technical Evidence Review Group published a 16-item checklist for reporting health interventions using mobile phones. The goals were to standardize and improve the quality of future publications, promote adequate review of emerging evidence, and identify crucial gaps in available evidence [5]. Therefore, this study aims to evaluate publications on mobile apps used for antiretroviral therapy (ART) adherence among PLHIV, assessing their compliance with the WHO’s standard reporting guidelines.

Methods

Study design

A comprehensive search of the literature was conducted on PubMed, PubMed Central, and MEDLINE databases. The literature search was conducted on 30th December, 2023. We selected only randomized controlled trials reporting mobile applications used in improving adherence to ART among PLHIV published from January 2013 to December 2023. The search result was reviewed by two authors while a third author broke the tie in case of disagreement in the decision of the two authors.

Search strategy

The search strategy comprised a combination of medical subheading terms and keywords. We combined the following terms: “mobile app” OR “mobile application” OR mHealth AND (adherence OR antiretroviral OR “HIV treatment”) AND (HIV OR HIV/AIDS OR AIDS).

WHO mERA criteria

mHealth interventions from each of the studied publications were disaggregated across the 16 variables in the mHealth evidence reporting and assessment (mERA) guidelines [5]: (1) infrastructure, (2) technology platform, (3) interoperability/health information systems content, (4) intervention delivery, (5) intervention content, (6) usability/content testing, (7) user feedback, (8) access of individual participants, (9) cost assessment, (10) adoption inputs/programme entry, 11) limitations for delivery at scale, 12) contextual adaptability, 13) replicability, 14) data security, 15) compliance with national guidelines, and 16) fidelity of the intervention [5].

Data extraction and management

The included studies were evaluated with the 16 criteria on the WHO standard reporting guideline for mHealth. A Google form was created including each of the criteria with the response “Yes” or “No” whether the study met the criteria with an excerpt of the evidence from the study. Other variables included the date of publication, country of study, and name of first author. Studies were also grouped into two based on the year of publication – whether it was published before the guideline was released (2013–2016) or after (2017–2023).

Inclusion and exclusion criteria

We considered articles published in English and conducted from January 2013 to December 2023. Only studies that focused on the use of mHealth intervention for ART adherence among patients and healthcare professionals associated with HIV were included. Articles were excluded if: (i) did not describe the mobile app intervention, (ii) focused on interventions different from mobile app (iii) population different from PLHIV or healthcare professional (iv) study design different from randomized control trials (v) published before December, 1988 and beyond December, 2019; (vi) not peer-reviewed, and (vii) studies that require standard subscription.

Data analysis

Data extraction and descriptive analysis were performed using Microsoft Office Excel and categorical variables were summarized in frequency and percentages. “Yes” depicted that the item on the checklist was fulfilled while “No” depicted otherwise. A score of “1” was awarded to each “Yes” and “0” to “No”. The total score for each study was calculated with the highest point attainable of “16”. The total score for each criterion was also calculated as the proportion of the number of included studies. Studies with scores of “11–16” were considered “good” reporting, “8–10” as “average”, and “less than 8” as “poor”.

This study was based exclusively on secondary data; hence, no ethical approval and informed consent were required.

Results

Our search retrieved 46 articles from the three electronic databases searched. A total of 16 studies were selected and included for data extraction and quality assessment, with all the articles published after the release of the consensus guidelines. Most of the studies were conducted in the United States of America (n = 7), followed by one study each in diverse locations including China, India, Kenya, South Korea, Singapore, Sweden, Thailand, Uganda, and Vietnam.

Despite the articles published after the release of the consensus guidelines, the analysis revealed a concerning trend of noncompliance with the WHO’s standard reporting guideline. Only 4 (25%) of the studies reported more than about 70% (11/16) of the items on the standard reporting checklist by WHO. More than 80% of the studies reported the intervention content (n = 15) and intervention delivery (n = 13). (Fig. 1).

Fig. 1

Compliance of Reporting of Included Studies to the WHO 16-item Guideline. The figure shows the number of publications that properly included each of the guideline characteristics. Majority of the studies reported the intervention delivery and intervention content while just a few reported infrastructure and interoperability/health information systems context

More than 50% of the studies reported access of individual participants (n = 13), fidelity of the intervention (n = 12), user feedback (n = 9), usability/content testing (n = 10), compliance with national regulatory guidelines or regulatory service (n = 10), replicability (n = 10), adoption units/program entry (n = 9), and technology platform (n = 9).

The least reported items were; interoperability/Health Information Systems (HIS) context (n = 2), infrastructure (population level such as electricity, internet connectivity, etc.) (n = 4), and cost assessment (n = 4). However, these are important factors that ensure the sustainability and usability of mHealth intervention, especially in low- and middle-income countries.

Discussion

Overall, our results revealed poor compliance with the WHO standard reporting guideline for mHealth in reporting mobile application interventions for antiretroviral therapy adherence among PLHIV. This may be due to inadequate knowledge and awareness among researchers and journal editors as awareness of the guideline might significantly influence how articles adhere to it [5, 6]. However, the WHO standard reporting guideline was released before the publication of all the included studies.

Moreover, our findings support the claims of poor reporting by a systematic scoping review of systematic reviews of adherence to reporting guidelines across different clinical areas and study designs [7]. On the other hand, a study noted a positive compliance trend after the publication of reporting system guidelines although the difference was not statistically significant [6]. Also, the majority of the studies included were conducted in the United States of America, this could be due to any reason.

One of the main limitations of this study is that it was limited to only randomized controlled trials. Quasi-experimental, field, and community trials that could be beneficial and acceptable to PLHIV were not covered in this review. Additionally, generic interventions were excluded unless they were specifically evaluated in PLHIV or health workers. There is also the possibility of the authors splitting the details of interventions, that is, reporting different parts of the checklist across several literatures, due to the word limitations on a manuscript. This could have contributed to the small sample size of the reviewed articles.

We recommend increasing the awareness and dissemination of the checklist and promoting adherence to the guidelines among the editors and publishers of major academic journals. This is because increased awareness and adherence to guidelines have been proven to be effective in healthcare literature [7]. Also, authors should explicitly refer to an external link or resource where interventions are well-detailed where necessary. Through widespread adoption, it is expected that the use of the guideline will standardize the quality of mhealth evidence reporting, and indirectly improve the quality of mhealth literature.

Conclusions

Our review showed that most mHealth interventions promoting ART adherence did not adequately comply with the standard reporting guideline. The lack of standardization for reporting mHealth interventions may be responsible for increased siloed mobile applications. Hence, there is a need for global adoption of the checklist by Ministries of Health, international organizations, journals, and relevant authorities.