Secondary Logo

Share this article on:

Evaluation of Smartphone Menstrual Cycle Tracking Applications Using an Adapted APPLICATIONS Scoring System

Moglia, Michelle, L., WHNP, MS; Nguyen, Henry, V., FNP, MS; Chyjek, Kathy, MD; Chen, Katherine, T., MD, MPH; Castaño, Paula, M., MD, MPH

doi: 10.1097/AOG.0000000000001444
Contents: Original Research

OBJECTIVE: To identify smartphone menstrual cycle tracking applications (apps) and evaluate their accuracy, features, and functionality.

METHODS: In this systematic evaluation, we searched the Apple iTunes store for free menstrual cycle tracking apps for patient use. We considered an application accurate if menstrual cycle predictions were based on average cycle lengths of at least three previous cycles, ovulation (when included) was predicted at 13–15 days before the start of the next cycle, and the application contained no misinformation. We modified the APPLICATIONS Scoring System to evaluate the features and functionality of accurate apps.

RESULTS: Our search criteria yielded 1,116 apps; 108 remained after excluding duplicate, non-English, nonmenstrual cycle tracking, and priced apps. We further eliminated 88 that did not meet inclusion or accuracy criteria. Of the 20 accurate, free apps, 80% contained information for conception and 50% for contraception. Common features and functionality included password protection (55%); no requirement for Internet connectivity (80%); no advertisements (65%); in-application technical support (70%); medical disclaimers (65%); health education (55%); tracking of menstrual flow (70%), symptoms (70%), and intercourse (75%); alerts for next menses (65%) and fertility (55%); and cycle length information (75%). Forty percent were available for Android. Usefulness for fertility medications (15%), professional involvement (5%), and cited literature (5%) were rare.

CONCLUSION: Most free smartphone menstrual cycle tracking apps for patient use are inaccurate. Few cite medical literature or health professional involvement. We list accurate apps to aid health care providers in understanding the key components they can use to evaluate and recommend apps for patients.

Of many free menstrual cycle tracking applications for patient use, few are accurate, cite medical literature, or were created by health care professionals.

Columbia University School of Nursing, the Icahn School of Medicine at Mount Sinai, and the Columbia University Medical Center, New York, New York.

Corresponding author: Paula M. Castaño, MD, MPH, Obstetrics and Gynecology, 622 W 168th Street, PH 16-69, New York, NY 10032; e-mail: pc2137@columbia.edu.

Financial Disclosure The authors did not report any potential conflicts of interest.

Presented as a poster at the Annual Clinical and Scientific Meeting of the American College of Obstetricians and Gynecologists, May 2–6, 2015, San Francisco, California, and at the Annual District II Meeting of the American College of Obstetricians and Gynecologists, October 24, 2015, New York, New York.

The authors thank Mary-Jane McEneaney, WHNP, DNP, and Adena Bargard, CNM, PhD, from Columbia University School of Nursing for their feedback during initial development of this study.

Tracking of menstrual cycles for self-understanding and achieving fertility goals is a common form of self-monitoring that promotes patient empowerment and informed decision-making.1,2 Nippita et al3 found 68% of women initiating contraception usually track their menses. Menstrual cycle tracking applications (apps) are the fourth most popular health app among adults4 and the second most popular among adolescent females.5 With approximately 80% of 18–49 year olds in the United States owning smartphones,6 and 23% of women smartphone owners using mobile health apps,4 the use of health apps has the potential to promote the provision of patient-centered, reproductive health care.7

Application overload is a challenge for health care providers and patients.8–10 The number of health apps in the U.S. Apple iTunes and Google Play stores exceeds 165,000.10 Of 90,088 health apps in the Apple iTunes store, 7% are for women's health and pregnancy.10

Surveyed health care providers believe information collected in patient apps can promote better understanding of patient health.11 Health care providers express interest in promoting the use of health apps but seek guidance about making app recommendations.9,11,12 Evidence-based, unbiased evaluations of health apps that address app accuracy are needed.13,14 Chyjek et al15 thus developed the APPLICATIONS Scoring System in response to a request for a sustainable app review model free of conflict of interest.13

The objectives for this study are to identify and evaluate free menstrual cycle tracking apps using an adapted APPLICATIONS Scoring System. Our evaluation will serve as a resource for health care providers on features and functionality common to these apps.

Back to Top | Article Outline

MATERIALS AND METHODS

This study did not require review by the institutional review boards at the Columbia University Medical Center and the Icahn School of Medicine at Mount Sinai based on the prespecified guidelines that this research did not involve or pose any risks to human subjects. We conducted a systematic evaluation of free menstrual cycle tracking apps. We searched the Apple iTunes store using the terms “menstrual cycle,” “period,” “fertility,” and “menstrual calendar” between December 15, 2014, and January 9, 2015. We compiled a master list of apps by reviewing the app store description. We excluded duplicate apps, those not available in English, apps for health care providers, and apps no longer available at the time of review. Exclusions on further review of content were nonhealth apps; nonwomen's health apps; other women's health apps (eg, fertility, pregnancy, contraceptive reminder, menopause); apps exclusively for partners; and apps specific to a medical practice, program, or device. To eliminate the barrier of cost, we included only free apps in further analysis, eliminating all priced (paid) apps, apps requiring a subscription for use, and free apps requiring purchase of a special commercial device.

Our primary criterion for ongoing inclusion in this study was accuracy. Ninety-nine percent of regular menstrual cycles range from 21 to 35 days, and only one in eight or nine women have 28-day cycles.16 Because women may not know their average cycle length, we decided that the ability to predict the next menstrual cycle based on averages of past cycles and not on a default (often 28-day) cycle length would be an important element of our accuracy criteria. We eliminated apps that did not allow input of four previous menstrual periods (three full menstrual cycles). This is equivalent to the number of tracked cycles required for entry into natural family planning research studies.17 An app was considered accurate if it predicted July 7, 2015, as the date of the next expected menstrual cycle after entering previous menstrual periods of April 6, April 27, May 22, and June 14, 2015. We chose to input dates for a 23-day menstrual cycle average to eliminate apps that work only for cycles with a 26- to 32-day range; 22% of women with regular cycles fall outside this narrow range.17

From June 21, 2015, to July 1, 2015, we downloaded the latest version of each free, patient-centered menstrual cycle tracking app from our original search and checked for accurate prediction of the next menstrual cycle. We further eliminated apps that contained erroneous health information such as a predicted date of ovulation outside 13–15 days before the start of the next menstrual cycle16 or gender predictions based on conception date.7,18,19

We recorded seller, version, and Android platform availability for each app. The components of the APPLICATIONS Scoring System were developed based on existing literature.15 The scoring system enhances filtering of irrelevant and inaccurate apps and has been used to systematically review and evaluate reproductive health app content.15,20 The five authors adapted the components of the APPLICATIONS Scoring System to assess the features and functionality that other researchers have determined as important for health care providers and their patients (Table 1).12,14,18,21 All authors then independently reviewed, evaluated, and scored each app.

Table 1

Table 1

We initially downloaded and opened each app in airplane mode to evaluate the connectivity component and determine whether functionality was dependent on Internet access. For free apps with premium (paid) versions, we assigned points only for features available in the free version. We determined app comprehensiveness by assigning 1 point for fertility tracking for women trying to conceive, 1 point for containing information on contraception or pregnancy prevention, and 1 point for tracking of fertility medications. We further evaluated each app for password protection, professional involvement, citation of scientific literature used, in-app purchases, third-party advertisements, access to technical support, and 14 “other” features.

We tabulated each “other” feature. These included a social media feature; medical disclaimer; health education; option for data backup and e-mail or export of cycle history; a custom reminder; pregnancy mode; tracking of flow amount, symptoms, or sexual intercourse; alerts for next menses and fertility; and calculation and display of average cycle length. A native Spanish speaker (P.M.C.) reviewed each app for Spanish language availability. We totaled all points for this component and awarded 0 points if zero to four features were present, 1 point if five to nine features were present, and 2 points if 10 or more features were present. Each reviewer individually evaluated navigation ease and subjective presentation using a Likert scale with 1=poor, 2=below average, 3=average, 4=above average, and 5=excellent.

To account for any interobserver differences, we reconciled ratings and recorded objective errors. Our reconciliation process included meetings of all authors during which we discussed each feature for each app. When there was a discrepancy, we determined whether it was a transcription or misclassification mistake and arrived at 100% consensus for the objective components of each app. For navigation ease and subjective presentation, we averaged reviewers' scores and awarded 0 points for an average rating of less than 3 and 1 point for an average rating of 3 or greater. We calculated a final total score for each app.

Back to Top | Article Outline

RESULTS

After obtaining a total of 1,116 results with the search terms, initial exclusions left 650 apps (Fig. 1). Further screening and review of content eliminated 425 apps, resulting in 225 menstrual cycle tracking apps. Of the 225 apps, 117 were paid apps or apps requiring a subscription for use and 108 were free. One author (M.L.M.) applied the accuracy criteria to the 108 free apps.

Fig. 1

Fig. 1

Twenty of the 108 apps (19%) were accurate. A second author (H.V.N.) confirmed accuracy of these apps. The 20 accurate apps were then evaluated with an adapted APPLICATIONS Scoring System by all authors (Tables 2 and 3). The highest scoring app earned 13 out of 15 possible points. Twelve apps scored above the mean score of 7.7. The lowest scoring app earned 2 points.

Table 2-a

Table 2-a

Table 2-b

Table 2-b

Table 3

Table 3

Evaluated features and functionality are shown in Figures 2 and 3. Eighty percent contained information for conception and 50% for contraception. Common features and functionality included password protection (55%); no requirement for Internet connectivity (80%); no advertisements (65%); in-app technical support (70%); medical disclaimers (65%); health education (55%); tracking of menstrual flow (70%), symptoms (70%), and intercourse (75%); alerts for next menses (65%) and fertility (55%); and cycle length information (75%). Forty percent were available for Android. Usefulness for fertility medications (15%), professional involvement (5%), and cited literature (5%) were rare features. The objective component reporting error rate was 30 out of 200 (15%), meaning that the authors independently assigned the same score to a given objective component of an app 85% of the time.

Fig. 2

Fig. 2

Fig. 3

Fig. 3

Back to Top | Article Outline

DISCUSSION

Most available menstrual cycle tracking apps are inaccurate, contain misleading health information, or do not function. We found only 20 free, English-language apps that allowed for accurate menstrual cycle tracking based on cycle length averages. Few apps cited literature used or were developed or recommended by reproductive health experts. The highest scoring app missed points for not having an option to track infertility treatments and not reporting health professional involvement. Of the 14 “other” features, this app missed only a social media feature and allowing the consumer to designate a pregnancy and thus temporarily stop menstrual cycle tracking. Our evaluation contributes to mobile health literature by identifying free, accurate, menstrual cycle tracking apps for patients and highlighting their features and functionality through a modified APPLICATIONS Scoring System.

Nineteen percent of apps evaluated for accuracy contained erroneous medical information. Because content of health care apps is not monitored by any regulatory agency,13,18,22 health care provider involvement in app content is key. When health care provider input is impossible, citation of medical literature legitimizes content. Our review, however, reveals that only one app reported professional involvement and one cited literature.

A strength of this analysis is the use of an adapted version of an existing scoring system endorsed by two of the original authors who served among the five app reviewers. We developed strict inclusion and accuracy criteria to focus on apps that would not mislead patients or health care providers in any critical way. The features and functionality assessed have been cited as important by researchers.12,14,18,21

Another strength is that reviewers were racially and ethnically diverse male and female reproductive health nurse practitioners and physicians ranging in age from their 20s to 40s. The aim of this evaluation was to provide health care providers with the explicit components of the scoring system and results so that they can better advise their patients in selecting apps. Patient evaluation of accurate apps is the next natural step.23

A limitation of our study is inclusion of only free apps from the Apple iTunes store. We restricted our evaluation to apps that would be most accessible. Cost contributes to a consumer's rating of an app.21 We searched the Apple iTunes and then the Google Play stores given their dominance in the app market. An equal proportion of young women use Apple and Android devices.24 Searching all platforms could have decreased ascertainment bias but was beyond the scope of this study.

Availability, content, features, and functionality may have changed during our evaluation. Technologic advances move at a faster pace than research.25 We carried out our evaluation systematically and during a limited timeframe to limit variability. We provide app version information to allow for independent assessment of improvements in subsequent versions.

Menstrual cycle tracking apps are frequently downloaded, yet challenges exist for both patients and health care providers when considering which app to use.4,5,12 We confirmed that the app stores do not allow for an easy search.9 Some apps have similar names; many search results are not menstrual cycle trackers.

Privacy and security concerns exist with the use of mobile technologies.2,13,23,26 A health care provider recommending apps may advise patients to enter only that personal information that they would share with third parties.26 Paper calendars may remain the best option for some patients until data use is more transparent.

Information contained in menstrual cycle tracking apps should not take the place of medical advice. In conjunction with other methods, patients may choose to use apps to help prevent or achieve pregnancy. Along with the potential for positive outcomes, there is potential for negative outcomes,7,18 including unplanned pregnancies. Patients may not understand the factors that can affect a menstrual cycle and that predictions by apps are estimates; alerts for next menses or for a fertile window require regular cycles. We did not evaluate for nor do we advocate use of these menstrual cycle tracking apps as a primary tool to prevent or achieve pregnancy.

Health care providers may use apps in counseling and educating patients or, given the importance of the menstrual cycle as a vital sign,27 to assist with diagnosis and surveillance of reproductive health problems. The utility of the 20 evaluated apps for such use merits further evaluation. Our findings serve to familiarize busy health care providers with the challenges, risks, and benefits of menstrual cycle tracking app use by patients. Only through gaining a better understanding of these apps will we be able to achieve the goal of safe and effective use by patients and harness the utility these apps hold for health care providers.

Back to Top | Article Outline

REFERENCES

1. Appelboom G, LoPresti M, Reginster J-Y, Connolly ES, Dumont E. The quantified patient: a patient participatory culture. Curr Med Res Opin 2014;30:2585–7.
2. Lupton D. Quantified sex: a critical analysis of sexual and reproductive self-tracking using apps. Cult Health Sex 2014;17:440–53.
3. Nippita S, Oviedo J, Velasco MG, Westoff CL, Davis AR, Castaño PM. A randomized controlled trial of daily text messages versus monthly paper diaries to collect bleeding data after intrauterine device insertion. Contraception 2015;92:578–84.
4. Pew Research Center. Mobile health 2012. Available at: http://www.pewinternet.org/files/old-media//Files/Reports/2012/PIP_MobileHealth2012_FINAL.pdf. Retrieved September 6, 2015.
5. Center on Media and Human Development. Teens, health, and technology: a national survey. Available at: http://cmhd.northwestern.edu/wp-content/uploads/2015/05/1886_1_SOC_ConfReport_TeensHealthTech_051115.pdf. Retrieved August 30, 2015.
6. Pew Research Center. U.S. smartphone use in 2015. Available at: http://www.pewinternet.org/2015/04/01/us-smartphone-use-in-2015/. Retrieved June 22, 2015.
7. Tripp N, Hainey K, Liu A, Poulton A, Peek M, Kim J, et al.. An emerging model of maternity care: smartphone, midwife, doctor? Women Birth 2014;27:64–7.
8. Kuehn BM. Is there an app to solve app overload? JAMA 2015;313:1405–7.
9. Aungst T, Clauson KA, Misra S, Lewis L, Husain I. How to identify, assess and utilise mobile medical applications in clinical practice. Int J Clin Pract 2014;68:155–62.
10. IMS Institute for Healthcare Informatics. Patient adoption of mHealth. Parsippany (NJ): IMS Institute for Healthcare Informatics; 2015.
11. Research Now. Are mobile medical apps good for our health? A new study by Research Now reveals that doctors and patients say yes. Available at: http://www.researchnow.com/mobile-medical-apps-good-health-new-study-research-now-reveals-doctors-patients-say-yes-infographic/. Retrieved August 26, 2015.
12. Terry K. Prescribing mobile apps: What to consider. Med Econ 2015;92:35–8, 40.
13. Powell A, Landman AB, Bates DW. In search of a few good apps. JAMA 2014;311:1851–2.
14. Lewis TL. Systematic self-certification model for mobile medical apps. J Med Internet Res 2013;15:e89.
15. Chyjek K, Farag S, Chen KT. Rating pregnancy wheel applications using the APPLICATIONS Scoring System. Obstet Gynecol 2015;125:1478–83.
16. Shulman L. The menstrual cycle. In: Hatcher R, Trussel J, Nelson A, Cates W, Kowal D, Policar MS, editors. Contraceptive technology. 20th ed. New York (NY): Ardent Media; 2011:29–43.
17. Jennings V, Burke AE. Fertility awareness-based methods. In: Hatcher R, Trussel J, Nelson A, Cates W, Kowal D, Policar MS, editors. Contraceptive technology. 20th ed. New York (NY): Ardent Media; 2011:417–34.
18. Boulos M, Brewer AC, Karimkhani C, Buller D, Dellavalle R. Mobile medical and health apps: state of the art, concerns, regulatory control and certification. Online J Public Health Inform 2014;5:e229.
19. West J, Hall PC, Hanson CL, Barnes MD, Giraud-Carrier C, Barrett J. There's an app for that: content analysis of paid health and fitness apps. J Med Internet Res 2012;14:e72.
20. Perry R, Lunde B, Chen KT. An evaluation of contraception mobile applications for providers of family planning services. Available at: http://www.contraceptionjournal.org/article/S0010-7824(15)30148-7/pdf. Retrieved February 29, 2016.
21. Mendiola M, Kalnicki M, Lindenauer S. Valuable features in mobile health apps for patients and consumers: content analysis of apps and user ratings. JMIR Mhealth Uhealth 2015;3:e40.
22. U.S. Food and Drug Administration. Mobile medical applications. Available at: http://www.fda.gov/MedicalDevices/DigitalHealth/MobileMedicalApplications/default.htm. Retrieved September 18, 2014.
23. Albrecht UV. Transparency of health-apps for trust and decision making. J Med Internet Res 2013;15:e277.
24. Pew Research Center. Smartphone ownership—2013 update. Available at: http://pewinternet.org/Reports/2013/Smartphone-Ownership-2013.aspx. Retrieved July 18, 2013.
25. Fiordelli M, Diviani N, Schulz PJ. Mapping mHealth research: a decade of evolution. J Med Internet Res 2013;15:e95.
26. Privacy Rights Clearinghouse. Mobile health and fitness apps: what are the privacy risks? Available at: https://www.privacyrights.org/print/mobile-health-and-fitness-apps-what-are-privacy-risks. Retrieved September 21, 2015.
27. Menstruation in girls and adolescents: using the menstrual cycle as a vital sign. Committee Opinion No. 651. American College of Obstetricians and Gynecologists. Obstet Gynecol 2015;126:e143–6.
Figure

Figure

© 2016 by The American College of Obstetricians and Gynecologists. Published by Wolters Kluwer Health, Inc. All rights reserved.