The Role of Symptom Clusters in Triage of COVID-19 Patients : Quality Management in Healthcare

Journal Logo

Original Research

The Role of Symptom Clusters in Triage of COVID-19 Patients

Wojtusiak, Janusz PhD; Bagais, Wejdan MS; Vang, Jee PhD; Guralnik, Elina MPH; Roess, Amira PhD; Alemi, Farrokh PhD

Author Information
Quality Management in Health Care 32(Supplement 1):p S21-S28, January/March 2023. | DOI: 10.1097/QMH.0000000000000399


Diagnosis of COVID-19 in the community continues to be fraught with difficulties. COVID-19 is a systemic disease that presents with a variety of symptoms.1,2 New variants of SARS-CoV-2 may manifest different symptoms.3 Patient age groups have been shown to present with different COVID-19 symptoms.4,5 Early and late progression of COVID-19 have different manifestations.6,7 The order of occurrence of symptoms may matter.8,9 Furthermore, simple rules of prediction based on symptoms most frequently associated with the disease are poor predictors of COVID-19.10 Furthermore, algorithms that assume that symptoms are independent of each other have proven to have low accuracy.8 One way to resolve these challenges in diagnosing COVID-19 is to examine clusters of symptoms.

Symptom clusters occur in many diseases, from chronic kidney disease11 to cancer,12 to COVID-19.13 Symptoms are often correlated with each other.14,15 For example, nausea-vomiting, anxiety-depression, and dyspnea-cough clusters are common in patients with cancer.16 Symptom clusters can differentiate response to treatment better than the clinician's diagnosis.17 For example, symptom clusters affect quality of life18 and patients' ability to function.19 It is evident from the published literature that symptom clusters matter. The current study examines whether symptom clusters can improve accuracy of triage decisions for patients suspected to have COVID-19.

This study compares 2 approaches for constructing symptom clusters: (1) hierarchical clustering and (2) use of interaction terms. Historically, scientists have used hierarchical clustering20,21 to combine variables and create clusters. In the second method, we rely on 2- to 5-way, statistically significant, interaction terms in the regression equation to define clusters of symptoms. Others have also pursued similar methods for clustering.22,23 In the second method, clusters indicate subgroups of patients not only with similar etiology but also with similar outcome and clinical response.24


Source of data

Data were obtained from 483 patients who had taken a laboratory COVID-19 test, recruited through online listservs (with permission of listserv administrator). Participants learned about this study via e-mails to neighborhood listservs in the Washington, District of Columbia. Study team members also shared the link to the survey with their personal networks. Individuals were directed to access the survey following a link at the end of the invitation message/e-mails. Before completing the survey, participants were required to complete the study consent.

Data were collected between November 2020 and January 2021, prior to widespread vaccination and emergence of the Delta variant of COVID-19. Participants were eligible if they were 18 years of age or older and had a COVID-19 test within 30 days of participation in the study. Participants with inconclusive or pending test results were excluded from the analyses, resulting in 461 patients in the final analysis set. The majority of patients (90%) had at least 1 symptom before taking the test.

Dependent variable

Study participants self-reported the result of their COVID-19 laboratory test. The models developed here predict whether the patient has COVID-19 or not. For patients who do not have COVID-19, the model does not determine which gastrointestinal, inflammatory, neurological, or other diagnosis is consistent with the patients' symptoms.

Independent variables

Study participants self-reported their current symptoms, history of chronic symptoms, and demographic characteristics (age, gender, race, and ethnicity). The 29 examined symptoms from different types of infections (and their prevalence in the study population) are shown in Table 1.

Table 1. - Frequency of Symptoms in the Study Sample
  1. General symptoms

    1. Fever or feeling feverish (62%)

    2. Muscle aches/myalgia (35%)

    3. Pinkeye/conjunctivitis (5%)

    4. Fatigue (more than usual) (38%)

    5. Chills (20%)

  2. Neurological symptoms

    1. Headaches (45%)

    2. Loss of balance (9%)

    3. New confusion (6%)

    4. Unusual shivering or shaking (11%)

    5. Loss of smell (8%)

    6. Loss of taste (20%)

    7. Numbness (7%)

    8. Slurred speech (3%)

  3. Gastrointestinal symptoms

    1. Diarrhea (28%)

    2. Stomach/abdominal pain (14%)

    3. Change in or loss of appetite (23%)

    4. Nausea or vomiting (16%)

  1. 4. Inflammatory symptoms

    1. Joint/other unexplained pain (myalgia/arthralgia) (18%)

    2. Red/purple rash or lesions on toes (5%)

    3. Unexplained rashes (5%)

    4. Excessive sweating (7%)

    5. Bluish lips or face (1%)

  2. 5. Respiratory symptoms

    1. Cough (57%)

    2. Sore throat (38%)

    3. Difficulty breathing (dyspnea) (21%)

    4. Shortness of breath (hypoxia) (20%)

    5. Runny nose (rhinorrhea/nasal symptoms) (58%)

    6. Chest pain (chest tightness) (19%)

    7. Wheezing (23%)

The history of chronic symptoms included 4 variables measuring history of symptoms due to chronic conditions: (1) history of respiratory (present in 21% of patients), (2) history of inflammatory (present in 6% of patients), (3) history of neurological (present in 9% of patients), and (4) history of gastrointestinal symptoms (present in 11% of patients).

Model construction

We examined the relationship between symptoms and COVID-19 test results using the Least Absolute Shrinkage and Selection Operator (LASSO) regression models. The response variable was the COVID-19 test result. In regressions, robust predictors were identified by randomly selecting an 80% sample of training data and repeating LASSO regressions 23 times. Variables that remained in 95% of the regressions were kept as robust variables. In addition, in all LASSO regressions, we focused on large effects. The variable was ignored if the absolute value of the regression coefficient did not exceed 0.01.

It is often useful to compare the performance of the model with accuracy of other models or clinical expectations. We have reported the accuracy of naïve models in a separate publication.10 A naïve Bayes model was not accurate enough to be used in a clinical setting. In contrast, the LASSO regression model provides a high enough cross-validated accuracy that can be used in clinical triage decisions. The main rationale for selecting LASSO regression is its ability to perform variable selection and straight-forward interpretation of coefficients. In contrast, methods such as ridge regression or elastic net perform “smoothing” of coefficients to remove extreme values, thus not allowing for the direct interpretation of the coefficients. In a separate article in this supplement, we provide details of how the cluster of symptoms can also be used to increase accuracy of home tests.25

Different clustering approaches

We examined 3 different methods of constructing symptom clusters:

  1. Main effect model: COVID-19 test results were LASSO regressed on main effects (with no clusters or interaction terms) of symptoms, demographics (age, race, and ethnicity), and history of symptoms due to a chronic condition. Robust predictors were then selected as those that are most predictive of COVID-19 diagnosis. The main effect model assumes that no symptom cluster is used in predicting COVID-19 (see Supplemental Digital Content 2, available at:, for complete analysis).
  2. Hierarchical clustering model: We combined predictors using hierarchical clustering and then built models for predicting COVID-19 using the identified clusters. Agglomerative clustering was applied to create a hierarchy of symptom clusters. Norm-1 similarity was used in which distance between 2 symptoms was defined as the number of people in the data who do not share the same symptom (other similarity measures, including Jaccard, led to comparable results). Hierarchical clustering was done separately for positive and negative COVID-19 cases. An optimal cutoff point in the hierarchical dendrogram was identified that maximized the accuracy of the models in the training data (see Supplemental Digital Content 3, available at:, for complete analysis).
  3. Interaction-terms model: Symptom clusters were identified through interaction terms in LASSO regression. As before, the response variable was COVID-19 test result. The independent variables were single, pair, and triplets of demographics, symptoms, and history of chronic symptoms.

For all models, robust LASSO regression was used to select the predictors. The regularization strength (lambda) was optimized to maximize cross-validated accuracy within the training set (see Supplemental Digital Content 4, available at:, for complete analysis).


The proposed regression models were developed in 80% of the data and cross-validated in 20% of the data set aside for validation. The process was repeated 30 times for randomly selected subsets. The average area under the receiver operating characteristic curve (AUROC) was reported as the measure of cross-validated accuracy of predictions.

Online supplements

Python source code and complete analysis are available in Jupyter Notebook and HTML format. Supplemental Digital Content 1, available at:, includes common functions and supplements 2 to 4 include construction and testing of specific models.


Table 2 describes the demographic distribution of the study participants. Most participants were White, Non-Hispanic/Latino, and between 25 and 44 years of age. A large proportion of the study participants (42%) were health care or essential workers.

Table 2. - Characteristics of Study Sample
Variable Number of Cases (%)
COVID-19 test results
Negative 330 (68.32)
Positive 131 (27.12)
Results pending 15 (3.11)
Inconclusive 7 (1.45)
Age, y
18-24 84 (17.39)
25-34 210 (43.48)
35-44 156 (32.30)
45-54 20 (4.43)
55-84 13 (2.69)
Female 279 (57.76)
Male 203 (42.03)
Hispanic Latino 60 (12.42)
Non-Hispanic/Latino 401 (83.02)
Unknown 22 (4.55)
Other 18 (3.75)
Asian 25 (5.18)
Black or African American 60 (12.42)
White 380 (78.67)

The hierarchical clusters of COVID-19-positive cases are shown in the dendrogram in Figure 1. Similarly, the hierarchical clusters of COVID-19-negative cases are shown in Figure 2. Comparison of the figures shows clear differences in clusters between positive and negative cases. The optimal number of clusters when using hierarchical clusters was 15, most of which were single-predictor clusters.

Figure 1.:
Hierarchical clustering among COVID-19-positive patients.
Figure 2.:
Hierarchical clusters for COVID-19-negative patients.

Table 3 provides coefficients from LASSO logistic regression models. Empty cells correspond to the variables dropped by LASSO or not present in the first place. The main effect model had average AUROC of 0.78 (variance: 0.00199), the lowest accuracy level. In this model, all predictors (except those aged 30 years or older) increased probability of COVID-19. By themselves, headache (1.06), chest pain (0.69), cough (0.43), and loss of taste (0.38) were the strongest predictors of COVID-19 diagnosis.

Table 3. - Robust LASSO Logistic Regressions
Model Main Effects Interaction-Term Clusters Hierarchical Clustering
Average area under the receiver operating characteristic curve 0.78 0.81 0.79
Variance of area under the receiver operating characteristic curve 0.0019 0.0027 0.0029
Average sensitivity 0.64 0.67 0.64
Average specificity 0.82 0.85 0.86
Total number of possible predictors 47 17 343 23
Number of robust predictors 11 23 10
Coefficients of LASSO Regression Models
Predictors Main Effect Interaction-Term Clusters Hierarchical Clustering
Intercept −2.21 −1.77 −2.03
Cough 0.43
Chills 0.37
Headaches 1.06 0.61
Joint pain 0.35
Chest pain 0.69
Runny nose −0.55
Loss of smell 0.17
Loss of taste 0.38 0.77
Loss of appetite 0.27
Difficulty breathing 0.20
Muscle aches 0.22
History of respiratory symptoms 0.24
Age, 30+ y −0.26 −0.55
White 0.11 0.38
Female 0.33
Cough, fever, runny nose, and headaches 1.57
Chest pain, difficulty breathing, wheezing, chills, and shortness of breath 2.22
Fever and age, 30+ y −0.44
Muscle aches and headaches 0.69
Numbness and non-Hispanic/Latino −0.57
Runny nose and non-Hispanic/Latino −0.91
Cough, fever, and runny nose 0.54
Cough, loss of taste, and fever 0.32
Headaches, cough, and runny nose 0.59
Headaches, cough, and White 0.64
Headaches, chills, and White 0.63
Sore throat, fever, and runny nose 0.47
Fatigue, chest pain, and White 1.56
Cough, loss of taste, and runny nose 0.06
Sore throat, chills, and female −0.21
Loss of appetite, cough, and White 0.47
Loss of smell, cough, and loss of taste 0.91
Cough, age 18-29 y, and female 0.36
Fatigue, cough, and non-Hispanic/Latino −0.26
Pinkeye, headaches, and non-Hispanic/Latino 0.71
History of respiratory symptoms, cough, and runny nose 0.20
History of respiratory symptoms, cough, and White 0.60
History of respiratory symptoms, muscle aches, and cough −0.63
Headaches, loss of taste, and non-Hispanic/Latino 0.39
History of respiratory symptoms, muscle aches, and age 30+ y −0.86
Abbreviation: LASSO, Least Absolute Shrinkage and Selection Operator.

Table 3 shows the average AUROC, sensitivity, and specificity for the 3 models. When hierarchical clustering was used as predictors of test results, the average AUROC was 0.79 (variance: 0.00295). At the most optimal cutoff point, the hierarchical cluster technique identified 10 clusters, 8 of which were individual predictors. The 2 combinations of symptoms were (1) “cough, fever, runny nose, and headaches,” with regression coefficient of 1.57; and (2) “chest pain, difficulty breathing, wheezing, chills, and shortness of breath,” with regression coefficient of 2.22.

The interaction-term cluster model had average AUROC of 0.81 (variance: 0.00279), the highest AUROC value (see Table 3). Surprisingly, no main effects were present in this model, showing that symptom clusters were more predictive of COVID-19 than any single symptom. The strongest predictor positively associated with COVID-19 diagnosis was “fatigue and chest pain among White individuals,” with regression coefficient of 1.56. The strongest predictor that was negatively associated with predicting COVID-19 was “runny nose, observed among non-Hispanic or Latino individuals,” with regression coefficient of −0.91.

The impact of symptom clusters on diagnosis of COVID-19 depended on the age, gender, and race of the patient, as patients in different demographic subgroups presented with different symptom clusters. Table 3 shows symptom clusters in distinct demographic subgroups. Race was a factor in 7 clusters. Ethnicity was a factor in 6 clusters. Age appeared in 2 clusters. Likewise, female gender appeared in 2 clusters.

In 30 bootstrap samples, the interaction-term cluster model was significantly more accurate than the main-effect model (paired t-statistic 5.43, 29 df, P = .00). The Hierarchical Clustering model was not more accurate than the main-effect model (paired t-statistic = 1.50, df = 29, P = .07).

Figure 3 visualizes the LASSO regression with clusters using network modeling procedures.26 All clusters that predict COVID-19 are shown, yellow nodes show clusters that increase the probability and blue rectangles show clusters that reduce the probability of COVID-19. The impact of listed single symptoms, single history of chronic symptom, or single demographic characteristics is mediated through these clusters. It is important to note that 10 single symptoms were not present in any of the clusters and are not listed in Figure 3. Those were slurred speech, diarrhea, bluish lips, confusion, unexplained rash, shivering, abdominal pain, excessive sweating, loss of balance, and red rash. In addition, history of neurological, gastrointestinal, and immune function symptoms did not show in the final regression equation and is not listed in Figure 3.

Figure 3.:
Bayesian network model for predicting COVID-19.


The interaction-term model has a cross-validated AUROC of 0.81, which is a relatively high level of accuracy, and can be a clinically meaningful screening method for COVID-19. This level of accuracy is comparable to in-home rapid antigen test results.25 Currently, the Centers for Disease Control and Prevention website provides a list of symptoms and no guidance on how to use these symptoms in diagnosis of COVID-19.27 Now that COVID-19 is endemic, clinicians need advice on how to diagnose COVID-19 from its symptoms. To date, this has not been possible because symptom screening has not been accurate enough to be used in clinics or triage decisions. This study showed that symptom clusters can improve the accuracy of diagnosis of COVID-19 to levels that are more acceptable in clinical management of patients.

We acknowledge that the interaction-term model developed here (and the associated symptom clusters), while accurate, is difficult to use by unaided clinicians. The existence of smartphones, and access of a large portion of the population to the Internet, suggests that these models can be made part of an internet service. A prototype of how this might look like is available at Patients can describe their symptoms to the service; it can score symptom clusters and report whether the patient is likely to have COVID-19. The patients can then report the service's presumed diagnosis to their clinician, who can use the service's findings as any laboratory test. It may be possible to overcome the difficulty of implementing the proposed screening of symptom clusters using an Internet service.

The interaction-term symptom cluster model was more accurate than the main-effect model, suggesting that clusters and not individual symptoms are helpful in diagnosis of COVID-19. The improvement in AUROC was small but more pronounced when examining sensitivity/specificity: At constant specificity (0.93), cluster of symptoms increased sensitivity by 9 points (from 0.45 to 0.55) compared with the sensitivity of the model built on individual symptoms. In addition, the LASSO regression with interaction terms excluded any single symptom, suggesting that symptom clusters are more predictive than single symptoms.

There are many explanations why symptom clusters can make diagnosis of COVID-19 more accurate. Clusters of symptoms are combinations of symptoms that typically do not occur in many patients but when they do, they are highly predictive and add more information to the diagnosis task. Similar findings have been reported in the literature, where symptom clusters, not single symptoms, determine the patient's response to depression treatment.28

A review of statistically significant variables in the interaction-term model suggests that based on age, gender, race, and ethnicity, different symptom clusters are useful in diagnosing COVID-19. For example, cough among 18- to 29-year-old women was predictive of COVID-19 (coefficient of 0.36). Similarly, runny nose among non-Hispanic/Latinos was helpful in ruling out COVID-19 (also see the study by Larsen et al.8). These and other clusters show that symptoms of COVID-19 may vary on the basis of demographic characteristics. There are many reasons for this, and younger patients have fewer comorbidities and may not express the same symptoms as older patients for the same disease.29 Men and women differ in expression of symptoms for same underlying disease.30,31 Finally, racial differences in presentation of diseases have been known to occur. This study contributes to the growing literature that symptoms must be evaluated within age/gender/race groups.


This study had several limitations. First, the models that were constructed relied on a relatively small data sample, collected between October 2020 and January 2021, with only 1 variant of SARS-CoV-2 virus known to be prevalent at the time of data collection. Larger sample size may allow for discovery of more combinations of symptoms uniquely present in COVID-19 patients. Bigger data many include variables excluded by current model but reported in the literature.19

The participants' age and gender distribution in our study sample does not correspond to the distribution of age and gender within the US population. Our findings may not generalize to US population.

COVID-19 test results were self-reported and established by a variety of diagnostic tests. This may have introduced bias in accurately assessing the true positive COVID-19 test results.


1. Menni C, Valdes AM, Freidin MB, et al. Real-time tracking of self-reported symptoms to predict potential COVID-19. Nat Med. 2020;26(7):1037–1040.
2. Alemi F, Guralnik E, Vang J, et al. Guidelines for triage of COVID-19 patients presenting with multi-systemic symptoms. Special Issue: Diagnosis of COVID-19 in the Community. Qual Manag Health Care. 2023;32(1 suppl):S3–S10. doi:10.1097/QMH.0000000000000398
3. Mahase E. Covid-19: sore throat, fatigue, and myalgia are more common with new UK variant. BMJ. 2021;372. doi:10.1136/bmj.n288.
4. Trevisan C, Noale M, Prinelli F, et al. Age-related changes in clinical presentation of Covid-19: the EPICOVID19 Web-based survey. Eur J Intern Med. 2021;86:41–47. doi:10.1016/j.ejim.2021.01.028.
5. Jin S, An H, Zhou T, et al. Sex- and age-specific clinical and immunological features of coronavirus disease 2019. PLoS Pathog. 2021;17(3):e1009420. doi:10.1371/journal.ppat.1009420.
6. Chen J, Qi T, Liu L, et al. Clinical progression of patients with COVID-19 in Shanghai, China. J Infect. 2020;80(5):e1–e6. doi:10.1016/j.jinf.2020.03.004.
7. Wagner T, Shweta F, Murugadoss K, et al. Augmented curation of clinical notes from a massive EHR system reveals symptoms of impending COVID-19 diagnosis. eLife. 2020;9:e58227. doi:10.7554/elife.58227.
8. Larsen JR, Martin MR, Martin JD, Kuhn P, Hicks JB. Modeling the onset of symptoms of COVID-19. Front Public Health. 2020;8,473.
9. Wojtusiak J, Bagais W, Vang J, Roess A, Alemi F. Order of occurrence of COVID-19 symptoms. Special Issue: Diagnosis of COVID-19 in the Community. Qual Manag Health Care. 2023;32(1 suppl):S11–S20. doi: 10.1097/QMH.0000000000000397.
10. Alemi F, Vang J, Guralnik E, Roess A. Modeling the probability of COVID-19 based on symptom screening and prevalence of influenza and influenza-like illnesses. Qual Manag Health Care. 2022;31(2):85–91. doi:10.1097/QMH.0000000000000339.
11. Lockwood MB, Chung S, Puzantian H, et al. Symptom cluster science in chronic kidney disease: a literature review. West J Nurs Res. 2019;41(7):1056–1091.
12. George MA, Lustberg MB, Orchard TS. Psychoneurological symptom cluster in breast cancer: the role of inflammation and diet. Breast Cancer Res Treat. 2020;184(1):1–9.
13. Yifan T, Ying L, Chunhong G, et al. Symptom cluster of ICU nurses treating COVID-19 pneumonia patients in Wuhan, China. J Pain Symptom Manage. 2020;60(1):e48–e53.
14. Miaskowski C, Dodd M, Lee K. Symptom clusters: the new frontier in symptom management research. J Natl Cancer Inst Monogr. 2004(32):17–21.
15. Barsevick AM, Aktas A. Cancer symptom cluster research: new perspectives and tools. Curr Opin Support Palliat Care. 2013;7(1):36–37.
16. Kirkova J, Aktas A, Walsh D, Davis MP. Cancer symptom clusters: clinical and research methodology. J Palliat Med. 2011;14(10):1149–1166.
17. Wise J. Covid-19: study reveals six clusters of symptoms that could be used as a clinical prediction tool. BMJ. 2020;370:m2911. doi:10.1136/bmj.m2911.
18. Miaskowski C, Aouizerat BE, Dodd M, Cooper B. Conceptual issues in symptom clusters research and their implications for quality-of-life assessment in patients with cancer. J Natl Cancer Inst Monogr. 2007;(37):39–46. doi:10.1093/jncimonographs/lgm003
19. Dodd MJ, Miaskowski C, Paul SM. Symptom clusters and their effect on the functional status of patients with cancer. Oncol Nurs Forum. 2001;28(3):465–470.
20. Johnson SC. Hierarchical clustering schemes. Psychometrika. 1967;32(3):241–254.
21. Murtagh F, Contreras P. Algorithms for hierarchical clustering: an overview. Wiley Interdiscip Rev Data Mining Knowl Discov. 2012;2(1):86–97.
22. Michalski RS. Pattern Recognition as Knowledge-Guided Computer Induction (Tech Rep No. 927). Urbana-Champaign, IL: University of Illinois, Department of Computer Science; 1978.
23. Wnek J. Constructive induction. In: Seel NM, eds. Encyclopedia of the Sciences of Learning. Boston, MA: Springer; 2012.
24. Miaskowski C, Barsevick A, Berger A, et al. Advancing symptom science through symptom cluster research: expert panel proceedings and recommendations. J Natl Cancer Inst. 2017;109(4):djw253.
25. Alemi F, Vang J, Bagais W, et al. Combined symptom screening and at-home tests for COVID-19. Special Issue: Diagnosis of COVID-19 in the Community. Qual Manag Health Care. 2023;32(1 suppl):S11–S20. doi: 10.1097/QMH.0000000000000404
26. Alemi F. Constructing causal networks through regressions: a tutorial. Qual Manag Health Care. 2020;29(4):270–278. doi:10.1097/QMH.0000000000000272
27. Centers for Disease Control and Prevention. Similarities and differences between flu and COVID-19. Last Reviewed January 18, 2022. Accessed November 19, 2020.
28. Bondar J, Caye A, Chekroud AM, Kieling C. Symptom clusters in adolescent depression and differential response to treatment: a secondary analysis of the Treatment for Adolescents with Depression Study randomised trial. Lancet Psychiatry. 2020;7(4):337–343.
29. Mueller AL, McNamara MS, Sinclair DA. Why does COVID-19 disproportionately affect older people? Aging (Albany NY). 2020;12(10):9959–9981. doi:10.18632/aging.103344
30. Cavanagh A, Wilson CJ, Kavanagh DJ, Caputi P. Differences in the expression of symptoms in men versus women with depression: a systematic review and meta-analysis. Harv Rev Psychiatry. 2017;25(1):29–38. doi:10.1097/HRP.0000000000000128
31. Regitz-Zagrosek V. Sex and gender differences in health. EMBO Rep. 2012;13(7):596–603. doi:10.1038/embor.2012.87.

clusters of symptoms; COVID-19 diagnosis; hierarchical clustering; LASSO regression

Supplemental Digital Content

© 2023 Wolters Kluwer Health, Inc. All rights reserved.