Secondary Logo

Journal Logo

Correspondence

Psychometric properties of the self-report instrument for somatic symptoms in general hospitals

Luo, Jia1,2; Tan, Ling1,2; Han, Xue-Qing3; Hu, Yong-Dong4; He, Jian-Hua5; Ma, Yun1,2; Yang, Xiang-Yun1,2; Sun, Jing6; Li, Zhan-Jiang1,2

Editor(s): Ni, Jing

Author Information
doi: 10.1097/CM9.0000000000001615

To the Editor: Medically unexplained physical symptoms (MUPS) and related disorders are especially common in healthcare-seeking populations. A recent meta-analysis revealed the prevalence of somatoform disorders of 34.8% (by International Classification of Diseases) and 26.2% (by Diagnostic and Statistical Manual of Mental Disorders [DSM]) in primary care patients. Further, at least one medically unexplained symptom was detected among 40% to 49% of primary care patients using questionnaires.[1]

Although MUPS and related disorders are painful, disabling, costly, and frequent in healthcare-seeking populations, they often remain unrecognized, and somatoform disorders in the strict sense have been severely underdiagnosed. The mean duration of untreated illness in patients with somatoform disorder is 25.2 years (median 23.1 years).[2] An important reason for the low recognition rate is the lack of an effective screening questionnaire for MUPS and related disorders.

Our research team has developed a multidimensional questionnaire, the self-screening questionnaire for somatic symptoms (SQSS), that includes four dimensions in the Chinese context; somatic symptoms (SS), negative perception (NP), illness behavior (IB), and social function (SF). More details can be found in our previous article.[3] The initial assessment indicated that the SQSS has sufficient reliability and validity, and this was derived from psychiatric patients in a psychiatric hospital.[3] Most patients with somatoform disorders normally visit a general hospital rather than a primary care provider as the first point of formal care contact in China; however, a psychometric evaluation in a large sample of Chinese patients in general hospitals is lacking. This research aimed to extend our previous research in the general hospital to fill this research gap.

The present study had two stages. The first stage was a multicenter cross-sectional study using the SQSS to screen and evaluate the outpatients in three general hospitals in Beijing to further verify the rationality of the scale structure, and to analyze the reliability and validity of the SQSS scale. Participants who met the following criteria were included in the first stage: (1) had visited the gastroenterology, neurology, or cardiology outpatient departments; (2) were aged 18 to 65 years (no gender restriction); (3) had junior high school or above education level; (4) were Beijing residents; (5) had voluntary treatment because of their own concerns about the somatic symptom; and (6) were able to complete the survey questions. Participants were excluded if they (1) had communication difficulties, or language or writing disorders; (2) had cognitive impairment, organic brain disorders, or dementia; (3) had serious mental disorders; or (4) had serious illness that precluded them from being able to complete the questionnaire.

The second stage involved analyzing the optimal cutoff of the SQSS. The inclusion criteria for patients in the second stage were as follows: (1) suffered from anxiety disorder, depression, somatoform disorder, or physical disease without mental health problems; (2) were aged 18 to 65 years; (3) had junior high school or above qualification; and (4) were willing to cooperate with the completion of the questionnaire and sign the informed consent form. The healthy controls without physical disease or mental health problems were enrolled with the same criteria as above (except for the first one) in the second stage. The exclusion criteria for the participants in the second stage were the same as the criteria for the first stage.

All data were collected in two waves between June and December 2017. This study was approved by the Medical Ethics Committee of Beijing Anding Hospital (Ethics batch: [2017] Scientific Research No. [45] 201774FS-2). All participants agreed to participate in this study and signed their informed consent forms.

In the first stage of the study, we used convenience sampling to select the participants. Before implementing the screening, all assessors were trained in research programs and processes to ensure the consistency of screening in each center. The questionnaires used in this study were preliminary development of the SQSS and the patient health questionnaire-15 (PHQ-15). In the second stage of the study, all study participants were evaluated by the psychiatrists to make a definite diagnosis (using the structured clinical interview for DSM-IV-TR AXIS I disorder [SCID-I/P]) and complete the SQSS questionnaire.

For the first sample of 1558 participants, data analysis was conducted using AMOS version 25.0 (Armonk, NY, USA) for confirmatory factor analysis (CFA) to confirm the factor structure. We evaluated the reliability of the SQSS with Cronbach alpha coefficient and Spearman-Brown split-half reliability and the criterion validity of the SQSS scale with PHQ-15. A multi-step analysis of invariance was used to examine whether the structure of the SQSS scale held constant across groups in relation to somatoform disorder, age, and gender.[4] Using the univariate analysis of variance (ANOVA) test, we compared the differences between the somatoform and the non-somatoform disorder groups (using a cutoff score of 29 or higher for the presence of somatoform disorder), between males and females, and between age groups on four factors (1 = SS, 2 = IB, 3 = NP, and 4 = SF). For the second sample of 279 participants, receiver operating characteristic curve analysis was conducted to analyze the sensitivity and specificity. Youden index was used to determine the appropriate SQSS cutoff points.[5] To evaluate the discriminative power of the SQSS for populations with different diseases or healthy people (somatoform disorder, anxiety disorder or depression, physical disease, and health control), we counted the average scores of the subscales and the total scale of the SQSS, and we carried out an F test.

In the first stage, a total of 2700 patients were screened, and 1800 patients agreed to participate in the study. The overall response rate was 67.00%. The main reasons for refusal were “time constraints,” “no psychological problems,” and “no interest.” There were 242 invalid questionnaires (missing answers, multiple choices, or incomplete information) and 1558 valid questionnaires in total. In the second stage, 290 study participants met the criteria for enrollment, and 279 agreed to participate in the study as follows: 72 with somatoform disorders, 63 with depression or anxiety disorders, 72 healthy members without physical or mental disorders, and 72 with physical diseases but no mental disorders.

Considering the results of the exploratory factor analysis in our previous study, an initial structural model was established [Figure 1]. According to the results of the initial model, it can be seen that the relevant indicators met the model fitting criteria. Among the main indicators of the model, Chi-square fit statistics/degree of freedom = 5.508, goodness-of-fit index = 0.936, adjusted goodness-of-fit index = 0.921, normed fit index (NFI) = 0.909, incremental fit index = 0.924, Tucker-Lewis index = 0.913, comparative fit index (CFI) = 0.924 and root-mean-square error of approximation = 0.054, and standardized root mean square residual = 0.039.

Figure 1
Figure 1:
Confirmatory factor model circles represent factors (latent constructs) corroborated by CFA. Rectangles denote SQSS indicators (item banks) that define the various constructs. Curved lines connecting circles indicate the correlations between the connected constructs. Straight lines connecting circles to rectangles indicate the factor loadings (correlations between constructs and the item banks). Numbers to the right of rectangles indicate the unique variances (residual variability for the item bank not accounted for by the factor). CFA: Confirmatory factor analysis.

The correlation coefficients between each item and other items ranged from 0.088 to 0.668, with an average of 0.295. The correlation coefficients between the subscales and the total scale were 0.378 to 0.612 and 0.726 to 0.858 (all P < 0.01). The Cronbach alpha coefficient of the SQSS was 0.899, and Spearman-Brown split-half reliability was 0.865. The Spearman correlation coefficient between the SQSS and the PHQ-15 was 0.683.

In our study, the AUC of the total score of the SQSS was 0.818 (P < 0.001) and the standard error was 0.027. When the cutoff point was 28.5, Youden index reached a maximum of 0.494, the sensitivity was 0.781, and the specificity was 0.714. Thus, we found a cutoff score of 29 to be the best threshold score for somatoform disorder in this study.

Invariance test results demonstrated that the structure of the SQSS applied to different age groups, genders, and the presence or absence of somatoform disorder groups using a cutoff score of 29 or higher to indicate the presence of somatoform disorder. Excellent levels of model fit results, including all error rates <0.05, CFI and NFI indices of >0.90 for age and gender, and 0.81 for somatoform disorder, were shown, which suggest that the structure of the SQSS applied to groups of patients in different age groups, males and females, and people with the presence or absence of somatoform disorder.

ANOVA test results demonstrated that there were overall statistically significant difference among the somatoform disorder, anxiety or depression, physical disease and health control group in each dimension (SS, NP, IB, and SF) of the SQSS and total score. Univariate ANOVA further identified the significant differences were between somatoform disorder and physical disease, and between somatoform disorder and healthy control group. Specifically, Somatoform disorder group had higher scores than physical disease and health group in all dimensions of the SQSS and total score [Supplementary Table 1; http://links.lww.com/CM9/A718]. In addition, anxiety or depression group had higher scores or more somatoform symptoms than people in physical disease and healthy groups in all dimensions of SQSS and total score. It is possible that people in the anxiety or depression group may have comorbid of somatoform symptoms. This further confirmed the results of the ANOVA test that SQSS can discriminate between patients with the presence of somatoform disorders and those with the absence of somatoform disorders.

The SQSS had been locally developed with strong cultural relevance to the Chinese population. This questionnaire had Cronbach alpha coefficient of 0.899 and the correlation of the total scores of all items was above 0.3, indicating that the questionnaire had good internal consistency reliability. Moreover, the Spearman-Brown split-half reliability was 0.865. All fit indices of CFA reached the measurement standard, and the whole model was acceptable. The correlations between the SQSS and the total scores of the PHQ-15 were statistically significant (r = 0.683, P < 0.01). It has good validity and reliability for the patients in general hospitals. The cutoff score of 29 applies to patients with somatoform disorders in Chinese general hospitals.

Funding

This research was supported by a grant from the Capital Health Development Research Projects (No. Capital development 2016-1-2121).

Conflicts of interest

None.

References

1. Haller H, Cramer H, Lauche R, Dobos G. Somatoform disorders and medically unexplained symptoms in primary care. Dtsch Arztebl Int 2015; 112:279–287. doi: 10.3238/arztebl.2015.0279.
2. Herzog A, Shedden-Mora MC, Jordan P, Löwe B. Duration of untreated illness in patients with somatoform disorders. J Psychosom Res 2018; 107:1–6. doi: 10.1016/j.jpsychores.2018.01.011.
3. Tang L, Li ZJ, Luo J, Yang XY, Su X, Yang XJ, et al. Preliminary development of self-screening questionnaire for somatoform symptoms (in Chinese). Chin J Behav Med Brain Sci 2019; 28:464–469. doi: 10.3760/cma.j.issn.1674-6554.2019.05.016.
4. Hu LT, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct Equ Model 1999; 6:1–55. doi: 10.1080/10705519909540118.
5. Lee SY, Park HS, Kim DJ, Han JH, Kim SM, Cho GJ, et al. Appropriate waist circumference cutoff points for central obesity in Korean adults. Diabetes Res Clin Pract 2007; 75:72–80. doi: 10.1016/j.diabres.2006.04.013.

Supplemental Digital Content

Copyright © 2021 The Chinese Medical Association, produced by Wolters Kluwer, Inc. under the CC-BY-NC-ND license.