Measuring Total and Domain-Specific Sitting

A Study of Reliability and Validity

Abstract

Purpose: 

Although independent relationships between sitting behaviors (mainly television viewing) and health outcomes have been reported, few studies have examined the measurement properties of self-report sitting questions. This study assessed gender-specific test-retest reliability and validity of a questionnaire that assessed time spent sitting on weekdays and weekend days: 1) traveling to and from places, 2) at work, 3) watching television, 4) using a computer at home, and 5) for leisure, not including television.

Methods: 

Test-retest reliability of domain-specific sitting time (min·d⁻¹) on weekdays and weekend days was assessed using data collected on two occasions (median = 11 d apart). Validity of domain-specific self-reported sitting time on weekdays and weekend days was assessed against log data and sedentary accelerometer data.

Results: 

Complete repeat questionnaire and log data were obtained from 157 women (aged 51-59 yr) and 96 men (aged 45-63 yr). Reliability coefficients were high for weekday sitting time at work, watching television, and using a computer at home (r = 0.84-0.78) but lower for weekend days across all domains (r = 0.23-0.74). Validity coefficients were highest for weekday sitting time at work and using a computer at home (r = 0.69-0.74). With the exception of computer use and watching television for women, validity of the weekend-day sitting time items was low.

Conclusions: 

This study confirms the importance of measuring domain- and day-specific sitting time. The measurement properties of questions that assess structured domain-specific and weekday sitting time were acceptable and may be used in future studies that aim to elucidate associations between domain-specific sitting and health outcomes.

In light of current research findings that demonstrate independent relationships between sedentary behaviors and health outcomes (^6,7), interest in the accurate assessment of time in sedentary behaviors has increased (²). The term sedentary behavior is used to define behaviors for which energy expenditure is low, typically between 1 and 1.5 METs (such as sitting), as opposed to being defined as "inactive" (¹). Accurate and reliable self-report measures of sedentary behaviors are now essential, especially for large-scale epidemiological and intervention studies where the use of more objective measures may be impractical.

Most of the work in this field has focused on leisure time sedentary behavior, presumably because of the potential to change this with health promotion initiatives. However, in a recent review of 60 articles that reported measurement of leisure time sedentary behavior, Clark et al. (⁴) found that only nine studies assessed reliability and only three examined the validity of the self-report measures used. Reliability coefficients were generally good (ranging from 0.32 to 0.93, with the majority >0.70), but there was wide variation in validity coefficients (from −0.19 to 0.80), depending on the criterion measure used (⁴). The majority of these studies focused entirely on the measurement of time spent sitting while watching television or using a computer at home. One study examined seven additional leisure time contexts (e.g., socializing, reading, listening to music) (¹⁴), and two measured sitting time while at work or during travel to and from work (^11,13). Other important domains for sitting time include at work (^11,17) and in travel to and from places.

As few studies have assessed the measurement properties of items that measure domain-specific sitting, the aim of this study was to assess the test-retest reliability and the criterion validity of five items to assess sitting time while (a) traveling to and from places, (b) at work, (c) watching television, (d) using a computer at home, and (e) in leisure time but not including watching television. To isolate sedentary behaviors in a way that is not possible using accelerometers alone, both behavior log and accelerometer data were used to assess validity. It was hypothesized that time reported in more habitual or regular sitting activities (e.g., at work or watching television) would be more reliably and accurately recalled than time spent sitting in less structured activities, such as that which might occur during leisure time or on weekends.

METHODS

Participant Recruitment

The Australian Longitudinal Study on Women's Health (ALSWH) monitors factors that affect the health and the well-being of Australian women. ALSWH participants were randomly selected from the national Medicare health insurance database (which includes all permanent residents of Australia) in 1995 (³). All Brisbane-based participants in the midage cohort of the ALSWH (N = 228) were invited to participate in this study by mail. Women (aged 53-59 yr in 2005) were given the option to decline participation by telephone (free call 1800 number). Those who did not opt out at this stage (n = 210) were telephoned, provided with more study information, and asked to provide verbal consent to participate. Up to 10 telephone call attempts were made before a participant was deemed uncontactable. Consenting women (n = 160) were also asked if they could identify and obtain permission to provide contact details for a midaged man to be invited to participate in the study. Contact details were provided for 70 men who were subsequently invited to participate. Additional men were also recruited via advertisements in local workplaces.

Ethical clearance for this study was obtained from the Human Research Ethics Committee at the University of Newcastle (H-963-0205) and from the Medical Research Ethics Committee at The University of Queensland (2005000201).

Procedure

Consenting participants were mailed and asked to complete a questionnaire that assessed sitting time adapted from the work of Miller and Brown (¹¹), physical activity, and standard sociodemographic and health characteristics (T1 assessment). They were then visited by a research assistant within 3 wk. During this visit, participants were given a behavior log and asked to record sitting times each day for the next week. The log had space for the participant to record the date and time they spent sitting (in the five domains assessed in the self-report questionnaire) over 7 d. Participants were asked to complete the log at the end of each day. When they reached the end of the log (day 7), participants were asked to complete another questionnaire that included the sitting time questions (T2 assessment) then return it in the mail with the behavior log. Participants were asked not to refer to the log when completing the T2 questionnaire. Participants who did not return the materials after 3 wk were reminded by telephone.

A random subsample of women (n = 56) and men (n = 18) were also given verbal and written instructions on how to wear an MTI ActiGraph accelerometer (model GT1M, Pensacola, FL) and asked to wear it during all waking hours for seven consecutive days, removing it only during water-based activities. They were asked to record the time the accelerometer was put on and taken off each day in the log and to return it at the end of the week with the other materials.

Measures

Demographic and physical characteristics.

Using standard items from the ALSWH (³), all participants were asked to provide information about their height and weight, date of birth (to calculate age), educational qualifications, marital status, employment status, and overall health status.

Questionnaire sitting time.

Five items were used to assess time spent sitting (hours and minutes) each day in the following domains: (a) while traveling to and from places (e.g., work, shops); (b) while at work; (c) while watching television; (d) while using a computer at home; and (e) at leisure not including watching television (e.g., visiting friends, movies, eating out) on a weekday and a weekend day. Our adaptation of the original four-item questionnaire (¹¹) included separating watching television and using a computer at home into two questions (see Appendix).

Logbook sitting time.

Participants were asked to record in the logbook the time they spent sitting (hours and minutes) each day across the domains assessed in the questionnaire.

Accelerometer sitting time.

The accelerometer was set to record movement counts per 1-min epoch. Accelerometer data were considered valid if there was more than 600 min of monitoring per day (excluding strings of zeros 20 min or longer), four or less bouts of 20-min strings of zeros recorded per day (because these data strings suggest nonwear time), and at least five valid days of monitoring, one of which had to be a weekend day. This amount of accelerometer data is considered to be sufficient to determine habitual activity (¹⁵).

Data Preparation and Analysis

Data were manually entered into a database by one researcher and checked by a second; discrepancies were checked and corrected on the basis of the original data collection sheets. All data preparation and analyses were conducted using the Statistical Package for the Social Sciences for Windows (Version 15.0; SPSS Inc., Chicago, IL) and used P < 0.05 to assess statistical significance. Descriptive statistics (means, standard deviations, and frequencies) were calculated to describe the demographic and the health-related characteristics for men and women.

Preparation.

Data from the sitting time questionnaire were used to create an estimate of total weekday and weekend-day sitting times (min·d⁻¹) by summing the time reported in each domain. The log data enabled the validity of domain-specific sitting time items to be assessed separately for weekdays and weekend days. The time recorded in each sitting category in the participant's log was summed and averaged separately for weekdays and weekend days (min·d⁻¹) to enable a direct comparison with the average weekday and weekend-day estimates of sitting time reported in the questionnaire.

Recorded accelerometer counts <100 were used to estimate time spent sitting (⁵). Time recorded sitting on each valid weekday or weekend day was summed and divided by the valid number of days to provide separate estimates of time spent sitting per weekday and weekend day (min·d⁻¹).

Descriptive statistics (means, medians, and interquartile ranges) were calculated for reported sitting time in each domain. Gender-specific test-retest reliability of the self-report domain and day-specific sitting time data were visually assessed using the Bland-Altman plots and statistically assessed using paired t-tests to determine the significance of the difference between individuals' T1 and T2 data. Where there was no statistically significant difference between the T1 and the T2 data (i.e., the difference was not significantly different from zero), the intraclass correlation coefficient (ICC) and the 95% confidence intervals (95% CI) of the coefficient were calculated. The ICC and the 95% CI were interpreted as follows: ICC < 0.4 indicated poor repeatability, 0.4-0.75 indicated fair to good repeatability, and ICC > 0.75 indicated excellent repeatability (¹³). Spearman correlation coefficients (r) for reported sitting time at T1 and T2 were also calculated for comparability with reports of previous test-test reliability studies.

To assess validity, the same statistical methods were used; however, the comparisons were 1) between domain- and day-specific self-reported sitting at T1 versus domain- and day-specific log data and 2) between total self-reported sitting during a weekday and a weekend day at T1 versus total accelerometer recorded sitting time.

RESULTS

Participants.

Of the 228 women eligible to participate, 18 were unable to be contacted, 45 declined, and 5 cancelled their appointments. Of the 160 who agreed to participate (76% of those contacted), 157 provided complete questionnaire sitting data at T1 and T2 for the reliability analysis and complete log data for use in the validity analysis. Of the 56 women assigned to wear an accelerometer, 48 agreed, 45 returned the monitors, and 44 (79%) provided usable data for the validity analysis.

Of the 70 nominated men, 62 gave consent to participate and 57 provided complete data. Another 39 men were identified via workplace recruitment. In total, 96 men provided repeat questionnaire sitting time data for the reliability analysis and complete sitting time log data for the validity analysis. Of the 18 men assigned to wear an accelerometer, 12 agreed and 11 provided usable data for the validity analysis; however, because the sample is insufficient to be able to draw meaningful conclusions, no further analysis of these data was conducted.

All women were aged between 50 and 59 yr, but there was a wider age range in men, up to 65 yr (Table 1). Most participants (regardless of gender) reported they were married, had post-high school education, were employed fulltime, and had good to excellent health (Table 1).

TABLE 1: Participant sample characteristics.

Time reported sitting in the questionnaire.

On weekdays, both men and women reported more time sitting at work than that in any other domain, with men sitting for about 50% more time at work than women (Table 2 for women's data and Table 3 for men's data). For both men and women, median sitting values were 2 h per weekday and 3 h per weekend day watching television, 1 h on a weekday and 2 h on weekend days for both travel and other leisure activities, and 0.5 h using a computer at home (data not shown). On weekend days, more time was reported sitting for watching television and leisure and less at work. Sitting time during travel and home-based computer use were similar on weekdays and weekend days (Tables 2 and 3).

TABLE 2: Descriptive profile and test-retest repeatability of the continuous sitting items among 157 women.

TABLE 3: Descriptive profile and test-retest repeatability of the continuous sitting items among 96 men.

Test-retest reliability.

The median time between T1 and T2 questionnaire completion was 11 d (range = 7-28 d, with two participants' data excluded because they were outside this range (35 and 43 d)). The Bland-Altman plots of the more reliable domain- and day-specific T1 and T2 data are shown in Figure 1 (women) and Figure 2 (men). These figures show that the distribution of error is equally spread along the range of time reported sitting at work. The plots for the less reliable questions (described below) showed that the data were less reliable as time reported sitting increased.

FIGURE 1: Test-retest repeatability Bland-Altman plots for 157 women: self-report work-related sitting items on a weekday (A) and weekend day (B). The y axis is the difference between the two (T1 − T2) assessments, and the x axis is the average of the two assessments ((T1+T2)/2).

FIGURE 2: Test-retest repeatability Bland-Altman plots for 96 men: self-report work-related sitting items on a weekday (A) and weekend day (B). The y axis is the difference between the two (T1 − T2) assessments, and the x axis is the average of the two assessments ((T1 + T2)/2).

For women, the differences between T1 and T2 weekday sitting times at work and using a computer at home were not statistically significantly different from zero. The corresponding ICC (95% CI) were good to excellent (Table 2). Reported times for travel and watching television were significantly higher at T2 than at T1 (mean difference = 17-18 min), but the ICC was higher for television time (0.71) than for travel (0.26; Table 2). The differences between T1 and T2 weekend-day sitting times at work, watching television, using a computer at home, and other leisure were not statistically significantly different from zero. However, the only ICC (95% CI) that fell within the fair to excellent repeatability range was using a computer at home; the rest were poor to fair. In general, women's sitting time data were less repeatable for weekend-day recall than for weekday recall (Table 2).

In most instances, the repeatability coefficients for the men's data were better than those observed for the women. For men, the differences between T1 and T2 weekday sitting times at work, watching television, using a computer at home, and other leisure were not statistically significantly different from zero (Table 3). However, the only ICC (95% CI) within the fair to excellent range were for sitting time at work, watching television, and using a computer at home. Sitting time during leisure appears less repeatable, with the lower limit of the 95% CI falling below 0.4. None of the men's domain-specific sitting time differences between T1 and T2 weekend day were statistically significantly different from zero, but the only fair to excellent ICC (95% CI) were for watching television and using a computer at home. Consistent with the findings for women, correlations were lower for weekend days than for weekdays in men.

Validity.

Validity of domain- and day-specific self-report sitting time data was assessed against log records, and validity of total day-specific self-report data were assessed against the accelerometer data. For the weekday domain-specific women's data, the differences between the T1 self-report and the log-recorded sitting times during travel, at work, watching television, and other leisure were not statistically significantly different from zero (Table 4). However, only the ICC (95% CI) for sitting time at work and watching television were within the fair to excellent range. None of the women's weekend-day sitting items could be considered valid (ICC < 0.37; Table 4).

TABLE 4: Validity coefficients (r, ICC) and descriptive profiles of time reported in the sitting questionnaire (Q data) at time 1 and the 7-d behavior log (Log data) for 157 women.

For men, the only domain- and day-specific difference between the T1 self-report versus the log-recorded sitting time was using a computer at home (Table 5). However, the only ICC (95% CI) in the fair to excellent range were for sitting time on a weekday for travel, at work, and using a computer at home. None of the men's weekend-day sitting time items could be considered valid (Table 5). With the exception of the comparison between the total weekday and the total weekend-day self-report against the accelerometer data (described below), the Bland-Altman validity plots showed similar patterns to the reliability plots.

TABLE 5: Validity coefficients (r, ICC) and descriptive profiles of time reported in the sitting questionnaire (Q data) at time 1 and the 7-d behavior log (Log data) for 96 men.

The Bland-Altman plots for the comparisons between the women's total weekday and weekend-day-specific T1 self-report sitting time data and the day-specific MTI-recorded data indicated there would be low levels of agreement between the two sets of data (Figure 3). As expected, the difference between the two measures for total weekday sitting time was statistically significant (total weekday mean difference = −63.6, 95% CI = −115.1 to −12.07); however, the difference between total weekend-day sitting times was not statistically significant (mean difference = 10.8, 95% CI = −52.6 to 74.2), but the confidence interval was quite large, resulting in a very low ICC of 0.04.

FIGURE 3: Bland-Altman plots for (A) weekday and (B) weekend-daysitting time in 44 women. The y axis is the difference between the two (T1 - MTI data) assessments, and the x axis is the average of the two assessments ((T1 + MTI)/2). The solid line shows the mean difference in minutes, and the dashed lines show the 95% CI of agreement.

DISCUSSION

This study assessed the test-retest reliability and validity (using both logbook and accelerometer data as criterion measures) of questionnaire items designed to assess time spent sitting in five domains on weekdays and weekend days in a sample of middle-aged Australian men and women. For men, test-retest reliability was high for weekday sitting time at work, watching television, and using a computer at home, and for women, it was high for sitting time at work and using a computer at home. The reliability estimates were generally lower for weekend-day sitting time across all domains: the exceptions being using a computer at home for both men and women and watching television for men. Clark et al. (⁴) have previously noted that assessment of weekday television sitting behavior was more reliable than weekend day. In terms of the validity coefficients observed in this study, the coefficients were highest on weekdays, especially for time reported sitting at work, using a computer at home, and travel for men and for work and watching television for women.

In terms of validity of the weekend-day sitting time data, none of the coefficients observed in this study reached acceptable levels relative to the log-recorded data or MTI-recorded data. However, for the weekday data, time reported sitting at work and watching television was valid against the log-recorded data for men, and time reported sitting for travel, at work, and using a computer at home achieved good to excellent correlation coefficients for women. For women, the validity of weekday reported sitting time watching the television was fair. However, the comparison between the self-report and the MTI data should be interpreted with caution because of several reasons. First, the intended comparisons were planned in good faith and informed by the minimum-wear criteria applied to the MTI data offered for the assessment of physical activity (minimum of 10 h·d⁻¹) (¹⁵). However, it might be that most adults could be awake for up to 16 h·d⁻¹; therefore, the MTI data may underrepresent the amount of time the person was sitting, particularly in the evenings. We investigated applying a longer-wear criterion to our sample (14 h·d⁻¹), but the sample available for analysis was reduced below a meaningful level. Second, to enable the comparison between the day-specific sitting time and the MTI-recorded data, estimates of total sitting on a weekday and on a weekend day were created. Because we cannot be certain that the time reported sitting in each of the domain-specific sitting questions is in fact mutually exclusive particularly for other leisure, we cannot recommend summing the individual items in this way. Finally, the pragmatic <100 count cut point applied to accelerometer data as an indicator of sitting time has never been validated in its own right and thus may not be a truly accurate estimate of time spent sitting. Research clarifying how to obtain meaningful MTI accelerometer data to accurately assess sitting time is required, including the minimum level of data required to represent total time spent sitting.

Our data confirm our hypothesis that time spent in more habitual or routine activities is more accurately recalled than time spent in less structured activities. This means that sitting time for travel to and from work and at work may be more regular than sitting for leisure activities (other than watching television) and more so in men than in women, who may work more irregular or variable hours (¹¹). Sitting time across all domains appears to be less reliably and validly reported on the weekend than for weekdays, which probably reflects the more variable nature of activities performed on weekends in general.

Overall, the reliability coefficients were similar to those reported in previous studies, which showed high reliability for television viewing and computer use at home (^9,10,16,17) and for work-related sitting (^11,13), and lower reliability for less structured activities such as relaxing and hobbies (¹⁴). To some extent, we would expect that reliability coefficients might vary with the test-retest interval because if the two recall periods do not overlap, it is impossible to tell whether lower coefficients are explained by difficulties in recall or actual change in behavior. However, in this study, the 11-d test-retest coefficients for weekday work hours (0.81 in women and 0.84 in men) were similar to the 7-d coefficient reported for a Queensland workplace sample (0.76) (¹¹). Similarly, the Spearman coefficients observed in our study for weekday television sitting time (0.79 in women and 0.82 in men) were similar with those reported by Wareham et al. (¹⁶) in a 3-month test-retest trial (0.78 for women and 0.75 for men) to evaluate the EPIC Norfolk questionnaire. It would appear, therefore, that higher reliability coefficients are observed for regular or consistent activities, regardless of the recall period.

Validity of domain-specific sitting time questions has rarely been reported. Overall, the validity coefficients (ρ) reported here were similar to or in some cases (e.g., for television and computer use at home) slightly better than those reported for other leisure activities (0.20 for reading to 0.30 for television and 0.60 for computer use at home) (¹⁴).

In our study, validity coefficients for the questionnaire data were higher when compared with the log data than the accelerometer data. This may reflect the log data's susceptibility to similar self-report biases as the questionnaire data or that the log data more precisely reflect actual behavior: as noted earlier, the accelerometer data were only able to provide a generic estimate of minimal movement in a day. Our validity coefficients against the accelerometer were low and much lower than those reported by Matton et al. (¹⁰), who used a combination of accelerometer and diary data to validate television sitting time in the Flemish Computerized Physical Activity Questionnaire. Their validity coefficients (ICC) were 0.69 for employed men and 0.83 for employed women; both television time and validity coefficients were higher in retired men and women (¹⁰). These higher validity coefficients reflect the fact that instead of interpreting all counts of <100 as sitting time, the Belgian researchers linked each 1-min accelerometer epoch data to the diary time denoted as television time so that they could specifically estimate the validity of the television sitting estimates. Unfortunately, our brief log was not able to provide sufficient detail to replicate these analyses.

In view of the recently reported links between sitting time for watching television and health indicators (^6,7), the finding that time spent sitting at work was about 50% greater than time spent sitting watching television is important. Jans et al. (⁸) recently reported that one-third of total daily sitting time is attributable to occupational sitting and that those with higher occupational sitting time do not necessarily compensate by sitting less during leisure time. If the recent suggestion that sitting time may be as important a risk factor for chronic health problems as physical activity is true (¹²), then it will be important to measure time spent sitting in all domains of sitting throughout the day in future studies. The results presented here show that sitting at work, while watching television and using a computer at home, can be reasonably accurately recorded by men and women. More work is needed to assess the measurement properties of sitting time during travel. The lower coefficients reported here may simply reflect variability in travel times (which is feasible given variable traffic conditions) rather than recall and reporting problems.

The major limitation of this study was that all the participants were middle-aged, and the women had slightly higher levels of education than the rest of the ALSWH cohort, which is more representative of the general population (³). We cannot therefore apply the findings of this study to younger or older people or those with lower levels of education, and further work is required with these groups. Working with the ALSWH participants, who now have a long history of completing questionnaires, may also affect the generalizability of the reliability and validity estimates to "untrained" samples. As in all reliability studies, there is the possibility that the first administration acts as a primer for recall in the second. In this study, it might also be true that those who completed the log would have better recall of the previous week's activities. However, if this as the case, we might have expected sitting times in all domains to be higher at the second survey, and this was not the case. The sample of men was a convenience sample, volunteered to the study by women participants. This resulted in a slightly older group of men with higher than average education levels. Finally, we were unable to control the time between questionnaire administrations, and in a few instances this was quite large. Unfortunately, this is one of the realities of working with community samples: it is not easy to get full compliance with the study protocols. Some people were not available when they said they would be; however, this in essence reinforces the external validity of the study findings.

Overall, the results presented here suggest that the measurement properties of these domain-specific sitting time questions are acceptable, but only for weekday sitting time when activities are likely to be more routine than at weekends. In light of the large amount of time spent sitting at work in contemporary societies and given the high reliability and validity of responses to the question about sitting at work, we suggest that the addition of these questions would strengthen future work and may help to elucidate the associations between sitting time and health outcomes.

This study was supported with funding from the Brisbane City Council, the Queensland Health, and the National Health and Medical Research Council (ID 301200). ALM was supported by an NHMRC Career Development Award (grant No. 553000). NWB was supported by a program grant (ID 301200) and a capacity building grant (ID 252977) from the National Health and Medical Research Council. YDM was supported by a capacity building grant from the National Health and Medical Research Council (ID 252977) and a Postdoctoral Research Fellowship from The University of Queensland. Mr. Anthony Walsh recruited the male participants from work sites and was responsible for the data collection. The authors are grateful to the participants who provided additional data for this study, including women from the ALSWH.

The authors also thank Dr. Stewart Trost who wrote the software for converting the raw accelerometer data into usable summary variables.

The findings of this study as presented in this manuscript do not constitute endorsement by the American College of Sports Medicine.

Conflicts of interest: The authors have no conflicts of interest to declare.