Of the 12 items within the job and worksite characteristics section (section 1), 7 had almost perfect reliability, 4 had substantial reliability, and 1 had moderate reliability (ie, item about the number of days missed because of problems with physical or mental health). All of the neighborhood around workplace items (section 2) and organizational and social environment supports items (section 7) had substantial reliability. Of the 30 items related to programs, facilities, and policies (section 3), 6 items had almost perfect reliability, 18 had substantial reliability, and 6 had moderate reliability. The reliability of 2 items that assessed use/participation in physical activity breaks during meetings and incentives to walk/bike to work could not be calculated because of an insufficient number of respondents reporting these supports. Both of the items that assessed the presence of these supports had moderate reliability. The 4 other items with moderate reliability were those that assessed use of or participation in other worksite supports (eg, use of outdoor exercise facilities or flexible time for physical activity during the work day). The cafeteria and vending items were asked among the respondents who reported having a cafeteria, snack bar, or food service (n = 47) or vending machine (n = 84). Of the cafeteria items (section 4), 1 had almost perfect, 6 had substantial, and 5 had moderate reliability. Items with moderate reliability included ones assessing availability of the following in the cafeteria: nonfried meat entrees, low-fat snack items, low-fat dairy products, food in smaller or half-sized portions, and posters/signs that encourage healthy eating. Section 5 had 2 items with almost perfect reliability and 3 with substantial reliability. The meals section (section 6) had 1 item with substantial reliability (frequency of bringing lunch to work) and 4 with moderate reliability (frequency of purchasing meals at a cafeteria/food service, sit-down restaurant, fast food restaurant, or other type of restaurant). Test-retest reliability and internal consistency of the WEBS scales were strong. Deletion of the 7 items with moderate reliability did not alter the internal consistency of their respective scales (ie, cafeteria scale and program, facilities, and policies scale).
When stratified by obesity status (obese vs nonobese), 3 items had an insufficient number of respondents to calculate stratum-specific reliability coefficients. Thirty items had reliability coefficients that differed by more than |0.20| between obese and nonobese participants, and half of the differences were higher among obese participants with the other half higher among nonobese participants. Fourteen of the notable differences in reliability coefficients were in the workplace programs, facilities, and policies section. In general, most of the differences were among items that were asked among respondents who had reported a specified worksite support (ie, with small numbers) and/or items that were asked about a behavior that may have changed between assessment periods (eg, frequency of fast-food consumption).
Three items had an insufficient number of respondents to calculate reliability coefficients by size of worksite (<100 employees or ≥100 employees). There were 26 instances in which the coefficients for an item were more than |0.20| different by size of worksite (data available upon request). Nine coefficients were higher among respondents from larger worksites, and 17 coefficients were higher among participants from smaller worksites. The program, facilities, and policies section had 11 of the 26 reliability differences, and 9 of those showed a higher reliability among the smaller worksites than the larger worksites. Eight of the differences in reliability coefficients in this section were for questions that asked about participation/use of a program or facility. Nine additional differences were found for items about food and beverage offerings among respondents with a cafeteria or food service. All but 2 of these had reliability coefficients that were higher among respondents employed in smaller worksites.
This study demonstrated high reliability of the WEBS among a diverse sample of working adults. Relationships between some of these worksite support measures and physical activity behavior have been observed in previous cross-sectional studies,7 , 8 , 12 – 14 but the psychometric properties of the measures were unknown. To our knowledge, no other self-report instruments for assessing worksite supports for healthy eating have been tested; thus, this study contributes to a growing literature on measurement of the food environment in different settings.15
All WEBS items had reliability coefficients that exceeded 0.40, with 80% of items exhibiting test-retest reliability coefficients above 0.60. The test-retest reliability coefficients were similar to findings from other studies of surveys used to assess worksite environments for physical activity22 , 38 as well as neighborhood environments. For example, the worksite neighborhood items and scale in the WEBS showed similar test-retest reliability as the home neighborhood environment items from which they were adapted.28 Altogether, of the 16 items with coefficients in the moderate range (0.40-0.59) across all sections, 9 items assessed behaviors or conditions, which had the potential to vary between assessment periods; thus, the lower reliability may reflect true changes in behavior or health condition rather than measurement error. Moreover, we observed a reduction in responses of “do not know” at retest. Taking the survey could have been enough to raise the awareness of some respondents about their worksite environment and policies, and in the time between the test and retest, respondents may have sought an answer to a survey question. This testing effect is only a concern if the tool were used as part of a pre- and posttest assessment for a worksite intervention.
Reliability differed for some items by obesity status and size of worksite. There were more items for which reliability was higher for respondents at smaller worksites (17 of 26 items) than for respondents at larger worksites. No clear patterns in reliability emerged by obesity status. Overall, the strength of coefficients within the various strata supports the reliability of the WEBS for diverse populations in worksites of various sizes.
A few limitations deserve mention. First, further evaluation is needed to document the validity of the WEBS and associations with physical activity and diet outcomes. Second, the relatively long length of the WEBS may preclude its adoption into surveillance systems. Yet, the WEBS can be shortened in the future when more evidence is available about associations with physical activity, diet, and obesity. Third, use or adaptation of existing items was a priority, resulting in some inconsistencies in response options. Specifically, factual items in the WEBS typically used a yes/no response format (eg, programs, facilities, and policies), with the exception of items pertaining to the neighborhood around the workplace, which used a Likert scale response because they were derived from an existing survey.7 , 14 Fourth, the sampling methods may limit generalizability in that participants were required to work at 1 primary location and possess a landline telephone; thus, wireless-only households were not represented.39
1. US Chamber of Commerce and Partnership for Prevention. Leading by Example: Leading Practices for Employee Health Management. Washington, DC: Partnership for Prevention; 2007.
2. Combs S. Gaining Costs, Losing Time: The Obesity Crisis in Texas. Austin, TX: Office of the Comptroller; 2011.
3. Ickes M, Sharma M. Worksite health promotion: a practical strategy for obesity prevention. Am J Health Stud. 2009;24:343–352.
5. Anderson LM, Quinn TA, Glanz K, et al. The effectiveness of worksite nutrition and physical activity interventions for controlling employee overweight and obesity: a systematic review. Am J Prev Med. 2009;37(4):340–357.
6. Pratt CA, Lemon SC, Fernandez ID, et al. Design characteristics of worksite environmental interventions for obesity prevention. Obesity (Silver Spring). 2007;15(9):2171–2180.
7. Crespo NC, Sallis JF, Conway TL, Saelens BE, Frank LD. Worksite physical activity policies and environments in relation to employee physical activity. Am J Health Promot. 2011;25(4):264–271.
8. Dodson EA, Lovegreen SL, Elliott MB, Haire-Joshu D, Brownson RC. Worksite policies and environments supporting physical activity in midwestern communities. Am J Health Promot. 2008;23(1):51–55.
9. Verweij LM, Coffeng J, van Mechelen W, Proper KI. Meta-analyses of workplace physical activity and dietary behaviour interventions on weight outcomes. Obes Rev. 2011;12(6):406–429.
10. Matson-Koffman DM, Brownstein JN, Neiner JA, Greaney ML. A site-specific literature review of policy and environmental interventions that promote physical activity and nutrition for cardiovascular health: what works? Am J Health Promot. 2005;19(3):167–193.
11. Engbers LH, van Poppel MN, Chin APMJ, van Mechelen W. Worksite health promotion programs with environmental changes: a systematic review. Am J Prev Med. 2005;29(1):61–70.
12. Lucove JC, Huston SL, Evenson KR. Workers' perceptions about worksite policies and environments and their association with leisure-time physical activity. Am J Health Promot. 2007;21(3):196–200.
13. Kaczynski AT, Bopp MJ, Wittman P. Association of workplace supports with active commuting. Prev Chronic Dis. 2010;7(6):A127.
14. Schwartz MA, Aytur SA, Evenson KR, Rodriguez DA. Are perceptions about worksite neighborhoods and policies associated with walking? Am J Health Promot. 2009;24(2):146–151.
15. McKinnon RA, Reedy J, Morrissette MA, Lytle LA, Yaroch AL. Measures of the food environment: a compilation of the literature, 1990-2007. Am J Prev Med. 2009;36(4 suppl):S124–S133.
16. Dejoy DM, Wilson MG, Goetzel RZ, et al. Development of the Environmental Assessment Tool (EAT) to measure organizational physical and social support for worksite obesity prevention programs. J Occup Environ Med. 2008;50(2):126–137.
17. Oldenburg B, Sallis JF, Harris D, Owen N. Checklist of Health Promotion Environments at Worksites (CHEW): development and measurement characteristics. Am J Health Promot. 2002;16(5):288–299.
18. Dannenberg AL, Cramer TW, Gibson CJ. Assessing the walkability of the workplace: a new audit tool. Am J Health Promot. 2005;20(1):39–44.
19. Shimotsu ST, French SA, Gerlach AF, Hannan PJ. Worksite environment physical activity and healthy food choices: measurement of the worksite food and physical activity environment at four metropolitan bus garages. Int J Behav Nutr Phys Act. 2007;4:17.
20. Kim S, Adamson KC, Balfanz DR, et al. Development of the Community Healthy Living Index: a tool to foster healthy environments for the prevention of obesity and chronic disease. Prev Med. 2010;50(suppl 1):S80–S85.
21. Hoebbel C, Golaszewski T, Swanson M, Dorn J. Associations between the worksite environment and perceived health culture. Am J Health Promot. 2012;26(5):301–304.
22. Blunt GH, Hallam JS. The worksite supportive environments for active living survey: development and psychometric properties. Am J Health Promot. 2010;25(1):48–57.
23. Voss C, Klein S, Glanz K, Clawson M. Nutrition environment measures survey-vending: development, dissemination, and reliability. Health Promot Pract. 2012;13(4):425–430.
24. Centers for Disease Control and Prevention, National Center for Health Statistics. National Health and Nutrition Examination Survey Questionnaire
. Hyattsville, MD: US Department of Health and Human Services, Centers for Disease Control and Prevention; 2009.
25. Hill EJ, Hawkins AJ, Ferris M, Weitzman M. Finding an extra day a week: the positive influence of perceived job flexibility on work and family life balance. Fam Relat. 2001;50(1):49–58.
26. Kessler RC, Barber C, Beck A, et al. The World Health Organization Health and Work Performance Questionnaire (HPQ). J Occup Environ Med. 2003;45(2):156–174.
28. Sallis JF, Kerr J, Carlson JA, et al. Evaluating a brief self-report measure of neighborhood environments for physical activity research and surveillance: Physical Activity Neighborhood Environment Scale (PANES). J Phys Act Health. 2010;7(4):533–540.
29. Moore LV, Diez Roux AV, Nettleton JA, Jacobs DR, Franco M. Fast-food consumption, diet quality, and neighborhood exposure to fast food: the multi-ethnic study of atherosclerosis. Am J Epidemiol. 2009;170(1):29–36.
30. Maxwell AE, Yancey AK, AuYoung M, et al. A midpoint process evaluation of the Los Angeles Basin Racial and Ethnic Approaches to Community Health Across the US (REACH US) Disparities Center, 2007-2009. Prev Chronic Dis 2011;8(5):A115. http://www.cdc.gov/pcd/issues/2011/sep/10_0187.htm
. Accessed August 9, 2012.
32. Jobe JB, Mingay DJ. Cognitive research improves questionnaires. Am J Public Health. 1989;79(8):1053–1055.
33. Forsyth BH, Lessler JT. Cognitive laboratory methods: a taxonomy. In: Biemer PP, Groves RM, Lyberg LE, Mathiowetz NA, Sudman S, eds. Measurement Errors in Surveys. New York, NY: Wiley-Interscience; 1991:395–418.
34. Frank LD, Sallis JF, Saelens BE, et al. The development of a walkability index: application to the Neighborhood Quality of Life Study. Br J Sports Med. 2010;44(13):924–933. doi: 10.1136/bjsm.2009.058701.
35. Sallis JF, Bowles HR, Bauman A, et al. Neighborhood environments and physical activity among adults in 11 countries. Am J Prev Med. 2009;36(6):484–490.
36. Cronbach LJ. Coefficient alpha and the internal structure of tests. Psychometrika. 1951;16:297–334.
37. Landis J, Koch G. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159–174.
38. Brownson RC, Chang JJ, Eyler AA, et al. Measuring the environment for friendliness toward physical activity: a comparison of the reliability of 3 questionnaires. Am J Public Health. 2004;94:473–483.
40. Heinen L, Darling H. Addressing obesity in the workplace: the role of employers. Milbank Q. 2009;87(1):101–122.