Survey Development: Community Involvement in the Design and Implementation Process

Abstract

Documenting Tribal health priorities is needed to inform research agendas, policy efforts, advocacy, and funding. However, published literature rarely documents the methods used to develop surveys in Indigenous communities. This methods paper includes two objectives: (1) increase knowledge and understanding about the importance of community involvement in public health activities; and (2) provide an example of how the Rocky Mountain Tribal Leaders Council Epidemiology Center (RMTEC) worked with one Tribal community to develop a health priorities survey. This paper describes how the RMTEC worked with a Tribal community and Tribal College students to develop, pilot, and revise a health priorities survey. Recommendations focus on the need for more culturally-responsive survey methods, the importance of building Tribal capacity for health research, and the value of piloting surveys in communities prior to implementation.

Documenting community health needs and priorities is a first step in public health practice. A public health approach includes surveillance, identifying risks and protective factors, developing and evaluating interventions, and implementing services.¹ A primary step is to define public health priorities and needs through the systematic collection of information, which is often achieved through various survey and research methods.

In the 19th century, surveys became predominant in public health as professionals transitioned from reacting to epidemics to improving measures for protection. Sanitary surveys during this time would later be instrumental in justifying the creation of special health boards and agencies to handle societal problems.^2,3 In Tribal communities, the observation and collection of data are part of the Indigenous knowledge system.⁴ For example, Northern Plains Tribes used detailed observations of weather patterns, animal behaviors, and plant conditions to learn and adapt to their environments. Through observation and data collection, Indigenous people became experts in understanding the connections between environmental conditions, behaviors, and the health and vitality of their people.

Many public health professionals who work in Tribal communities utilize a community-based participatory research (CBPR)⁵ approach or Tribal participatory approach.⁶ CBPR is based on building relationships and trust between community members and professionals.⁵ Although CBPR was designed as an approach to research, it has also been used by public health professionals working in community settings to conduct public health practice work, which includes surveys. CBPR supports the codesign of surveys in the public health process where community members work in partnership with professionals to define health needs and to develop health programs and policies in their communities.⁷ A tenet of the CBPR approach is that it allows all members of a team to contribute equally, and it relies on shared decision-making and ownership in the process.⁸

Published literature rarely documents the methods used to develop surveys and implement research in Indigenous communities. CBPR approaches lend themselves to community engagement in the survey design and research process; however, such approaches rarely explain the actual methods used to codesign health surveys with Tribal communities. Learning about survey development or principles of participatory research is far different from practicing these frameworks or partnering with a community to develop or implement a research study.⁹ Tribal communities have their own knowledge systems,¹⁰ and survey development and research in Tribal communities must take these knowledge systems into account.

History of Misuse

Unfortunately, Tribal communities have been negatively impacted by research activities that not only failed to take Indigenous knowledge systems into account but also used methods that were both unethical and culturally inappropriate. In the 1950s, 2 research experiments were conducted that used harmful practices. The first research study was conducted by the US Air Force and recruited 120 non–English-speaking Alaska Natives to ingest radioactive iodine over 200 times, resulting in unsafe exposure to radiation.¹¹ The purpose of this study was to explore the role of the thyroid gland in acclimatizing humans to cold weather.¹² Participants were under the impression that they were receiving medical care, when in fact the experiment had no prospect of medical benefit. This experiment also raised serious concerns about the risk, disclosure, consent, and subject selection. During this time, the US Public Health Service conducted a uranium experiment with Navajo miners to examine how radon in mines impacted health outcomes. Navajo participants were never made aware of the lung cancer risks from exposure to radon.¹³ More recently, in the 1990s, the Havasupai Tribe partnered with researchers at Arizona State University to address high rates of type II diabetes of Tribal members living in a remote area of the Grand Canyon. Researchers went on to use blood samples and DNA from Tribal members to study conditions including schizophrenia, migration, and inbreeding that were unauthorized by Tribal leadership.¹⁴

With this history and context in mind, this article aims to increase knowledge and understanding about the importance of community involvement in public health practice and provide an example of how to develop a health priorities survey in a Tribal community.

Context

The Rocky Mountain Tribal Leaders Council Epidemiology Center (RMTEC) serves as a public health authority for more than 77 000 Tribal members in Montana and Wyoming. Its mission is to empower the American Indians of Montana and Wyoming in the development of services, systems, and epidemiologic capacities to address their public health concerns. The organization uses multisector, community-driven partnerships to provide technical assistance, leadership, program support, and advocacy for its Tribal constituents.

In 2017, RMTEC aimed to document Tribal health priorities to inform future program development, technical assistance, research, policy, advocacy, and funding efforts. Although regional health priorities are established by the Indian Health Service, identifying the immediate priorities of community members would improve efforts to address public health concerns and detect best practice interventions across Tribes.

To achieve this objective, RMTEC developed a health priorities survey in partnership with 1 Tribal community and Health Department, 1 senior researcher, and 2 Tribal college students.

Survey Development Phase

The RMTEC staff created a 12-question initial survey to gauge health priorities and evaluate its programs' services (Figure). The assessment incorporated preexisting resources, including regional health surveys, Healthy People 2020 indicators, the California Tribal Epidemiology Center Health Priorities Survey, and the Alaska Native Tribal Health Consortium's Health Research Priorities tool. RMTEC staff members administered this survey at a Tribal health conference in April 2018. The survey was developed without any feedback from Tribal community members. Survey questions covered a range of topics including Tribal affiliation, reservation versus urban residence, job sector, knowledge and use of RMTEC services, and health priorities. The second part of the survey included open-text response questions designed to elicit feedback on successful health interventions, definitions of research, and research interests. The last question asked respondents for permission to share responses with the public (see Supplement Digital Content 1, available at https://links.lww.com/JPHMP/A574). The results of this initial survey were not acceptable for several reasons. First, the RMTEC team did not codesign the survey with Tribal members and this resulted in questions that were not relevant or answerable by respondents. Second, the survey was administered to a diverse group of conference attendees, many of whom were not Tribal members. This meant that several of the questions that were designed for Tribal members or professionals were left blank and lacked contextual fit.¹⁵ Third, the initial survey was not designed using an iterative process. The iterative process is more culturally responsive and has been described as being carried out with and by local people rather than on them.¹⁶

FIGURE: Survey Development Process

The RMTEC team reviewed scant literature on survey development in Tribal communities. Although limited, one of the more illuminating pieces was Hodge and Lester's⁴ 2006 article, “Indigenous Research: Whose Priority? Journeys and Possibilities of Cross-cultural Research in Geography.” The team worked to actively incorporate the article's recommendations to use reflexivity and cross-cultural methodology. Furthermore, the team acknowledged that the initial survey questions may have predisposed biases toward health priorities based on RMTEC staff perspectives of Indian Country. Community investment in the process of developing survey questions was critical.¹⁷

After several meetings in person, over the phone, and in communities, the RMTEC team drafted a version of the community health priorities assessment to be piloted in the Tribal community. The revised survey included 11 questions. The first 8 questions were fixed-choice responses and included gender, age group, Tribal affiliation, 5-digit zip code, important public health issues related to lack of access to care, disease, environmental conditions, and mental health/substance abuse. One question asked respondents to describe the health of their community using a Likert-type scale from poor to excellent. The last 2 questions were open text and asked respondents to (1) describe successful Tribal interventions to address health issues and (2) list any questions they have about public health. The RMTEC team chose to pilot the tool in one rural Tribal community in Montana. This community was selected on the basis of existing partnerships with the Tribal Public Health Department and the availability of Tribal college interns to assist with the pilot phase of the project (see Supplement Digital Content 2, available at https://links.lww.com/JPHMP/A575).

RMTEC considered the role of Tribal institutional review boards (IRBs) in the survey development process. This was important since RMTEC planned to pilot the survey in one Tribal community with an active IRB. Although most survey development projects do not meet the federal definition of research,¹⁸ Tribal communities have their own definitions of research. Tribal IRBs are unique because they are linked to Tribal governments, sovereignty, self-determination, and cultural knowledge and community protections.¹⁹ Understanding the differences in how Tribal and non-Tribal communities view research is critical for public health practice. Previous investigators have identified 3 major differences.^19,20 First, Tribes are sovereign nations with inherent rights to self-determination based on the 2007 United Nations Declaration on the Rights of Indigenous Peoples.²¹ Tribal sovereignty means that Tribal governments have the authority to speak for their Tribe and are responsible for protecting Tribal knowledge and lands.²⁰ In some Tribal communities, researchers and public health professionals must apply to conduct research and receive permits from the Tribal government. Second, research and public health practice ethics in Tribal communities are value-based, context- and culture-based, and may be subjective.¹⁹ Tribal members may be more vulnerable and experience adverse outcomes related to research or public health practice. Third, data collection and sharing in Tribal communities require that researchers and public health professionals identify how data will be handled beyond the scope of the project, how data will be used, and intellectual property rights.²⁰ Therefore, RMTEC consulted with the community's Tribal IRB of record before piloting the survey. The Tribal IRB indicated that RMTEC could proceed with the development process and requested that RMTEC share results of the pilot with their members.

Protocol approval was not required by an ethics committee for human participant compliance because this was not research.

Survey Pilot Phase

Following published guidelines, the team followed set criteria to pretest the survey: establish intended meaning of questions, agree upon the criteria used to judge appropriateness of questions, select methods for judging appropriateness of methods for survey questions and pilot approach, and review and revise questions based on community context and cultural norms.²² The criteria used in the pilot and revision process included the following: no negative survey questions and double negative answers, only one question at a time, appropriate language for the community, simple questions that are grammatically correct, include local issues and possible health priorities, and questions make sense to everyone.²²

Tribal college interns working on the project piloted the survey with 5 Tribal members who had diverse life experiences and public health views. Tribal members represented various groups in the community including elder, traditional society, young adult, youth and family worker, and mother. The selection of Tribal members was consistent with current literature on survey design, which prioritizes having a sample that accurately represents the population that will be completing the survey rather than having a large sample size.²³ The survey took less than 5 minutes to complete per participant. As Tribal members went through the survey, they asked interns questions and to elaborate on survey items. After participants completed the survey, the interns browsed through their comments and clarified responses when needed. The interns also recorded revisions based on Tribal member feedback.

The interns met afterward to discuss the pilot: what they experienced, what needed to change in the survey, and what general feedback should be reported to RMTEC. Two Tribal members did not have any questions or suggested revisions. One Tribal member did not understand 2 of the questions. Specific suggestions for change related to the use of language that was appropriate for the community. For example, 2 Tribal members suggested the question of “Tribal Affiliation” with a list of Tribes to be modified to “Which tribe are you enrolled in?” Other feedback related to the use of zip codes to identify communities. With small and rural communities, zip codes were not appropriate since Tribal members could live in 2 separate districts but share the same zip code. Tribal members also commented on the listed health priorities items. For example, kidney dialysis was not on the health priorities list but was a major health priority to the community. Similarly, accidents and motor vehicle crashes were listed as environmental issues, but Tribal members felt these were wrongly categorized. Furthermore, language regarding unintended pregnancies was not appropriate for the community. The recommendation was to change this to a lack of sexual health education. Other recommendations were to simplify the survey, provide clear directions at the beginning of the survey, and allow multiple response selections for health priorities. One Tribal member said, “These are all major public health priorities, we cannot simply choose.”

Survey Revision Phase

The 2 Tribal college interns, who were critical in driving the pilot process, relayed their findings to the RMTEC staff in a report. Community participants pointed out several culturally ineffective survey characteristics, which emerged as important considerations for continued survey development (Table).

TABLE - Pilot Participant Feedback, Survey Revisions, and Rationale Behind Changes

Feedback	Participant	Survey Revision	Rationale
Change zip code to prefix and define community	Elder	What town and county do you live in?	Town, County will clarify respondents' distinctive communities
Change Tribal affiliation	Elder society member	Which Tribe are you enrolled in?	Language that is more culturally acceptable
Add kidney dialysis	Elder	Item added to Access to Care	Cross-cultural awareness: Kidney dialysis access was not acknowledged as a priority in original survey
Accidents and MVCs are not environmental issues	Elder	...	Unintentional injuries were left as originally categorized because of best fit
Unintended pregnancies inappropriate language and category	Society member	Contraceptives and Health Education added to Access to Care	Language and jargon that are more culturally appropriate
Add option to select more priorities	Elder society member	Option added to Choose all that apply and rank health issues	There are many major issues facing Tribal communities, which makes it difficult to select only 2 items
Clarify qualitative questions	Mother	...	Questions left as is based on broadness of value received in pilot responses
No questions or suggestions	Youth/family worker young adult male	N/A	N/A

Abbreviations: MVC, motor vehicle crash; N/A, not applicable.

The team collaborated to incorporate these recommended revisions to ensure that any future data collected through the survey would be relevant for informing community-driven health agenda. This process was a back-and-forth progression to absorb Tribal community recommendations in the survey design. A comparison of the survey before and after piloting demonstrates differences in language, values, and the approach used to assess health priorities based on differences in perspective among public health professionals and community members (see Supplement Digital Contents 1 and 3 available at https://links.lww.com/JPHMP/A574 and https://links.lww.com/JPHMP/A576, respectively).

The involvement of community members, Tribal college students, and a senior researcher helped improve the community health priorities assessment (see Supplement Digital Content 3, available at https://links.lww.com/JPHMP/A576). The result was an increased likelihood that the community health priorities assessment would be a valid and reliable measure for the communities served by RMTEC.

Lessons Learned

Effective public health practice in Indigenous communities calls for public health professionals who are participatory-oriented and familiar with Tribal public health practice and research guidelines. Public health professionals who value community partnerships and the trust-building process are critical. Professionals must also view community members as educators and knowledge holders.²⁵ Public health practice and research in Indigenous communities should perform the following:

• Honor the unique language, culture, and history of Tribal communities in the survey design process. This broadens discourse to include Indigenous paradigms and alleviates tensions between communities and professionals.²⁰
• Identify key partners early in the survey development process and compensate community partners for their time and work. Interns were compensated for their time developing and piloting the survey. Community members were not compensated for completing the pilot survey, but this is recommended for future efforts.
• Determine what information is needed and how this information should be collected. Know Tribal-specific guidelines and protocols for collecting data in communities. Keep the survey as short and as simple as possible.
• Collect survey data only if it will be used.
• Integrate community input into surveys through piloting. Failure to pilot surveys may result in a poorly designed survey and poor-quality data. Poor-quality data are not relevant, meaningful, or useful in addressing public health priorities in Tribal communities.

Implications for Policy & Practice

• Tribal epidemiology centers are critical in leading public health efforts to document community health priorities and needs.²⁴ RMTEC's effort to engage community members, Tribal college interns, and a senior researcher in the development of a Tribal health priorities survey provides a participatory model for which other Tribes, professionals, and agencies may follow.
• Results from this process demonstrate the importance of involving community members in public health practice.⁷ In this example, community members helped establish trust, communication, and strengthen relationships between Tribal communities and health organizations. This is consistent with previous research that has found that community engagement in survey development bridges a critical information gap between science and practice.¹⁵
• Key strategies that may be useful for public health professionals as they promote community-engaged partnerships in the development of public health surveys include the following:
• Consult with Tribal IRBs and know community definitions of research, evaluation, and public health practice.
• Pilot surveys in communities prior to implementation. This results in a more meaningful process and quality data.
• Cultivate partnerships between Tribal, private, and community organizations. Partnerships can lead to more culturally responsive survey methods.²⁰
• Seek equity and funding to support the partnership building process and the time it takes to engage community members.

Next Steps

These results underscore the need for culturally responsive survey methods, the importance of building Tribal capacity for public health practice and research, and the value of piloting surveys in communities. Through this effort, 2 Tribal college students learned more about survey development and dissemination of results as coauthors of this article. Five Tribal community members from diverse backgrounds learned more about the survey design by participating in the pilot and follow-up discussions about recommended changes. This process also strengthened the relationship and trust between RMTEC and the Tribal Health Department. Using this process as a guide, RMTEC hopes to institutionalize the engagement of community members, Tribal college students, and Tribal Health Departments in all aspects of the survey development and research process.

Although the team accomplished its objective of developing and piloting a health priorities survey in a Tribal community, RMTEC has delayed use of the survey until additional funding is secured for the project. RMTEC is building on the success of this effort and plans to partner with Tribal colleges to develop a public health associates program with a survey development focus. RMTEC plans to use the process outlined in this article as a template for all surveillance and community-engaged public health practice efforts. The ultimate goal is to support Tribes in their use of data that will inform public health priorities, policies, and research: a survey that will give power and voice to the community members regarding public health issues that matter to them.