Surveys are one of the cornerstones of public health research.
Depending on whether questions are conducted by phone, mail, online, or in person, public health surveys can be cost-effective, convenient, and generate high response rates.
However, survey efforts can also encounter problems if there is a lack of adequate planning and development of a specific research question before design and distribution. The absence of thoughtful consideration of the population being studied can also create complications.
Assistant Professor Vicki Hart, who recently joined the UVM Larner College of Medicine on a full-time basis this year, teaches in the online UVM Master of Public Health Program. She also runs Hart Dataworks, which helps health care organizations, non-profits, and small businesses with survey design and data analysis.
Hart, an expert in survey design and data collection, says identifying an objective and creating a target audience are the most critical steps to take in survey design.
Ambiguous, complex, or vague questions and inconsistency in definitions, scale, or wording can compromise the validity of a survey, she says. For questionnaire design, it’s important to focus on the layout, length, and organization as surveys that are too long and poorly organized are problematic.
“After you get the data back, you can’t go back and change your survey,” Hart says. “To me, 90 percent of the work done in a survey should go into the preparation, so you can so you can get best quality data.”
Hart says public health research surveyors should also ask themselves the following questions:
- What exactly do I want to know from the survey?
- What question(s) am I trying to answer with this survey?
- Who are the people I need to gather information from, and who do I not need data from?
- What are the characteristics of those people? Do they live in a particular place? Are they of a certain age? Etc.
Self-Selection Bias in Public Health Research Surveys
Hart says one of the biggest pitfalls with surveys is self-selection bias. Self-selection bias can happen anytime data is collected on a volunteer basis. People with certain characteristics may be drawn to respond to a particular survey. If the people responding are not representative of everyone that you want to understand, then the results could be biased. Hart explains that biased survey results mean we can’t generalize our findings to the rest of the population because not everyone’s views in that population were adequately represented.
“Self-selection is almost impossible to avoid because we often rely on people to volunteer to take our surveys. So it’s not likely you’ll get a perfectly reflected representative sample of population you are trying to understand,” she says. “The key is being aware of the issue so you can take steps to gather the most representative data possible.”
For example, if you’re looking to reach senior citizens, for example, think about where seniors spend their time and what kind of technologies they use—mobile phones, landlines, computers, or none of the above. Hart says it’s ideal to distribute surveys using multiple methods—online, phone, mail, in person—to reach a well-represented population.
Creating Successful Public Health Research Surveys
Even with the challenges that come with designing surveys, public health surveys can help to shape policy which can lead to powerful change.
For example, Hart recently worked with non-profit and health organizations in southern Vermont to understand the public’s perception of Vermont’s new marijuana laws. The results from these public health surveys highlighted a few distinct areas of concern in the community. These concerns are being communicated to local and state lawmakers and will help shape further legislation and community education in response to the new laws.
According to a 2014 study, several factors need to come together for a successful public health campaign. Rigorous, real-time monitoring, evaluation, and program improvement; partnerships and coalitions with public and private sector organizations; communication of accurate and timely information to effect behavior change and engage the community; and political commitment to obtain resources and support for effective action.
Smallpox eradication, reducing the rate of SIDS, tuberculosis control, tobacco control, polio eradication, and others have made progress by addressing these areas.
Not a One-Size-Fits-All Approach
Hart earned her master’s degree in biostatistics from UVM in 2010 and her doctorate in epidemiology from the University of Massachusetts in 2014. Following graduate school, she launched Hart Dataworks. As a statistical consultant, Hart’s research has helped Vermont organizations, including Vermont Works for Women, Change the Story, and others help shape policy.
Hart, who is teaching an epidemiology course at UVM as well as teaching biostatistics, says there is no one-size-fits-all approach to interpret survey data.
“It’s different for everybody because every survey is different and every population is different,” she says. “What works for Vermont wouldn’t work in Micronesia, Zambia, Kenya, or even in New York City. That’s part of the challenge and part of the fun of doing this kind of work is that it has to be very personalized. There are certainly things to think about when designing a survey—such as the questions, objective and target audience—but the answers that come out of the survey will always be different.”
Learn more about the UVM Master of Public Health Program
