Researching Research: A Quest for Valid,

Reliable Measurements
The media is ripe with flashy headlines about the latest health
research. These headlines focus on glamorous-sounding studies. They
tout possible cures to cancer, the key to weight loss, and secrets to
living longer.
However, there is a side to research that gets less attention, despite
being more important.
It is the research that studies research, where the tools and techniques
that make good research possible are developed.
Jennifer Hanson, associate professor and the director of the Didactic
Program in Dietetics at Kansas State University, studies the validity
and reliability of tools used in food safety and dietary intake research.
Hansons food safety research looks at how food-handling behaviors
are influenced by environmental messages, such as warning labels,
foodborne illness news stories, and educational information.
Her research about dietary intake focuses on how dietary omega-3
fatty acid consumption affects the levels found in blood.
Omega-3 fatty acids are naturally occurring fats that are found in high
amounts in fish, she said, and we can measure those levels in your
blood, but how much they appear in your blood varies between
individuals.

These may seem like two unrelated areas, but Hanson said both
dietary practices and food handling behavior involve instrumentation
and trying to figure out what people do when theyre not being
watched.
Hansons interest in instrumentation, or the way researchers measure
variables, stems from her belief that unless the assessment and
measurement tools used in a study are accurate, the results are always
going to come into question.
So you dont know, she said, if you didnt find what you
hypothesized was due to your hypothesis being inaccurate or your
ability to measure what you hoped to measure not being valid or
reliable.
Hanson has faced this in her own research, especially in her studies
about omega-3 fatty acid intake.
When she began studying omega-3 consumption, Hanson assumed the
levels found in blood were good predictors of dietary intake. She
discovered there was not always a good match between blood levels
and dietary consumption. This intrigued her.
I actually have some archived blood samples, she said, so were
trying to get permission to use those to look at genetic differences and
hope that it explains some of the mismatch between diet and blood
levels.
The reliability of blood levels as an indicator of dietary omega-3 intake
was not the only methodology Hanson had to reassess.

The original food frequency questionnaire, or survey, she used to have

subjects estimate their dietary omega-3 intake did not include common
contemporary sources, like omega-3 enriched eggs, milk, and juice.
A food frequency questionnaire is a checklist of foods and drinks
researchers use to measure subjects dietary intake. Subjects mark
how often they have consumed a food or drink over a specified period
of time, like one year.
I primarily use food frequency questionnaires because theres a
difference between episodic and general memory, she said, and it
seems like for your omega-3s its more about your general pattern of
consumption versus your exact consumption over the past 24 hours.
Hanson realized the original questionnaires did a poor job reflecting
subjects actual intake by neglecting newer omega-3 sources.
She solved this problem by developing a food frequency questionnaire
with modern sources of omega-3 fatty acids. Hanson also included
pictures to help subjects recognize foods they may not have realized
contained high amounts of omega-3 fatty acids.
The way Hanson assesses food-handling behaviors is similar to the
food frequency questionnaire.
Its also a basic survey, she said. Weve asked about their selfreported food handling behaviors, so similar to the self-reported dietary
intakes.
Shes chosen a familiar setting to validate the survey.

This fall, were going to observe and try to validate the self-reported
usual habits, she said. Were actually going to some tailgate sites
and were going to observe food handling at the tailgate sites and see
if that kind of goes with what they say theyre doing at home.
A decade of research experience is unnecessary to hypothesize what
they will find there.
Valid and reliable measurements are crucial for producing meaningful
research. How could a scientist claim to have found the key to weight
loss if the scales in their study were twenty pounds off?
Creating and validating research tools may not sound as glamorous as
trying to discover the cure for cancer, but it is what makes that and all
other research possible.