Eliot Sperling

Professor Winter

TA Welch

PLAP 3270

April 2017

Millennials: Guns & The Second Amendment

Survey Experiment & Research Paper

Research Question:

Does terminology affect the way collegiate millennials feel about guns and gun control in

the United States?

Both the control and treatment surveys consisted of 12 comprehensive questions. These

surveys were identical except for two manipulations; wording and background information

provided. By conducting this survey experiment, I wanted to gain some insight into whether or

not wording has a real, noticeable effect on the way collegiate millennials (people ages 18-24,

who are currently attending a College or University) reported feeling about guns and gun control

in the United States. Similar to the study done conducted by Price, Vincent, Lilach Nir, and

Joseph N. “Framing Public Discussion of Gay Civil Unions.” My focus was on the effects of

framing and terminology. I chose to focus on terminology because I feel, that in the media and

21st century society, there is a generally negative connotation surrounding the word “gun”. By

changing the word “gun” to “firearm” on the treatment survey, I was hoping to gain insight into

whether or not public opinion would differ based on wording.

I chose to study and conduct a survey experiment on guns and gun control because

over the past few years, guns have been a major part of political and policy discussion.

Additionally, I believe that opinions surrounding guns and gun control are largely not private and

1
that people would be more willing to answer questions about guns rather than more private

beliefs such as sexuality or race. Both surveys were anonymous to protect the opinions of the

respondents and hopefully encourage truthful answers. Elisabeth Noelle-Neumann defined

public opinion as “opinions on controversial issues that one can express in public without

isolating oneself”. By conducting an anonymous survey experiment respondents were able to

answer truthfully knowing that their responses would be confidential.

Hypothesis

When collegiate millennials are presented with the term “firearm” rather than “gun,” they

will respond more favorably to guns and gun control.

Methodology & Sampling Strategy:

To gather responses, I posted in several different Facebook groups such as: The

University of Virginia (UVA) class of 2019 and 2020 and The Duke University (Duke) class of

2019 and 2020 as well as the North Carolina State University (NC State) class of 2019 and

2020 pages. Additionally, I sent emails to a couple listservs (University of Virginia men’s rugby

club and my fraternity listserv) that I am on, here at UVA. When posting and emailing different

groups, I asked for those interested in participating to respond directly to me (Facebook and

email) or like the post (Facebook) and then I would flip a coin and provide them with the survey

that corresponded with the face of the coin (heads = control, tails = treatment). I was able to

limit the amount of responses that fell outside the 17-24 age range by posting in groups and

listservs that are for current college students only. Initially, the age range was meant to be 18-24

however after discussing with several different students here at UVA, I learned that it is common

2
for first years (freshmen) to be 17. To include a larger sample population the age range was

changed.

In total, across both surveys, 113 responses were collected; 47 responses on the control

survey and 66 responses on the treatment survey. I believe my survey was effective in

gathering responses from collegiate millennials. Future researchers can easily replicate this

survey experiment. Posting in public Facebook groups is easy and requires no permission.

However, future researchers will not be able to email the same listservs I did as they are private

but the process can be replicated.

Initially when I started collecting responses I was worried that I might struggle to obtain

35 responses per survey; however, the students of UVA, Duke and NC State were very

receptive to my survey and were eager to help. Both surveys had a total of 12 questions, all of

which were multiple choice (except for age which was fill in the blank). I believe students were

willing to complete the surveys because they were not only short but they did not require

respondents to formulate their own responses. This is both a positive and negative. Positive

because people are willing to complete short surveys that require little effort. However, the

negative is that respondents were limited in their responses and might have been unable to

properly voice their opinions.

The survey experiment was conducted via Google Drive. Google Drive proved to be the

best program to use as it has very few restrictions and allows researchers to freely conduct

surveys. By using this online program I was able to create a free survey that would not record

names or emails of the respondents. Additionally, using Google Drive allowed me to create two

separate, automatically linked and updated response sheets.

Demographics

3
How old are you?

17 y/o 18 y/o 19 y/o 20 y/o 21 y/o 22 y/o Total

Control
Count 11 19 9 7 1 0 47
Percent 23.4% 40.4% 19.1% 15% 2.1% 0% 100%

Treatment
Count 2 24 27 10 2 1 66
Percent 3% 36.4% 40.9% 15.2% 3% 1.5% 100%

Total
Count 13 43 36 17 3 1 113
Percent 11.5% 38.1% 31.9% 15% 2.7% 0.8% 100%
Target mean: 19.5 years old
Control mean: 18.32 years old
Treatment mean: 18.83 years old
Overall sample mean: 18.58 years old

Initially, when I began drafting this survey experiment I had a target age range of 18-30,

however after much discussion with other students and a little bit of research into what age

college students are, the target age range was modified to 17-22. The minimum parameter was

lowered from 18 to 17 to include a wider range of college students, specifically first years

(freshmen). The maximum parameter was also lowered because I wanted to gauge collegiate

millennials and in most cases, students over the age of 22 are in graduate school, there are

exceptions but nonetheless, 22 was set. With a range of five, the survey was able to capture a

wide variety of ages of collegiate students across all four years.

Across both surveys the average age of respondents is 18.58, this is not surprising. The

surveys were primarily posted in Facebook groups that were for students graduating in either

2019 or 2020. The sample average is slightly below the target mean of 19.5 years old. The

difference seems to be insignificant; however, it is important to note that the average

respondent would be an underclassman at their respective school.

4
What race do you identify with?

Caucasian Asian African Hispanic Other Total

American

Control
Count 30 9 2 2 4 47
Percent 63.8% 19.1% 4.3% 4.3% 8.5% 100%

Treatment
Count 48 10 3 5 0 66
Percent 72.7% 15.2% 4.5% 7.6% 0% 100%

Total
Count 78 19 5 7 4 113
Percent 69% 16.8% 4.4% 6.2% 3.6% 100%

A large majority of the respondents are Caucasian; with Asians, African Americans,

Hispanics and Others making up about 30% of respondents across both surveys. This is not as

surprising as would seem. According to collegedata.com UVA, Duke and NC State are all

disproportionately populated by Caucasians. In 2016, 63.3% of UVA, 53.3% of Duke and 76.1%

of NC State students reported as White/Caucasian. The sample population of Caucasians is not

far from the target population in terms of race. In 2016 the United States Census Bureau found

that 77.1% of the American population was White/Caucasian.

Having a non-representative sample population can skew results. The control survey

had significantly fewer Caucasians answer compared to the treatment survey with a difference

of about 10%. Additionally, the control survey has a larger percent of both Asian and Other

respondents, while the treatment survey has a larger percent of Hispanic respondents. For the

most part both the control and treatment surveys are extremely close to the target population,

as defined by the United States Census Bureau population estimates. The sample population is

extremely close to the target population. Although this experiment did not follow the proper

5
steps for random selection, participants were assigned randomly to either the condition or

treatment surveys which supports the validity of the results.

What is your gender?

Male Female Prefer not to say Total

Control
Count 26 21 0 47
Percent 55.3% 44.7% 0% 100%

Treatment
Count 29 35 2 66
Percent 43.9% 53% 3% 100%

Total
Count 55 56 2 113
Percent 48.7% 49.5% 1.8% 100%

As far as non-representative surveys go, both the control and treatment surveys are

again extremely close to the target population in terms of gender representation. Both surveys

have almost an even 50-50 split of male and female respondents. Gender and political ideology

are the two demographic criteria I was most worried about. If the respondents had been

disproportionately male or female, I believe the results would have been severely skewed. It is

hard to say which way the results would have been skewed but, in a longitudinal study

conducted by Gallup Polls men were found to own notably more guns than women (Agresti).

The same study also found that men are far more accepting of guns, compared to women who

oppose guns and gun ownership. From this, we can assume that if there had been a

disproportionate amount of male respondents we would see more support for guns compared to

if a majority of the respondents had been female.

6
Which word best describes where you are from?

Urban Rural Suburban Total

Control
Count 8 3 36 47
Percent 17% 6.4% 76.6% 100%

Treatment
Count 8 5 53 66
Percent 12.1% 7.6% 80.3% 100%

Total
Count 16 8 89 113
Percent 14.2% 7.1% 78.7% 100%

I was surprised to see that a majority of respondents across both surveys reported that

they are from suburban areas. This in part may be due to a lack of clarification in the survey.

Neither survey presented a metric for respondents to use to judge whether they are from an

urban, rural or suburban area. Furthermore, this question lacks clarification as to whether

respondents should reply with where they currently live or their “hometown”. This was only

brought to my attention after both surveys had been live for several days and a respondent

messaged me back asking for clarification.

Nevertheless, the majority of respondents selection “suburban” as where they are from,

this is not necessarily a bad thing. When considering guns, people in urban vs. rural are very

likely to have opposing opinions. For example, people in rural areas may have a positive outlook

on guns because they can be used for hunting and farming purposes. Whereas in urban

settings, guns are often used for crime rather than hunting animals. I do not believe the

implications of this criteria will affect the overall data in a significant way.

Which word best describes your political beliefs?

7
 Republican Democrat Independent Libertarian Total

Control
Count 11 30 4 2 47
Percent 23.4% 63.8% 8.5% 4.3% 100%

Treatment
Count 20 32 13 1 66
Percent 30.3% 48.5% 19.7% 1.5% 100%

Total
Count 31 62 17 3 113
Percent 27.4% 54.9% 15% 2.7% 100%

As stated above, when the survey first went live I was worried that one or both surveys

would receive a skewed amount of a given political belief. My fear proved to be right in this

case, with the respondents heavily leaning Democratic. However, the objective was to gauge

collegiate millennials opinions and according to a survey done by the Los Angeles Times,

millennials support the Democratic party significantly more than the Republican Party. The study

found that 61% of millennials (in the case ages 18-29) favor the Democratic party and only 33%

favor the Republican party. Responses to both the control and treatment surveys reflect this

belief, 63.8% of control responses and 48.5% of treatment responses indicate that respondents

identify with the Democratic party.

The implications of political ideology, unlike most of the other criteria, may prove to have

a large role in respondents opinions on guns. A poll conducted by pollingreport.com found that

77% of Democrats favor less guns, whereas 79% of Republicans favor more guns. The poll also

found that 83% of Democrats favor stricter gun laws while only 26% of Republicans favor stricter

gun laws. With that being said, the sample population is very close to the target population in

terms of ideological beliefs so the responses should be somewhat accurate.

8
Cross-Tabulations:

Below is a comprehensive analysis of the data broken down by different demographic

factors into cross-tabulations as well as graphs highlighting the difference in opinions and

responses between the control and treatment. After sorting through all 113 responses across

the two surveys, I have discovered that on all seven questions (questions #6-12) measuring

respondents’ opinions on guns and gun control wording appears to have had a significant effect.

Responses to the treatment survey are far more accepting of guns than responses to the control

survey. Analysis of the data suggests that there is in fact a relationship between wording and

opinions regarding guns.

Note: For the following cross-tabulations, the percentages are calculated out of the total number of

respondents from a given survey. For example, 5 out of 47 (10.7%) control respondents were Caucasians

that responded to question 8 saying that the United States would be safer if more people carried guns.

If more people carried firearms, do you think the United States would be safer... by race

Control Treatment

Yes No Unsure Yes No Unsure

Caucasian
Count 5 21 4 11 28 9
Percent 10.7% 44.7% 8.5% 16.8% 42.4% 13.6%

Asian
Count 9 2 6 2
Percent 19.1% 3% 9.1% 3%

African American
Count 2 3
Percent 4.3% 4.5%

Hispanic
Count 1 1 5
Percent 2.1% 2.1% 7.6%

Other

9
 Count 1 2 1
Percent 2.1% 4.3% 2.1%
The above table highlights the differences in responses based on respondent’s race.

This table would be much more informative if more people across all races had taken the

survey. One interesting point is that Caucasians responded with about the same frequency of

answers across both surveys. Another interesting statistic is the difference in Asian respondents

responses. The treatment survey appears to have had a 10-point difference for Asian

responses. This difference supports the hypothesis that wording does have an effect, and that

collegiate millennials generally have more positive views on guns when presented with the word

“firearm” rather than “gun”.

Do you believe that less control over firearms will lead to a reduction in crime… by race and

gender

Control Treatment

Yes No Unsure Yes No Unsure

Caucasian Male
Count 2 16 2 6 14 2
Percent 4.3% 34% 4.3% 9% 21.2% 3%

Caucasian Female
Count 8 2 1 22 3
Percent 17% 4.3% 1.5% 33.3% 4.5%

Asian Male
Count 3 4 1
Percent 6.3% 6% 1.5%

Asian Female
Count 6 2 1
Percent 12.7% 3% 1.5%

Asian No Gender Response

Count 1 1
Percent 1.5% 1.5%

10
 African American Male
Count 1
Percent 1.5%

African American Female

Count 2 1 1
Percent 4.3% 1.5% 1.5%

Hispanic Male
Count 1 1
Percent 2.1% 1.5%

Hispanic Female
Count 1 4
Percent 2.1% 6%

Other Male
Count 1 1
Percent 2.1% 2.1%

Other Female
Count 2
Percent 4.3%
This table breaks down responses to the question of whether or not respondents believe

that less control over guns will lead to a reduction in crime. Among Caucasian males the

treatment survey shows a slight belief that less control over guns will lead to a reduction in

crime; however, the treatment survey appears to have had the opposite effect on Caucasian

females. Caucasian females responded more negatively to the treatment survey. Asian females,

similar to Caucasian males, showed an 11.2-point difference. Asian females seem to support

the hypothesis as well.

In general, do you feel that the laws covering the sale of firearms should be made less strict,

more strict, or kept as they are now… by partisanship and gender

Control Treatment

More Less Kept as More Less Kept as

Strict Strict they are Strict Strict they are

11
 Democrat Male
Count 12 8 1
Percent 25.5% 12.1% 1.5%

Democrat Female
Count 17 1 23
Percent 36.2% 2.1% 34.8%

Republican Male
Count 6 1 3 4 3 6
Percent 12.7% 2.1% 6.4% 6% 4.5% 9%

Republican Female
Count 1 3 3
Percent 2.1% 4.5% 4.5%

Republican No Gender
Response 1
Count 1.5%
Percent

Independent Male
Count 1 1 4 1 2
Percent 2.1% 2.1% 6% 1.5% 3%

Independent Female
Count 2 4 1
Percent 4.2% 6% 1.5%

Independent No Gender
Response 1
Count 1.5%
Percent

Libertarian Male
Count 1 1
Percent 2.1% 2.1%

Libertarian Female
Count 1
Percent 1.5%
The above table shows the difference in responses based on ideology and gender.

republican males show the biggest swing between the control and treatment surveys.

Republican males responded more favorably to making gun laws less strict on the treatment

12
survey than they did on the control survey. However, the treatment survey appears to have an

adverse effect on Independent males & females, as well as Republican females.

Data Analysis

Below is graph containing a breakdown of results from “Question 10” (all questions

below found on the bar graphs are taken from the treatment survey and thus contain the term

“firearm” rather than “gun”). The results show a 13.9 point difference, with 85.1% of the control

group responding with gun laws should be more strict, whereas only 71.2% of the treatment

group responded with gun laws should be more strict. The 13.9 point difference is significant in

showing the effect of wording on public opinion of guns. Nonetheless, one cannot ignore that a

majority of respondents on both surveys responded saying that gun laws should be more strict.

This should not come as a surprise, as the millennial generation has grown up in a war torn,

crime ridden world that has used guns for bad rather than good.

Here is another graph showing the difference in results found on Question 8. Again the

graph highlights the differences in responses across the two surveys, indicating a 10.9 point

13
difference in responses. 74.5% of control respondents said the United States would be less safe

if more people carried guns, compared to only 63.6% of treatment respondents. The results of

this question are interesting to note. There is a 6.9 point difference in “yes” responses to

Question 8, 19.7% of the treatment respondents believe that more guns would make the United

States safer. In total 19 respondents across both surveys selected “yes” to Question 8. This

question suggests that guns might not be the problem and that the people that have them are

the problem. By increasing the number of people who carry guns, the public will be more ready

to protect themselves from intruders and attackers.

The following graph shows a slight difference in responses across the two surveys; with

a 6.7 point difference the results of this question suggest that wording and terminology had an

effect on treatment responses compared to control responses. 91.5% of control respondents

said that they favor universal background checks, whereas only 84.8% of treatment

respondents said they favor universal background checks. I found the responses to the question

to be very interesting. I imagined that collegiate millennials would have had overwhelming

14
support for universal background checks, as they would be more effective in ensuring potentially

problematic people do not get their hands on weapons.

Question 9 proved to have the largest difference in responses across the control and

treatment surveys. There is a 13.9 point difference between the responses from the control

survey and the treatment survey. Only 71.2% of treatment respondents responded to Question

9 saying that they do not believe less control over firearms would lead to a reduction in crime,

85.% of control respondents said that they believe less control will not lead to a reduction in

crime.

15
 Breakdown and analysis of the data indicates that there is difference in opinion on guns

when collegiate millennials are presented with “firearm” rather than “gun”. However, the data

does not conclusively prove that there is a positive connection between “firearm” and opinion on

guns. Although, I cannot conclusively say that wording and framing had the sole effect on the

differences in control and treatment responses, it is fair to say that the data suggests that there

was an influencing factor that appeared in treatment responses more so than in control

responses.

Does terminology affect the way collegiate millennials feel about guns and gun control in

the United States? The data appears to prove that yes, terminology does affect the way

collegiate millennials feel about guns and gun control. Responses from the control survey

proved to be significantly more negative than responses from the treatment survey. Early

analysis and investigation of the data does prove that there is validity and legitimacy to the

research question.

16
 To be able to conclusively answer the research question, this study needs to be

conducted several more times on a much larger scale. The survey experiment can easily be

replicated by any researcher who is interested in the topic. Future experiments need to conduct

more experimentally sound survey experiments by ensuring scientific guidelines and

methodology are followed, such as: random sampling and random assignment to condition.

Sources:

1. Agresti, James D. & Smith, Reid K. “Gun Control Facts.” Just Facts. 22 August 2016.

Web. <​www.justfacts.com/guncontrol.asp​>.

2. "Duke University - College Profile." Collegedata.com. N.p., n.d. Web.

<​http://www.collegedata.com/cs/data/college/college_pg01_tmpl.jhtml?schoolId=1026​>.

3. McClatchy, and Marist. "Guns." Pollingreport.com. N.p., n.d. Web.

<​http://www.pollingreport.com/guns.htm​>.

4. "North Carolina State University - College Profile." Collegedata.com. N.p., n.d. Web.

<​http://www.collegedata.com/cs/data/college/college_pg01_tmpl.jhtml?schoolId=1598​>.

5. United States Census Bureau. "United States Population Estimates." US Census

Bureau. N.p., n.d. Web. <​https://www.census.gov/quickfacts/​>.

6. "University of Virginia - College Profile." Collegedata.com. N.p., n.d. Web.

<​http://www.collegedata.com/cs/data/college/college_pg01_tmpl.jhtml?schoolId=1571​>.

17
Honor Pledge: I have not cheated nor have I helped another student cheat on this assignment.

18