ANNUAL REPORT (AR)

Part I
Reporting On Activities For Academic Year
June 1, 2016- May 31, 2017

Name Scot McNary Rank Associate Professor

Department of Educational Technology and Literacy

Area of Specialization Educational Research

Appointed to TU faculty: at rank Assistant in year 2007 .

Promotion History:

To rank Associate Professor in year 2013 ,

To rank in year , and
To rank in year .

I. Formal Degrees

A. Highest degree earned, with date and name of granting institution.

Ph.D. Clinical/Community Psychology, University of Maryland, College Park, 2000.

B. If candidate for an advanced degree, indicate work completed since June 1, 2010
and present status. Corroborative material and/or transcript must be attached.

II. Teaching (percentage of workload: 60 %)

A. Evaluations and Grade Distributions

1. Teaching Evaluations for all courses Fall 2016-Spring 2017.
Summarizing across means, my average overall course evaluation was 4.45 (SD = 0.28), and
ranged from 4.18-4.86.
%
Course Mdn M SD N resp.
Summer 2016
ISTC 695 1
Fall 2016
EDUC 605 4.57 4.39 0.40 10 58%
EDUC 790 4.70 4.62 0.40 5 71%
ISTC 685 (combined with EDUC 761) 4.08 4.07 0.22 6 50%
ISTC 694 1
ISTC 695 1
Spring 2017
EDUC 605 4.20 4.18 0.42 16 31%
ISTC 685 (Howard County) 4.60 4.65 0.25 9 56%
ISTC 685 (combined with EDUC 761) 5.00 4.86 0.24 10 80%
ISTC 694 5
TSEM 102 4.44 4.40 0.20 16 56%
 Note: ISTC 694 Directed Reading; EDUC 605 Research and Information Technology; EDUC 761
Research in Education; EDUC 715 Statistical Principles of Research Design and Analysis; ISTC 685
Research in Instructional Technology. ISTC 694/695 students do not complete student evaluations.

2. Insert below your class GPA and grade distribution.

These data are provided to your dean’s office by the Office of Institutional Research (fall
data are sent in February and spring data are sent in mid-June). Your dean’s office will distribute
these data to departments. You may fill out this table by indicating the number of students in
each grade category, or you may electronically insert the information by cutting and pasting the
entire section from the report.
Course A A- B+ B B- C+ C D+ D F FX Total W O Mea
n
Summer 2016
ISTC 695 1 1 4.00
All Summer classes 1 4.00
Fall 2016
EDUC 605 7 7 3 1 1 19 3.62
EDUC 790 2 3 1 6 1 3.72
ISTC 685/EDUC
761 3 3 2 3 1 12 3.39
ISTC 694 1 1 4.00
ISTC 695 1 1 4.00
All Fall classes 39 4.24
Spring 2017
EDUC 605 4 9 2 1 16 3.61
ISTC 685 (Howard
County) 5 3 1 9 3.15
ISTC 685/EDUC
761 3 2 2 3 11 1 3.84
ISTC 694 4 1 5 3.87
TSEM 102 1 3 3 2 3 3 1 19 3 2.81
All Spring classes 60 3.35
All classes 99 3.70
Note: A=4.0,A-=3.67,B+=3.33,B=3.0,C=2.0,F=0.0. Semester means are weighted by class size.

3. Attach syllabi for all courses listed (must contain all elements required for syllabi in
Policies and Procedures for the Classroom: Course Syllabus).
See attached.

B. Non-classroom assignments which are part of your regular on-load teaching

assignment (i.e., coaching, directorships, supervision of student teachers).
Dissertation Committee Member
I was a member of 15 students’ committees this academic year, and am continuing as
Program Chair for two. Chip Boling, Alex Greenwood, Jessica Stansbury, and Zhuo Wang
completed their dissertations in 2016-2017.
 2012-2013 2013-2014 2014-2015 2015-2016 2016-2017
Chip Boling Chip Boling Chip Boling Chip Boling Chip Boling
Tamara Burton Tamara Burton Tamara Burton Tamara Burton
Lisa Carey
LaTonya Dyer LaTonya Dyer LaTonya Dyer LaTonya Dyer LaTonya Dyer
Mila Fuller Mila Fullere
Kathy Goulde
Alex Greenwood Alex Greenwood Alex Greenwood Alex Greenwood
Kathryn Lee Kathryn Lee Kathryn Lee Kathryn Lee
Chris Magalisp Chris Magalisp
Amy Martin Amy Martin Amy Martin Amy Martin Amy Martin
Patricia McAteere
Lynne Murphy Lynne Murphy Lynne Murphy Lynne Murphye
Andrea Parrish Andrea Parrish Andrea Parrish
Stefani Pautz
Arkhadi Pustaka Arkhadi Pustaka Arkhadi Pustaka Arkhadi Pustaka
Matt Rietschelp Matt Rietschelp Matt Rietschelp Matt Rietschelp
Valerie Riggs Valerie Riggs
David Robinsone
Cora Roush Cora Roush Cora Roushp
Ben Smith Ben Smith
Jessica Stansburyp Jessica Stansburyp Jessica Stansburyp Jessica Stansburyp
Shannon Tucker
Zhuo Wang Zhuo Wang Zhuo Wang Zhuo Wang
Dean Whitfield Dean Whitfield
Emily Ziegler Emily Ziegler Emily Ziegler Emily Ziegler Emily Ziegler

New instructional procedures which you have introduced (special projects, new
courses and/or materials)
Group Work
I use Blackboard in all my courses as a repository for assignments, course notes,
discussion boards, and class demonstration materials. Students’ grades are maintained in the
Blackboard gradebook. Students preferred to use Google Docs or email to share work.
I continued to offer the option for EDUC 605 and ISTC 685/EDUC 761 students to work
as a group. Not all students elected to work with a partner, but those that did tended to produce
longer and more thoughtful assignments. Students organized themselves into groups. I received
feedback from a couple of students suggesting this was helpful:
 “The opportunity to choose to work independently or in pairs.” “group work available”
EDUC 605
I taught both EDUC 605 classes as blended courses this year, 60% face to face and 40%
online. In the fall I experimented with providing audio feedback to groups on their literature
review draft. I received no feedback from students on it either in person or in evaluations, so I’m
unclear of the utility of it.
TSEM
In the Spring 2017 TSEM course I continued to use two strategies from prior
experiences: 1) including the They Say/I Say exercises from the Graff & Birkenstein (2014) text
and asked students to write in class virtually every class meeting. This resulted in many short
essays, focused on both content, but also specific tasks like formulating thesis statements or
identifying citation formats for non-print media. Students gave feedback about the writing in
their course evaluations:
 “I also found it useful that we went step by step in the essay writing process”

I also 2) used journal articles and general media sources to present opposing views of
various controversial educational practices like paying students for grades, or eliminating grades
from educational practice altogether. The articles seemed to generate interest, and end of
semester student reflective essays suggested at least some students were strongly affected by the
ideas they encountered.
 “One concept that I think affected me the most was from the documentary “Waiting for
Superman.” In this documentary, the idea that education is failing many students was
looked into. It personally affected me because it really got me thinking is the current
education fair and positively affecting me so I receive the best education that I can?”
 “I learned various educational methods of teaching such as paying students to do
well,video games, environmental learning, introduction of technology and flipped
classroom. Thiscourse introduced me to in depth analysis of all those methods what is it?
Why? And made uscritically question the effectiveness of those teaching children in the
21st world.”
In part due to attending the gamification conference in Fall 2016 and also having
supervised Jessica Stansbury’s dissertation on gamification, I was inspired to create four
gamified exercises on APA formatting for the TSEM class this spring. Each assignment focused
on a different concept (e.g., title page/abstract formatting or headings and seriation), and entailed
the students receiving a copy of an error-filled version of a document. To complete the
assignment, students had to pick a partner, review the Owl Purdue website section on the focus
for the assignment (e.g., title/abstract formatting), and then as a team discover and repair as
many errors they could find. Their score for the assignment was the number of corrections
detected and correctly repaired out of 10. I typically made the total number of errors 15 or so, so
there was some leeway in finding the errors. Students who found more than ten earned grades
that exceeded 100%. Students typically took 30-45 minutes to complete these on their own, so
they were engaged in the process, although I expected they would find them more fun than they
appeared to. Although several students said they found learning about APA formatting useful to
them, only one student referred to the gamified APA assignments in particular:
 “ It also taught me very useful information on APA formatting which we learned through
various APA investigation assignments”
I will want to reflect and consult with colleagues about my experience with the gamified
APA assignments, and at least try them in one other class before deciding to abandon or continue
to using them
Graff, G. & Birkenstein, C. (2014). They Say/I Say: The Moves That Matter in Academic
Writing. 3rd ed. W.W. Norton & Co. ISBN-13: 9780393935844.
EDUC 790
I revised this class in the fall of 2016 to accommodate the new research sequence in the
doctoral program. This course is no longer an elective, and its prerequisite has a minimal
introduction to statistical analysis, unlike EDUC 715 which was more extensive. This means that
students enter EDUC 790 with less preparation than in past semesters, but yet all students must
now take this course. This meant that the course had to be revised to accommodate a
heterogeneous group of students, all of whom would be motivated to do well in the course.
Based on student feedback in previous semesters, specifically, that working on analyses
collectively could be more useful (and anxiety-reducing), I decided to try and teach EDUC 790
as a flipped classroom. Course notes, slide presentations, and readings would be assigned for
students to complete prior to coming to class, and class sessions would be devoted to labs in
which we analyzed a common dataset. Students could pick variables to investigate, and I would
circulate the classroom helping as needed. In principle, this worked well, however there were
issues in implementation. First, there were differences in software/hardware and access. It is a
perennial problem for students to get access to SPSS through TU. Virtual Workspace up through
Fall 2016 has been continually problematic. As a consequence, most students elected to purchase
SPSS on their own. This is problematic because they pay technology fees and spend a lot of
money on textbooks. It’s an unfair burden for students to undertake. However, it is expedient.
Students had varying degrees of success installing and using SPSS, which was not unexpected.
Second, the textbook, which was new to me and the students this semester, was almost
universally disliked. I was surprised by this because the textbook is not as difficult as the one I
used before, uses a lot of examples, is tied intimately to SPSS, and abounds with humor. Here is
what some students had to say about the text:
 “While the text is fairly accessible, I'm not sure it is appropriate for the students in this
class in a flipped classroom model. Many students (myself included) sought out
other texts and resources to supplement Fields.”
 “I think a different textbook would be helpful, I don't think anyone loved the book as
much as Dr. McNary. Something shorter and more to the point would be better.”
I’m not sure whether this reflects general distaste with any statistics textbook, or a specific
concern about this one. One the one hand, I expect the textbook to be useful, but on the hand, I
quite support looking up multiple resources to aid in understanding. I’m not ready to give up on
it just yet.
A third problem in implementation was student accountability. Not all students came
prepared for class. This is not unusual given that the preparation is time-consuming, and it is
tempting to catch up with others available to ask questions about. But this also meant spending at
least some time in class each week using lecture and discussion to make sure all students had a
minimum level of understanding to proceed. I received feedback about how to improve the
flipped approach that I believe I should implement for the next trial:
 “…to fully utilize this model I would advocate for recorded videos explaining concepts in
basic terms with example statistical analysis.”
The slide presentations that were available were from prior EDUC 790 classes, with minor
updates. Updating them and adding audio would very likely improve the self-directed learning
experience at home.
Despite the problems, I covered more content in this class than I have in any other methods class
since I began teaching EDUC 790. I was able to teach from an assumption-based framework
(what assumptions are required to conduct specific tests), to show students how to test various
assumptions, and then if assumption tests fail, show students what remedies are available. This
meant that for the first time I was able to teach non-parametric statistical analysis, which has
been a long term goal for me. I was also able to cover repeated measures ANOVA, another topic
that has always eluded me. We didn’t spend nearly enough time on it, but that we at least ran
some analyses and interpreted them was quite gratifying.
Finally, inspired by the gaming conference and working with Jessica Stansbury on her
dissertation, I created gamified homework assignments. For example, I created a dataset that was
problematic in some way, then asked students to imagine they were data analysts whose clients
had asked them to perform some (unadvisable) statistical analysis. The set-up:
“You have two clients in this research group who have questions about comparing the groups, but
differ in how they prefer to receive their interpretations. Do your best to communicate with this pair of
professionals in the school of interpretation they prefer best. Make a mental note to give them credit for
attempting to triangulate their findings from divergent points of view in your presentation.”

A sample problem:
“Diogenes (remember him?) runs in to your office out of breath and says he has a control group for
the CAI vs. BT study. This is a group of kids who took the same vocabulary test at the beginning of the
summer, and again at the end of the summer, before the experiment ever took place. He claims this is a good
control group, because no formal instruction occurred over the summer and so these students’ data provide an
estimate of how much vocabulary growth could be expected in the absence of instruction. Save your
arguments about the adequacy of the control for a moment and consider what analytic approach you should
take to compare the post-test scores between three groups. Which is it?”

Some students found them entertaining, others did not:

 “The assignments could be improved. I struggled to understand some of the homework
questions.”
 [What could be improved?]“The wording of the homework questions.”
Simulations
Several simulation websites I like to use rely heavily on Java, and since it has become
vulnerable to security risks, it has been untenable to continue to use simulation in courses. This
suggests building simulations using local software readily available on Mac or Windows
software would be important to develop. I have created a brief simulation in Microsoft Excel to
demonstrate the t-test using permutation methods. I need further testing of the software with
students before it will be fully useful. I have also begun to use non-Java based simulation
applications built by other educators to supplement content on statistical analysis. Here are two I
used last year and continued to use this year:
http://rpsychologist.com/d3/correlation/
The correlation simulator website allows students to observe the effect of changing the
numeric value of a correlation coefficient and observe the effects on the shape of the scatterplot
and slope of the regression line. I ask students to create a strong positive correlation, a strong
negative one, and then some arbitrary ones. We then try to imagine educational research
questions that might fit the pattern of points generated by the simulation. I also ask them to
increase the sample size and comment on how the relationship changes visually as the sample
size increases.
http://guessthecorrelation.com/
 A colleague discovered this delightful game (“Guess the Correlation”) that allows
participants to guess the numeric value of a correlation based on a randomly generated
scatterplot. The graphics are primitive (intentionally) and the game is cleverly simple. Just like
last year I had to ask them to stop after ten minutes of play so we could move on to the next
activity (!)
http://www.rossmanchance.com/applets/
This website contains multiple simulation applications for demonstrating randomization
procedures for statistical analysis. This is a different approach than frequentist methods, but I
believe students get a feeling for the intuition underlying frequentist statistics by observing the
resampling graphics. I used this website to replace all my lecture/discussion on inferential
statistics. After our general description, I asked each student to describe their own planned
statistical analysis and we ran it with permutation simulations with the Rossman & Chance
website so we could “run” an analysis matching their proposed data analysis plan. In the future, I
would like students to be able to customize a simulation approach like this on their own to run
simulated data through an analysis, but such customization would require considerable
programming.
Advising (including number of students, whether majors, undeclared, or
interdisciplinary students)
These are doctoral students I have consulted with since 2007, but not as a committee
member. I provided research design and statistical analysis guidance in my role as
methodological consultant for the doctoral program. There have been nine students in the last
four years, three in the past year.
2012-2013 2013-2014 2014-2015 2015-2016 2016-2017
Atiya Afsanio Atiya Afsanio Atiya Afsanio
Liz Berquiste
Barbara Blummer
Andy Cavanaughe
Hyun-Kyung Kim
Carrie McFadden Carrie McFadden Carrie McFadden
e
Debbie Nolan
Sagar Rainao
Nancy Shipleyo
Note: oComputer Science

Doctoral student dissertation committee membership, described above as non-class

mentorship, also falls into the category of advising. These are detailed in the tables above. All
my consulting activities are detailed in an accompanying spreadsheet.

Correlation Statement. If your productivity did not match your projections for academic period
2016-2017, please explain. N/A

III. Scholarship (percentage of workload: 30%)

A spreadsheet in the scholarship section provides detail on type of project conducted and
amount of time spent from Summer 2016 to Spring 2017 on each project.

Correlation Statement. If your productivity did not match your projections for academic year 2016-
2017, please explain. N/A
Manuscripts under review:
Lohnes Watulak S., Wang Z., & McNary S.W. (submitted). An exploration of professional

knowledge needed for reading and responding to student writing.

Manuscripts accepted:
McQuitty, V., Ballock, E. &McNary, SW. (2017). An exploration of professional knowledge

needed for reading and responding to student writing. Journal of Teacher Education, 1-

13. DOI: 10.1177/0022487117702576

Conference Presentations
McNary SW. (2016, July). Research methods: A brief survey of methods and ideas. Invited

address for the American Psychological Association Minority Fellowship Psychology

Summer Institute, Washington, DC.

Grants submitted
N/A
Consultation
The following is a brief description of the amount of consulting activity I have conducted
since 2007 by stakeholder group. See the Service section for a more extensive description of time
spent and activities completed. Note that the number of external researchers and agencies I have
consulted with has decreased to zero whereas the amount of consulting I do with doctoral
students has increased. The strong increase in doctoral student consultation is related to
dissertation supervision for one student and two independent studies I conducted with doctoral
students on dissertation methodology. The decrease in faculty consultation hours that has
occurred over the past three years, is due to the loss of a course release I had previously received
for working with faculty on their research.

Year/Source Number of Consultees Hours

COE Faculty Researchers
2016-2017 5 46
2015-2016 9 45
2014-2015 11 88
2013-2014 13 140
2012-2013 11 122
2011-2012 16 87
2010-2011 17 90
2009-2010 9 83
2008-2009 8 62
2007-2008 5 33
 Doctoral Students
2016-2017 15 141
2015-2016 15 181
2014-2015 15 148
2013-2014 15 54
2012-2013 11 67
2011-2012 11 87
2010-2011 8 45
2009-2010 13 49
2008-2009 10 58
2007-2008 8 52

Masters in Education Students

2016-2017 2 4
2015-2016 0 0
2014-2015 0 0
2013-2014 1 5
2012-2013 0 0
2011-2012 0 0
2010-2011 6 39
2009-2010 1 21
2008-2009 7 33
2007-2008 3 10

Outside the College Researchers

2016-2017 0 0
2015-2016 2 7
2014-2015 2 6
2013-2014 6 45
2012-2013 4 14
2011-2012 1 8
2010-2011 3 12
2009-2010 4 57
2008-2009 8 35
2007-2008 4 41

Conferences Attended
2016-2017
New Designs for Learning: Games and Gamification September 2016 College Park MD
APA Minority Fellowship Research Day (invited speaker) July 2016 Washington DC

Correlation Statement. If your productivity did not match your projections for academic period
2015-2016, please explain. N/A

IV.Service (percentage of workload: 10%)

Profession
Reviewing
  Editorial Board Member, Child Abuse and Neglect. (reviewed 9 journal manuscripts)
 Ad hoc reviewer
 Special Issue of Computers in the Schools (2 manuscripts)
 AERA Instructional Technology Special Interest Group (2 presentations)
 APA Division 5 conference presentations (4 symposia, 9 poster
presentations)
Non-college/non-University consultation
None in 2016-2017.
University
Committees
Institutional Review Board (Summer 2014-present). I reviewed 30 applications in the
2015-2016 academic year for the committee.
Faculty Consultation
None in 2016-2017.
College
Committees
I served on the COE Scholarships committee this year. The committee received in excess
of 130 applications and awarded over $20000 in scholarships.
I served on the Merit Task Force committee. The committee evaluated current COE merit
criteria and sought alternate merit criteria from other institutions and after careful deliberation,
produced a white paper summarizing findings and making recommendations to the Dean for
revising merit evaluation procedures. The task force met once this year to review feedback from
faculty on the recommendations.
Faculty Consultation
I consulted with 5 different COE faculty members for a total of 46 hours. In addition, I
co-chaired the Faculty Research professional development group along with Rebecca Shargel
Ed.D., which was approved by the College Council for 2016-2017. We met three times, twice in
the fall 2016, and once in the spring 2017. It is worth noting through spring 2013, I received a
course release for faculty consultation, but that course release was withdrawn in 2014.
Department
Student Advising
I am currently a dissertation committee member for 10 Ed.D. students and am program
chair for one of those students, and initial advisor for another student. I further consulted with
two doctoral students from Computer Science for whom I am not a committee member.
Committees
Admissions committee (2008-2017), Department Promotion and Tenure Committee,
Doctoral Program committee (2008-2017), Merit Committee (2011-2017, chair 2014-2017).
 Correlation Statement. If your productivity did not match your projections for academic
period 2015-2016 please explain. N/A
ANNUAL REVIEW (AR)
Part II
Agreement On Faculty Workload Expectations For Academic Year
June 1, 2017 - May 31, 2018

I. Teaching (percentage of workload: 70 %)

A. List all of the regular classroom teaching assignments planned for the 2017-2018 academic year.

Course Title/Number Credit Hours

Fall 2017
ISTC 685 (off campus) 3
ISTC 685 (off campus) 3
ISTC 685/ EDUC 761 3
ISTC 694 1
Spring 2018
Sabbatical

B. Non-classroom assignments which will be part of your regular on-load teaching assignment (i.e.,
coaching, directorships, supervision of student teachers) for the 2017-2018 academic year.
a. Dissertation committee member for the following ten students: Tamara Burton, Lisa
Carey, La Tonya Dyer, Kathryn Lee, Chris Magalis, Amy Martin, Andrea Parrish,
Arkhadi Pustaka, Matt Rietschel, Shannon Tucker, and Emily Ziegler.
b. I no longer have a course release for student and faculty research support and
consultation in the College of Education but retain one course release for my own
research.

C. New instructional procedures which you plan to introduce this year (special projects, new courses
and/or materials). Also include interdisciplinary, diversity, international and new technology
projects, if appropriate.

I will be refining online modules and assignments for the Spring 2017 EDUC 670 course.

D. Advising (including number of students, whether majors, undeclared, or interdisciplinary students)

I will be providing consultation for M.Ed. and Ed.D. students (other than thesis committee work). This includes
doctoral students (2 estimated) with research design/statistical analysis questions.

II. Scholarship (percentage of workload: 20 %)

I expect to produce one conference presentation and one manuscript publication in the following year. Data analyses
for the Facebook Mattering with Dr. Lohnes-Watulak project are ongoing. My sabbatical research will focus on an analysis of
reading clinic data, to begin in January of 2018.
III. Service (percentage of workload: 10%)
[For any of these activities which are part of your workload, please indicate.]

Community: Consultation with non-profit and non-for-profit agencies in research design, data analysis, and
psychometrics

Profession: Reviewing, external researcher consultation

University (all levels): University Institutional Review Board, College Scholarships Committee, Department
Admissions committee, Department Doctoral Program committee, Department Merit Committee (chair),
student and faculty research consultation

SIGNATURES:

Faculty Member Date

Chairperson of Department Date

Dean of College Date