Running head: PBL AND PERCEIVED STUDENT PREPAREDNESS

Project Based Learning and Perceived Student Preparedness in First Year Engineering Classes

Albert Bangma, Baljeet Gill, Bryn Hammett, Michael Hengeveld, and Kenny Jamieson

Research Proposal

Janet McCracken

ETEC 500 - Research Methodology in Education

Master of Educational Technology Program

University of British Columbia

PBL AND PERCEIVED STUDENT PREPAREDNESS
2

Abstract 3

Objective 4

Theoretical framework 5

Methods 7

Description of the participants 7

Conducting the study 8

Description of the survey instrument 8

Step by step description of all aspects of our data collection 9

Description of our procedures for data analysis 9

Assumptions made about our methodology 9

Description of limitations of our study 10

Description of data sources 10

Results and Conclusions 10

Educational significance 11

References 12

Appendix A 14
PBL AND PERCEIVED STUDENT PREPAREDNESS
3

Abstract

This study involved 700 first year engineering students from three different post-secondary

institutions in British Columbia. In their K-12 education, some students were exposed to various

levels of project-based and inquiry based learning strategies, while others learned through

traditional methods. The questionnaire contained 20 statements concerning students high school

learning experiences and their perceptions of their preparedness for the rigors of a

post-secondary learning environment . A correlational analysis of the results of the study reveals

the effect of project-based and inquiry-based teaching strategies on the perceived preparedness of

students during a critical period of transition in the STEM pipeline. An analysis of task-level

aspects of project-based and inquiry-based learning systems, could provide best-practice policy

insights for curriculum designers and other key stakeholders during a time of changing

curriculum in the province.

Keywords: Project-based learning strategies; Inquiry-based learning strategies; STEM;

engineering; curriculum design.
PBL AND PERCEIVED STUDENT PREPAREDNESS
4

Objective

In 2016, the British Columbia government implemented a complete overhaul of the K-12

curriculum thus asking educators to focus on personalized learning and 21st century skills such

as Big Ideas and Curricular Competencies as opposed to just content retention; a shift that

some believe is conducive to Project Based Learning (PBL) and Inquiry Learning (IL).

Although it has been established that PBL and IL pedagogies increase long term retention of

concepts (Schwartz & Martin, 2004) and engagement through increased student agency and

real-world application (Ahlfeldt, Mehta & Sellnow, 2005; DeWitte & Rogge, 2014), we found a

lack of research in the areas around students perception of their preparedness for post-secondary

education.

The purpose of this study is to determine if there is a correlation between a students

self-reported preparedness in first year STEM classes and their level of exposure to PBL and IL

in secondary school.

As policy advisors in British Columbia continue to monitor the implementation of the

new BC curriculum, we aim to inform their decisions as they advise other stakeholders in the

education system both within BC and abroad; although they are not our primary audience, our

research will inform all stakeholders impacted by the BC Education system.

Through an extensive literature review and with the motivation of guiding our own

teaching practice, we posed the following research question and developed the following

hypothesis:

Does the extent of exposure to project based learning (PBL) and Inquiry Learning (IL) have a

correlation with students self-reported levels of preparedness in first-year post-secondary

PBL AND PERCEIVED STUDENT PREPAREDNESS
5

engineering programs?

H1: There is a positive correlation between exposure to project based learning (PBL) and inquiry

learning (IL) in high school with student levels of preparedness in first-year post-secondary

engineering programs.

Theoretical framework

In general, PBL learning systems have been demonstrated to outperform traditional

learning ones (Scott, 2012). While in some instances, such as memorization or factual recall,

traditional approaches have shown advantages, PBL has higher levels of retention when

measuring student learning of big ideas into their own lifeworlds(Strobel & van Barneveld,

2009). Studies have shown that an International Baccalaureate (IB) approach to learning has

lasting impacts on students, beyond the immediate course (Schwartz & Martin, 2004). What has

not yet been shown is the longitudinal effects of PBL and IL approaches to learning used in

secondary school, especially as students transition into post-secondary.

Although PBL and IL are well established in theory, their application in the classroom is

hardly uniform (Ahlfedt, Mehta & Sellnow, 2005; Erdogan, Navruz, Younes, Capraro, 2016).

Given that these methodologies have received uneven treatment by instructors, it is not clear

what makes PBL and IL effective. There is some evidence to suggest, for example, that

collaboration may not be an essential element of PBL, but that engagement with the problem is

an essential element (Pease & Kuhn, 2010). It is particularly important at this point to position

our investigation of PBL or IL with clearly defined task-level elements and the degree to which

they improve self-reported for post-secondary education.

In PBL pedagogy projects are central to the curriculum, with a focus on the processes of
PBL AND PERCEIVED STUDENT PREPAREDNESS
6

investigation, collaboration, designing, and prototyping (Ahfeldt, et. al., 2005; Erdogan, et. al.,

2016; Hugerat, 2016); projects should contain relatively high-levels of student self-direction and

reflection (Ahfeldt, et. al., 2005); projects should be realistic and not 'school-like' (Hugerat,

2016); and, there should be multiple ways to solve the problem (Erdogan, et. al., 2016). To

measure the degree to which a secondary school student has been exposed to PBL at a task-level,

or IL more generally, Stearns, Morgan, Capraro, and Capraro (2012) have created a teacher

observation instrument for PBL classroom instruction, and has been adapted by other

researchers, like Erdogan, et. al. (2016).

While there is some consensus around what constitutes the task-level elements of PBL or

IL, making measurements of these elements and how they affect self-perceived preparedness is

less clearly defined. In response to high university drop out rates, Jansen and van der Meer

(2012) measured self-perceived preparedness for post-secondary education at a task-specific

level by developing an instrument to investigate the effects of time management, written

communication, group work, information processing, ICT, and verbal communication on

self-efficacy in students graduating from the K-12 system. Although they were not targeting

PBL programs, the tasks in their study are highly related to elements of PBL and IL. Their

instrument and analysis techniques will be adapted to explore the specific effects of PBL and IL

on student preparedness at a task-specific level.

Adapting a teacher observation instrument to construct our survey questions is only one

method for maintaining objectivity. We will also have to be aware, and try to eliminate where

possible, a wide variety of confounding variables. Chin, et. al. (2011) suggests that prior

knowledge from parents and life experiences, rather than secondary school, may be a motivating
PBL AND PERCEIVED STUDENT PREPAREDNESS
7

factor in students enrolling in engineering programs. Heslop (2013) provides a further, and

much longer, list of variables for student success in post-secondary institutions, many of which

are outside of secondary school preparation, such as gender, ethnicity, language and geography.

The complex lives of students is a further confounding factor; personal likes and dislikes of

secondary teachers or school life can all contribute to a student's feeling of preparedness for

post-secondary education. The collection of biographical and historical data will be an important

control measure.

Engineering and other STEM related post-secondary programs have a high dropout rate

(Jansen & van der Meer, 2012) despite the increasingly prevalent adoption of PBL and IL

programming in K-12 and its ability to increase engagement, retention, and preparedness of

students (Pease & Kuhn, 2010) . A survey that quantifies the students' perspectives on

preparedness, through a task level investigation of PBL and IL, may reveal what elements are

best able to prepare students for the transition to a post-secondary education setting.

Methods

Description of the participants

Selected students enrolled in their first year engineering programs at the University of

British Columbia (UBC), the British Columbia Institute of Technology (BCIT) and the

University of Northern British Columbia (UNBC) will participate in this study by completing an

online survey. Students will be sampled using equal stratified sampling to ensure equal

representation from each educational facility. In total, several hundred students are expected to

be surveyed.
PBL AND PERCEIVED STUDENT PREPAREDNESS
8

Conducting the study

This study will be conducted using a web-based survey tool. Initially, consent to share

the cover letter and survey directions with first year engineering instructors will be asked from

the Deans of Engineering from each institution via e-mail. Upon approval, instructors will be

informed on the nature and rationale of the study and the survey cover letter and directions will

be sent directly to the academic e-mail account of each selected student by the instructor. The

cover letter will detail the approaches taken to ensure the confidentiality of the students

throughout the survey process. Participants will be informed that their participation in this study

is voluntary and by completing and submitting the survey, they are providing their consent to

participate. If they choose to not to participate, they will be instructed to simply disregard the

e-mail message. Students will be able to access the survey through a hyperlink and will be asked

to answer each question to the best of their abilities and then submit their responses

electronically. The survey will be available for 3 weeks from the date that it is first sent.

Description of the survey instrument

The web-based survey that has been developed for this study will instruct participants to

respond to 20 statements concerning their level of exposure to PBL and IL instructional methods

during secondary school and perceived preparedness for post-secondary. They are instructed to

respond to each statement on a five-point Likert scale, ranging from Strongly Disagree to

Strongly Agree. Additionally, six demographic questions are included in the survey in order to

control for the socioeconomic and cultural variability of the respondents. Measures will be taken

to ensure the confidentially of each respondent.

Step by step description of all aspects of our data collection

PBL AND PERCEIVED STUDENT PREPAREDNESS
9

In order to collect the data that is needed to conduct the research, contact will need to be

made with the post-secondary schools selected for the study at the start of the September

semester. With the assistance of the office of the registrar, those students will be reached by the

e-mail address they have provided through their respective student account. Surveys will be sent

in the latter third of the course to ensure students have had a chance to reflect on their

post-secondary classroom content yet still clearly recall their high school experience.

Description of our procedures for data analysis

The data will be analyzed with Statistical Package for the Social Sciences (SPSS) using

descriptive and inferential techniques, including advanced and multivariate analyses. Initial

descriptive statistical analyses will provide insight into frequency distributions and measures of

central tendency, dispersion, and relationship. Further inferential statistical analysis will test the

strength of relationship between exposure to PBL/IL instruction in high school and the degree to

which participants feel prepared entering the study of engineering at a postsecondary institution.

Variance will be analyzed using ANOVA by comparing the data from those who were exposed

to PBL/IL instruction in high school with data from those who were not. All ANOVA results

will be evaluated at an alpha value of 0.05.

Assumptions made about our methodology

For this study, assumptions are made that the students participating in the study will be

able to clearly recall the experiences they had as secondary school students. In addition, it is

assumed that the students will be able to successfully recognize that the approaches their

secondary school teachers used match the descriptions we have used in our survey. Further, the
PBL AND PERCEIVED STUDENT PREPAREDNESS
10

survey instrument that is being used will help the students accurately connect their secondary

experiences to the level of preparedness they felt upon entering a first-year engineering program.

Description of limitations of our study

The study intends to use a web-based survey that will rely on students accurately

self-reporting the information required for effective analysis. No attempts will be made to

confirm the reporting of the usage of various PBL forms of learning in secondary schools.

Additionally, there will be no follow up with the students who have submitted the survey, to

confirm the accuracy of their responses with respect to their actual levels of preparedness or the

clarity of their recollections of their secondary school experience. The study is also dependent

on a certain level of voluntary participation from the students.

Description of data sources

The data source used for this study will be the participant survey. Responses will be

collected, collated and analyzed. Inferences will be made about the correlation between the level

of exposure to PBL instruction in high school and the self reported level of preparedness for first

year engineering courses.

Results and Conclusions

The survey will attempt to see if any correlation can be found between the questions

related to PBL and IL and the questions related to preparedness, while also checking to see if

there are socio-demographical extraneous influences. Our expectation is that students exposed to

a PBL/IL model will self report a higher level of preparedness for university. If, after statistical

analysis, the exposure to PBL/IL and level of preparedness appear correlational, then the

inference that PBL/IL increases student preparedness will be made. Further analysis may reveal
PBL AND PERCEIVED STUDENT PREPAREDNESS
11

some of the task-specific descriptors of PBL have higher instances of correlation with

self-reported preparedness and may warrant further analysis in future studies.

Educational significance

Education is a high-stakes enterprise with each grade building on the skills and learning

of the previous to scaffold students towards ever higher, more critical levels of understanding

and engagement. Within the K-12 system, the curriculum, social, and educational requirements

of the successive years are fairly well known and understood; students progress through a system

that changes mostly incrementally and carefully scaffolds their understanding from year to year.

The transition from secondary to post-secondary education, however, is filled with uncertainty

and greater change. The various stakeholders involved all want to ensure students are prepared

for the transition, but, there is an opacity around the teaching and learning at the post-secondary

environment.

All the stakeholders want to be assured that systems of teaching and learning, like PBL

and IL, which work so well transitioning students from grade to grade, will continue to work

when students pass beyond secondary school. This study would be one more reassurance that

PBL and IL approaches to learning can and do have a lasting effect that does prepare students for

the type of learning they will be experiencing in post-secondary education.

PBL AND PERCEIVED STUDENT PREPAREDNESS
12

References

Ahlfeldt, S., Mehta, S., & Sellnow, T. (2005). Measurement and analysis of student engagement

in university classes where varying levels of PBL methods of instruction are in use. Higher

Education Research & Development, 24(1), 5-20. doi:10.1080/0729436052000318541

Chin, J., Zeid, A., Duggan, C., & Kamarthi, S. (2011). A unique methodology for implementing

high school capstone experiences through teacher professional development. American

Journal of Engineering Education, 2(2), 17.

Erdogan, N., Navruz, B., Younes, R., & Capraro, R. (2016). Viewing how STEM project-based

learning influences students science achievement through the implementation lens: A latent

growth modeling. EURASIA Journal of Mathematics, Science & Technology Education,

12(8). doi:10.12973/eurasia.2016.1294a

Heslop, J., Project, B. C. S. T., Canadian Electronic Library (Firm), & desLibris - Documents.

(2013). Student transitions into post-secondary education sectors B.C. public, B.C. private

and non-B.C. institutions: Ten research questions. British Columbia Student Transitions

Project.

Hugerat, M. (2016). How teaching science using project-based learning strategies affects the

classroom learning environment. Learning Environments Research, 19(3), 383-395.

doi:10.1007/s10984-016-9212-y

Jansen, E. P. W. A., & van der Meer, J. (2012). Ready for university? A cross national study on

students' perceived preparedness for university. Australian Educational Researcher, 39(1),

1-16. doi:10.1007/s13384-011-0044-6
PBL AND PERCEIVED STUDENT PREPAREDNESS
13

Pease, M. A., & Kuhn, D. (2011). Experimental analysis of the effective components of

problem-based learning. Science Education, 95(1), 57-86. doi:10.1002/sce.20412

Schwartz, D. L., & Martin, T. (2004). Inventing to prepare for future learning: The hidden

efficiency of encouraging original student production in statistics instruction. Cognition and

Instruction, 22(2), 129-184. doi:10.1207/s1532690xci2202_1

Scott, C. (2012). An investigation of science, technology, engineering and mathematics (STEM)

focused high schools in the U.S. Journal of STEM Education : Innovations and Research,

13(5), 30.

Stearns, L. M., Morgan, J., Capraro, M. M., & Capraro, R. M. (2012). A teacher observation

instrument for PBL classroom instruction. Journal of STEM Education : Innovations and

Research, 13(3), 7.

Strobel, J., & Van Barneveld, A. (2009). When is PBL more effective? A meta-synthesis of

meta-analyses comparing PBL to conventional classrooms. Interdisciplinary Journal of

Problem-Based Learning, 3(1), 4.

PBL AND PERCEIVED STUDENT PREPAREDNESS
14

Appendix A

Survey of exposure to project based learning (PBL) and inquiry learning (IL) with student
self-reported levels of preparedness in first-year post-secondary engineering programs.

The purpose of this survey is to assess prior exposure to Project Based Learning and Inquiry
Learning and to report levels of preparedness for first year university engineering courses. This
survey has 20 questions and should take no more than 5 minutes of your time. Your
participation in completing the survey is voluntary and greatly appreciated!

This survey is anonymous. It is not a part of your grade and your instructor will not have access
to the data. The researchers involved in this study will not know which students have or have
not chosen to participate in the survey and information collected will be used to study statistical
trends only and cannot be used to identify individuals.

Please respond to each of the statements below, indicating the extent to which you agree or
disagree with each one, using the following scale:

5 = Strongly Agree
4 = Somewhat Agree
3 = Neutral
2 = Somewhat Disagree
1 = Strongly Disagree

Question Response

Strongly Agree -------------- Strongly Disagree

In my high school classes I was
frequently asked to study complex, open 5 4 3 2 1
questions that you couldnt just google
the answers to.

Most of my coursework in high school 5 4 3 2 1

involved completing worksheets.

I was expected to conduct my own 5 4 3 2 1

research in high school.

In most of my high school classes, the 5 4 3 2 1

teacher delivered the lesson and I took
notes.

I frequently had the ability to study topics 5 4 3 2 1

and questions of personal interest to me
in high school.
PBL AND PERCEIVED STUDENT PREPAREDNESS
15

I was expected to brainstorm questions Strongly Agree -------------- Strongly Disagree

about new topics when they were
presented in high school. 5 4 3 2 1

I frequently used concept-maps in high 5 4 3 2 1

school.

My course subjects and units were mostly 5 4 3 2 1

focused on real world problems in high
school.

I was expected to participate in group 5 4 3 2 1

feedback before submitting my
assignments in high school.

I was presented with problems that 5 4 3 2 1

required me to research a solution in high
school.

I presented or displayed my work to the 5 4 3 2 1

class for most units of study in high
school.

Part of my unit evaluation in high school 5 4 3 2 1

was to reflect on my own learning.

I feel my high school experience prepared 5 4 3 2 1

me well for my current engineering
courses.

I had seen most of the topics covered in 5 4 3 2 1

my post-secondary courses I had seen
previously in high school.

Compared to my peers in post-secondary 5 4 3 2 1

education, I am more effective at working
in groups.

Compared to my post-secondary peers, I 5 4 3 2 1

am more effective at managing my time.

I felt prepared for the core content of my 5 4 3 2 1

post-secondary courses.

Compared to my post-secondary peers, I 5 4 3 2 1

feel that I am effective at presenting my
findings in front of an audience.
PBL AND PERCEIVED STUDENT PREPAREDNESS
16

I feel comfortable participating in Strongly Agree -------------- Strongly Disagree

post-secondary classroom discussions.
5 4 3 2 1

My high school experiences prepared me 5 4 3 2 1

for writing post-secondary exams.

What is your gender? Open field

What is your age? Open field

Do you have a family member who is in a Open field

science, technology, engineering or
mathematics related field?

Would you consider your high school to Open field

be urban or rural?

Did you graduate from a BC high school? Open field

If you graduated from a BC high school, Open field

how many years did you study within the
BC education system?

Click here to submit your responses

Thank you for your assistance!
If you have any questions about this survey, please feel free to contact me:
Researcher - name@email.com