Вы находитесь на странице: 1из 34

Running head: EVALUATING THE LIBYAN ESL PRE-SERVICE TEACHERS’

PREPARATION PROPGRAM

Evaluating The Libyan ESL Pre-service Teachers’ Preparation and Development


Program

Khalifa Elgosbi

Northern Illinois University

ETR 531 / Spring 2018

 
Evaluating The Libyan ESL Pre-service Teachers’ Preparation Program 2  

TABLE OF CONTENTS

Introduction

Introduction and Background.................................................................................................. 4

About the Needs Assessment .................................................................................................. 4

The Purpose of the Assessment................................................................................................ 5

Research Questions................................................................................................................... 6

Conceptual Framework..............................................................................................................6

Methods and Data Collection.....................................................................................................7

Strengths and Limitations...........................................................................................................8

About the interview................................................................................................................... 8

About the data........................................................................................................................... 10

Key findings............................................................................................................................. 14

Significance of the proposal .................................................................................................... 15

Purpose of the proposal............................................................................................................ 16

Methodology

Methodology............................................................................................................................. 16

Research questions.................................................................................................................... 18

Design of the Evaluation........................................................................................................... 18

The Quantitative Phase.............................................................................................................. 19

The Qualitative Phase................................................................................................................ 20

Ethical Principles/ Human Subject Compliance........................................................................ 21


Evaluating The Libyan ESL Pre-service Teachers’ Preparation Program 3  

Research Questions and Hypotheses......................................................................................... 21

Variables.................................................................................................................................... 22

Population and Sample.............................................................................................................. 23

Instrumentation.......................................................................................................................... 23

Research Setting......................................................................................................................... 24

Participants................................................................................................................................. 25

Validity Issues............................................................................................................................ 25

Data Collection........................................................................................................................... 27

Description of Data Analysis Methods...................................................................................... 28

Timeline...................................................................................................................................... 29

Program Evaluation Summary.................................................................................................... 30

References................................................................................................................................... 32
Evaluating The Libyan ESL Pre-service Teachers’ Preparation Program 4  

Evaluating The Libyan ESL Pre-service Teachers’ Preparation Program

Introduction and Background

In the 21st century educational context, technological tools are not only manifestations of

the learners’ cultural environment, but also a form of literacy that is psychologically significant

for developing cognitive progress and higher thinking levels of conceptualizing (Moll, 1991).

Learners use this kind of literacy to interact with the sociocultural environment and to mediate

concepts before they can internalize them (Smolutcha, 1989). Teachers on the other hand need to

be facilitators, so their Technological Content Knowledge (TCK) and Constructivist Pedagogical

Content Knowledge (PCK) need to meet their students’ abilities for interacting with technology.

Unless they are prepared in a methodology that integrates technology in teaching, they are more

likely to fall back on the traditional methods and techniques in which they had been prepared

(Manfera & Hammond, 2006). This concern alone makes it an urgent issue for teacher educators

and teacher education institutions in Libya to assess and evaluate their teacher preparation

programs and consider technology integrating as soon as possible.

About the Needs assessment

The context of the assessment is the English Department in Janzour Education College of

Tripoli University. This department is responsible for preparing Libyan students to be teachers of

English as a second language (ESL) and the educational content is delivered in English. The

syllabus of the program is designed for ESL pre-service teachers who would like to become non-

native teachers of English language with the latest teaching skills. The teacher preparation

programs are designed to improve English language skills, theories of learning and teaching,

linguistic content knowledge, and practical teaching methods. Upon graduation, Libyan ESL pre-
Evaluating The Libyan ESL Pre-service Teachers’ Preparation Program 5  

service teachers will be able to teach English to public school pupils of middle and high schools.

Their teaching practice usually consists of two weeks classroom observation, two weeks of co-

teaching with the classroom teacher, and four weeks of teaching independently taking full

responsibility of the activities from planning to classroom management in the same way the

classroom teacher does. Educators from the English department follow these student teachers

into the school as supervisors to evaluate them, provide guidance and give feedback when

needed. The student teachers are urged to take full responsibility of the classroom as soon as they

are ready and confident to do so. Most of the student teachers find it interesting to build up their

strategies during these eight weeks of teaching practice and benefit from the help that the in-

service classroom teacher provides for them. However, neither their educators nor their future

schools are satisfied with these pre-service teachers in terms of performance and confidence

when they start teaching in the real classroom once they have been assigned jobs as secondary

public school teachers. Furthermore, pre-service teachers themselves complain that their

preparation is lacking. Nearly everyone agrees that Libyan ESL pre-service teachers are not able

to apply teaching strategies or use of instructional technology into real classroom practice once

they become working teachers. The use of technology in their teaching styles is mostly limited to

traditional audio-visual teaching aids such as tape recorders, pictures, and flip charts rather than

computers, the Internet, tablets, and smartphones.

The purpose of the Assessment

The purpose of this needs assessment is to evaluate the effectiveness of the Libyan ESL

pre-service teachers’ preparation program to integrate pedagogical strategies, linguistic

competency, content knowledge with technology. The evaluation of the Libyan ESL pre-service

teachers preparation would provide rich and deep data so that the weaknesses and the strengths
Evaluating The Libyan ESL Pre-service Teachers’ Preparation Program 6  

of the program are identified and then stakeholders can be informed about better ways to meet

the challenges.

Research Questions

The following questions will be the guidelines for this assessment:

RQ1. RQ1. How do two specific constructs of TPACK; (Technological Content Knowledge

(TCK) & Pedagogical Content Knowledge (PCK), relate to the attitudes of Libyan ESL pre-

service teachers of Janzour Education College toward integrating technology in teaching

activities?

RQ2. To what extent does Libyan ESL pre-service teachers’ technological knowledge (TCK) or

lack of technological knowledge (TCK) predict their ability to use constructivist PCK in their

teaching practicum?

RQ3. According to three stakeholders, what technological knowledge is needed for the program?

RQ4. How do the 3 stakeholders describe the goals and objectives regarding use of technology in

preparing Libyan ESL pre-service teachers to teach with technology in their teaching practicum?

Conceptual Framework

According to the Theory-Based Model, which I am going to use in my evaluation, it is

essential to have a theoretical framework. Pedagogical Content Knowledge (PCK) was originally

developed by Shulman in 1986 and as technology continued to advance, the focus on Shulman’s

PCK enhanced an experimental interest for integrating technology with pedagogy and content. In

order to include technology in the model, studies of teacher preparation development have
Evaluating The Libyan ESL Pre-service Teachers’ Preparation Program 7  

developed PCK into a new framework called TPACK, i. e. Technological, Pedagogical, and

Content Knowledge. The Teaching skills and strategies are mainly a result of educational college

instruction, and are, consequently, considered a vital component of pre-service teacher

knowledge. According to Park and Ertmer (2007), teacher performance is measured in terms of

these skills and strategies, which are also connected to the pre-service teacher evaluation during

the preparation modules of teaching practice. Thus, the impact of teacher preparation programs

on the pre-service teachers’ knowledge and skills will reflect, as instructional techniques, on the

students’ achievements. If pre-service teachers’ TCK and PCK can be identified and

incorporated into their teaching practice tasks, the student teachers will acquire career skills and

strategies for working as a classroom teacher after graduating from college (Park & Ertmer,

2007). Integrating technological content knowledge (TCK) and pedagogical content knowledge

(PCK) is a requirement to meet the cultural tools that are available for learners in 21st century.

Newly existing technological tools in the form of mobile phones, tablet devices, and laptop

computers tend to allow school students to use applications and services such as text messaging,

social webs, webinars, and the like (Geng, & Disney, 2014; Kablan, & Kaya, 2014). This review

of the literature also showed that this area is still not researched in Libya despite some studies

that have investigated ICT utilization in educational institutions of the country (Rhema, &

Miliszewska, 2010).

Methods and Data Collection

This proposed assessment aims to measure the English Department’s ability to

incorporate technology in the practicum and classroom teaching of pre-service ESL teachers.

Using two constructs of TPACK, (namely PCK and TCK), the assessment will determine the

success of ESL pre-service teachers in applying knowledge in teaching practice. I utilized an


Evaluating The Libyan ESL Pre-service Teachers’ Preparation Program 8  

interview design with a set of 10 questions about use of technological knowledge in ESL

teaching. All the interviews were recorded and later will be transcribed. Transcriptions of the

interviews will then be coded and analyzed to obtain enough information about the participants’

contributions to the integration of technology into the pre-service teacher preparation program

and how they use their teacher knowledge and technological knowledge to help them teach the

subject matter.

Strengths and Limitations

The results of this assessment are consistent with a number of most recent studies.

Lately, research has concluded that technology-based instructional design enhanced students’

learning experiences and supported meaningful learning and more effective pre-service teaching

practice (Geng, & Disney, 2014). Thus the findings will help in making decisions of including

and integrating instructional technology into the daily practices of Libyan ESL pre-service

teachers to enable newly graduated instructors to teach with technology.

As it is the case in social studies, there are a variety of challenges and limitations. For

example, the longer the study will take the more tired the participants will feel and might not

choose to continue. Another limitation is my familiarity with this situation in Libya, which can

be counted as one of the risks. Although I will be in a position where I am aware of the varieties

and the details, my biases will be of high influence. I think I should be more careful not to

depend too much on my opinion or my calculations.

About the interview

The needs assessment is in use of a semi-structured interview protocol, which the

evaluator had prepared of the purpose of collecting data from stakeholders and faculty of the
Evaluating The Libyan ESL Pre-service Teachers’ Preparation Program 9  

English Language Department at interview Tripoli Education College. The interview protocol

consists of a set of questions and prompts to be asked during the interviews in order to allow the

interviewees time and space for their comments, suggestions, and opinions. The interview was

conducted on line by means of Skype Telecommunication Application software and the

interviewees were notified about the time and the duration of the interviews so that they access

an Internet hotspot in enough time for the meetings. First, on Wednesday 02/07/18 the Chair of

the English department, Dr. Nadia Nusseir, was contacted on Viber to be informed and briefed

about the idea and to offer evaluating their program as a contribution to solving one of the

teacher preparation problems in Libya. During this contact, the evaluator made it clear that it

might be difficult to collect data for this needs assessment online and that collaboration of

everyone involved would be urged and needed. Dr. Nadia was enthusiastic and helpful and she

offered to provide more than one session for the interview to be accomplished. Also, Dr. Nadia

suggested involving two of the faculty members in this assessment for the benefits of the data

collection procedures. Therefore, the evaluator contacted two faculty members who are involved

in designing the teaching practicum for ESL pre-service teachers as part of their teacher

preparation at the English language department, namely, Dr. Ahmed Maher and Dr. Muhssen

Abubaker.

It was decided that Dr. Nadia’s interview would be conducted on Sunday 02/11/18 at 9:15 am

Libyan local time, which corresponds to 1:15 am Illinois/Dekalb time. In case there was a need

for another session it would be on Monday 02/12/18 at 5:00 pm Libyan local time (9:00 am

Illinois/Dekalb).

In addition it was decided that the interviews with the faculty members would be on Tuesday

02/13/18 at 9:30 am and 11:45 am Libyan local time (1:30 am & 3:45 am Dekalb Time).
Evaluating The Libyan ESL Pre-service Teachers’ Preparation Program 10  

It is worth noting here that the same interview protocol was used with all three interviewees.

However, the prompts were not the same because in the course of the interview the evaluator had

to adjust the prompts depending on the interviewees’ voluntary responses and their willingness

to share as much information as possible. More over it was agreed that the Skype interview

would be video recorded on the laptop by means of Webcam, which is part of the Skype

application used to contact the respondents.

All the three interviewees were informed that their participation is voluntary and they have the

right to withdraw from the assessment procedures at any time. Their agreement, on the Viber

call, has been considered as a form of consent to be part of this needs assessment.

About the Data

Data for this assessment were collected during three interviews that were conducted on

the dates, which had been previously decided for every participant. Therefore, after applying to

the interview protocol the data included responses to the semi-structured interview items:

Tell us a bout yourself and how you are related to the ESL pre-service teachers training

program?

1- Dr. Nadia Nusseir, the Head of English Department at Tripoli Education College.

2- Dr. Ahmed Maher, Department schedule Coordinator and syllabus designer.

3- Dr. Muhssen Abubaker, head of the scientific and syllabus committee of the department.

Dr. Nadia Nusseir is responsible for successful implementation of the pre-service teacher

preparation program at the college and also for evaluation of the student teaching practice, which

is every semester. Dr Ahmed and Dr Muhssen are the department’s program designers and

teaching practicum supervisors. Also, Dr Ahmed is the department’s coordinator whose job is to

negotiate and administrate the program elements among other departments in the college and
Evaluating The Libyan ESL Pre-service Teachers’ Preparation Program 11  

other institutions in the university in addition to his communication with school districts and

school principals to discuss and facilitate the program implementation during each semester.

How do you state the goals and objectives of the ESL pre-service teachers program of your

department?

To enable the department to prepare the pre-service teachers for their real teaching career once

they become working teachers in Libyan public schools to perform in the real classroom

assisting with learning ESL through the national textbook for learners such as Middle school and

high school students.

How would you like to walk me through the current structure and the context of your program?

Tell us more about the design of your program. For example, who does it serve?

The program is designed to prepare the department’s students who are in their last

semester and to graduate as teacher of ESL and to do their jobs successfully in the future.

Therefore, preparation training includes knowledge of learning and teaching theories, linguistics,

pedagogical knowledge, classroom management and lesson planning, teaching methodologies

and approaches to language teaching, as well as English language skills of reading, listening,

speaking and writing. In addition, the students’ command of English has to be raised all the time

to meet the instructional techniques of using the target language as the medium of instruction.

Can you walk me through the demographics and details of the targeted students?

Students of the English department are mostly of Libyan nationality students, who have

finished their high school study and joined university level. They are students who have chosen

to become teachers of English once they have completed their college level study at this English

Department. Their ages are between 17 and 25 depending on their starting age as children (5/6)

and on their pace of completing each stage of their study until they are university students. Most
Evaluating The Libyan ESL Pre-service Teachers’ Preparation Program 12  

of them are single, but sometimes there are some married students depending on their social

background. Libyan society is a mixture of Arabs, Berber, Tuareg, and Tebou, with about 65%

of Arab ethnic background citizens. Thus, the majority of the students at the department are

Arabs and the other ethnicities are only a minority. Most of the students are females as it is

common sense in Libya that education colleges are usually an educational destination for female

students. Perhaps, the most important thing in our context is the fact that all the students speak

Arabic as their first language because Arabic is the official language in the country. This fact

makes them real ESL learners who have to struggle with the linguistic differences and other

learning problems.

So, why have you decided to go for a program evaluation? Is there a purpose of conducting such

an evaluation?

The English department in Tripoli Education College noticed that the performance of its

graduates when they become ESL teacher in public schools is not satisfactory. All parents,

school principals, educational mentors, and the ministry authorities and the society at large have

always complained that the output of education colleges especially English departments is not

meeting the national standards for teaching school children to speak, read and write ESL. So the

English department has decided to evaluate their program in order to identify the weaknesses and

the strengths of their programs and to adopt policies for improving their output.

And what are the reasons that make your program and/or your policies in need of evaluation?

Students study for four years or eight semesters and they have to finish 92 credits in

English language, Linguistics, methodology, Arabic language, IT, and practical teaching. Their

overall achievement should lead to their graduation from the department and from the college.

However, due to attainment and English background problems most of the students find it
Evaluating The Libyan ESL Pre-service Teachers’ Preparation Program 13  

difficult to reach a competency levels even if they pass academically and complete the

requirements for graduating. This issue is affecting their future career because their weaknesses

will be carried over into their teaching performance. The department wants to evaluate so that a

change in the policy would be made on strong grounds and would lead to a change in the

competency of the department’s output.

What challenges or potential barriers may threaten the program evaluation?

Perhaps there would be some limitations to the evaluation of this program, especially because

time will restrict us from interviewing the students themselves or some of them. I will be able to

interview the Department Chair, and the coordinators and I am using a survey to collect data

from the staff and faculty at the department, but I will not be able to collect data from the parents

for example. I think this program is multifaceted and need s more tools of evaluation and would

require more time to collect the most data available regarding the implementation of the

program.

What data, if any, currently exist relative to program goals, objectives, and activities?

The target data is related to use of technology in the pre-service teachers knowledge and in the

departments facilities and learning environment of the college.

Who are the stakeholders?

Dr. Nadia Nusseir, the Head of English Department at Tripoli Education College, Dr. Ahmed

Maher, Department schedule Coordinator and syllabus designer, and Dr. Muhssen Abubaker,

head of the scientific and syllabus committee of the department are the stakeholders who are

already involved in the evaluation and the needs assessment.

However, the list can be extended to include other people in charge such as The dean of the

college, the Libyan Ministry of higher and Scientific research, other educational authorities such
Evaluating The Libyan ESL Pre-service Teachers’ Preparation Program 14  

as the school districts and the educational guidance and inspection, the school counselors the

parents’ board and in-service teachers’ Union, and some NGO’s whose field is related to

educational work and teaching and learning.

Key Findings

Libyan ESL pre-service teachers in particular are in need of using technology because

language learning and teaching has become dependent on technological tools such as the

Internet, wiki, podcasts, Youtube technologies and a lot more. In consistency with Geng and

Disney (2014), this assessment has indicated that most of these pre-service teachers belong to the

digital generation and can display enough abilities and positive attitudes for using technology

and embracing its potential in teaching. It also revealed that easier to use technologies such as

smartphones, tablets, and email services are available on the personal level and can be utilized by

teacher education ESL pre-service and student teachers. Moreover, Libyan teacher preparation

programs do not contain integrating technology. Due to problems in infrastructures, lack of plans

to use proper technology, and other sociopolitical problems, colleges of education in Libya are

not benefiting from the facilities and advantages that technology might bring to teacher

education.

The needs assessment revealed the fact that the research methodology needs to be adjusted.

In other words, instead of using a case study qualitative design only, there is a need for using a

mixed method design so that information about the program can be collected from the students

as well. Also, according to the Theory-Based Model, which I am using to guide the program

evaluation, it is required that quantitative approach is utilized in the evaluation design. Therefore,

a quantitative survey will be administrated to collect data from the pre-service teachers, who are

students of the English department where the evaluation is taking place. More will be explained
Evaluating The Libyan ESL Pre-service Teachers’ Preparation Program 15  

in the next chapter about the methodology of the study and about including another phase, which

will be quantitative design. Thus the findings the needs assessment has resulted in a decision to

alter our research design, which now is going to be mixed methods design with two phases. The

first phase is the quantitative phase in which data about the program will be collected from the

ESL pre-service teachers. Then, the other phase will be qualitative and data will be collected

from stakeholders by means of online interviews.

Significance of the proposal

The findings of the needs assessment indicated an evaluation of the Libyan ESL pre-service

teachers’ program is needed and a mixed method design is essential for conducting the program

evaluation. ESL pre-service teachers at Janzour Education College in Tripoli, Libya should be

prepared in knowledge of integrating and using technology to improve their teaching techniques

and strategies in the language classroom.

The evaluation proposal is to investigate the effectiveness of the current program in

terms of integrating technology in the components of the teaching practicum for the ESL per-

service teachers. This proposal is going to help these pre-service teachers in acquiring

technological, pedagogical and content knowledge that is sufficient to meet their needs as pre-

service teachers and later to enhance their ability to use technology once they are working

teachers. The idea of the proposal is to use constructivist theoretical framework for designing a

teaching practice module through integrating technological knowledge with pedagogical and

content knowledge. The plan of the program is to start in Spring 2018 and introduce the

evaluation results to the English department in Janzour Education College of Tripoli University.

The pre-service teachers will benefit from this proposal because at the moment they are
Evaluating The Libyan ESL Pre-service Teachers’ Preparation Program 16  

being prepared through traditional teaching methods and approaches. The outcomes of the

current preparation do not match the content in the textbook so pre-service teachers find it

difficult to apply what they have learnt to the real situation in the public school curricula. This

program will solve this problem and provide the pre-service teachers with module content that

includes technological knowledge, principles of instruction design, educational technology

solutions, practice of using technological tools, web based learning design, and constructivist

theory principles. The content for the teaching practice module of this program will include

practices of using technological tools to facilitate learning and to create learning opportunities

when teaching English language to speakers of other languages.

Purpose of the proposal

Evaluating the English department’s preparation program will include collecting both

quantitative and qualitative data. The purpose of the quantitative data is to examine the current

the ESL pre-service teachers’ technological content knowledge TCK and their constructivist

pedagogical Content knowledge PCK (as independent variables) and how these variable are

related to the ESL pre-service teachers’ perceptions and attitudes toward integrating technology

into their practicum program. On the other hand, the purpose of the qualitative data is to explore

some phenomenal factors regarding the implementation of the current program. The qualitative

interviews are with three stakeholders in the English Department, who have already said that

they are enthusiastic to participate in this evaluation and to provide the needed responses for the

interview questions.

Methodology

The research design for this evaluation will be utilizing a mixed method design, which
Evaluating The Libyan ESL Pre-service Teachers’ Preparation Program 17  

consists of two phases. The first phase will be the quantitative part of the proposal, in which data

will be collected from the ESL pre-service teachers by means of an electronic survey previously

developed and used in a similar research. The second phase will be the qualitative part of the

study, and data will be collected from three stakeholders who are in charge of the English

Department’s decisions and syllabus design for the teacher preparation program.

According to the Theory-Based Model, which I am going to use in my evaluation, it is

essential to have a theoretical framework. Pedagogical Content Knowledge (PCK) was originally

developed by Shulman in 1986 and as technology continued to advance, the focus on Shulman’s

PCK enhanced an experimental interest for integrating technology with pedagogy and content. In

order to include technology in the model, studies of teacher preparation development have

developed PCK into a new framework called TPACK, i. e. Technological, Pedagogical, and

Content Knowledge. The Teaching skills and strategies are mainly a result of educational college

instruction, and are, consequently, considered a vital component of pre-service teacher

knowledge. According to Park and Ertmer (2007), teacher performance is measured in terms of

these skills and strategies, which are also connected to the pre-service teacher evaluation during

the preparation modules of teaching practice. Thus, the impact of teacher preparation programs

on the pre-service teachers’ knowledge and skills will reflect, as instructional techniques, on the

students’ achievements.

In the quantitative phase, I will collect data related to the TCK and PCK of a Libyan

English department ESL pre-service teachers’ in terms of pedagogical strategies, linguistic

competency, content knowledge, and ability for integrating technology using an adjusted

TPACK model. The needs assessment results indicated deficiency on the part of the teaching

practice modules of pre-service teachers who are studying at the English department in Janzour
Evaluating The Libyan ESL Pre-service Teachers’ Preparation Program 18  

Education College of Tripoli University. This deficiency is associated with the fact that the

modules do not include much technological knowledge. The pre-service teachers who are

studying at the department are not exposed to any information or practices of how to integrate

technology in their teaching. Also, they are suffering from lack of facilities and technological

tools that can be use when incorporating technology into teaching tasks. Both the learner analysis

and the stakeholders interviews showed that pre-service teachers in the English Department have

problems in applying what they have learned from their teaching practice modules into the

classroom teaching which they are expected to perform when they become working teachers in

the Libyan public schools around the country. Their teaching practice usually consists of two

weeks classroom observation, two weeks of co-teaching with the classroom teacher, and four

weeks of teaching independently taking full responsibility of the activities from planning to

classroom management in the same way the classroom teacher does. However, neither their

educators nor their future schools are satisfied with these pre-service teachers in terms of

performance and confidence when they start teaching in the real classroom once they have been

assigned jobs as secondary public school teachers. Furthermore, pre-service teachers themselves

complain that their preparation is lacking. The use of technology in their teaching styles is

mostly limited to traditional audio-visual teaching aids such as tape recorders, pictures, and flip

charts rather than computers, the Internet, tablets, and smartphones.

This chapter is a description of the research methodology and procedures that will be used

to answer the following research questions:

Research Questions

Integrating is a requirement to meet the cultural tools that are available for learners in 21st
Evaluating The Libyan ESL Pre-service Teachers’ Preparation Program 19  

century. The following questions will be the guidelines for this assessment:

RQ1. How do two specific constructs of TPACK; (Technological Content Knowledge (TCK) &

Pedagogical Content Knowledge (PCK), relate to the attitudes of Libyan ESL pre-service

teachers of Janzour Education College toward integrating technology in teaching activities?

RQ2. To what extent does Libyan ESL pre-service teachers’ technological knowledge (TCK) or

lack of technological knowledge (TCK) predict their ability to use constructivist PCK in their

teaching practicum?

RQ3. According to three stakeholders, what technological knowledge is needed for the program?

RQ4. How do the 3 stakeholders describe the goals and objectives regarding use of technology in

preparing Libyan ESL pre-service teachers to teach with technology in their teaching practicum?

Design of the Study

The research design, which I am going to use in my proposal, is a mixed-method design.

This research design is appropriate in my context because the quantitative survey has the

potential of measuring the relationship between the dependent and independent variables;

whereas, the qualitative data will be useful for further in-depth data that can be used in

discussions of the results. Both, quantitative and qualitative data will be analyzed and used in

understanding the factors that would affect the implementation of a new program to improve the

English Department’s performance.

The Quantitative Phase

In the quantitative phase, I need to determine the relationship between integrating

Pedagogical Content Knowledge (PCK) and teacher Technological Content Knowledge (TCK)
Evaluating The Libyan ESL Pre-service Teachers’ Preparation Program 20  

(i.e. independent variables) and the improvement of Libyan ESL pre-service teachers’

preparation program (i.e. independent variable). I am going to administrate the survey during the

Fall 2018 semester. Survey design would be feasible because it would allow me measure the

relationships between the ESL pre-service teachers’ technological knowledge and their attitudes

and perceptions toward using technology in their practice as training teachers. The changes in the

independent variables of TCK and PCK would predict the use of technology in the teacher

performance in the future of the English department’s ESL teacher preparation courses.

This quantitative part of the proposal will apply a correlational research design. An online

survey research will be used to collect data, which will be statistically analyzed to identify the

relationship between integrating Pedagogical Content Knowledge (PCK) and teacher

Technological Content Knowledge (TCK) and the improvement of Libyan ESL pre-service

teachers’ preparation program. This proposal will be using an existing survey, which was

developed by Koh, Chai1, and Tsai (2014). They developed a survey instrument that aimed to

examine “the constructivist-oriented TPACK perceptions of 354 practicing teachers in

Singapore” (p. 188).

For questions Q1 and Q2, the researcher will use inferential statistics analysis to measure

the regression between the study variables, as these two questions will measure the relationship

between two or more variables. According to Creswell (2012), a regression analysis approach is

used to describe and measure the relationship between two or more variables (Creswell, 2012).

For example, this approach will use simple regression to help in finding the answer to Q2 and

predict whether relationships exist among the pre-service teachers’ technological knowledge

(TCK) (as a dependent variable), and the their PCK (as independent variables), which represent

components that can predict their use of this PCK in the future teaching activities.
Evaluating The Libyan ESL Pre-service Teachers’ Preparation Program 21  

The Qualitative Phase

Qualitative data analysis and theme-coding procedures will be used to interpret the

interviewee responses in order to address questions Q3 and Q4. I have already designed an

interview protocol that includes questions about the current status of the preparation program, the

goals and objectives of the program, the demographics of the targeted students, challenges and

potential barriers that might affect the evaluation, the department’s previous experience of

integrating technology, and the availability of technological facilities and resources. In general,

the interview protocol will be used to collect data about the department’s current performance

and potential for change. As a researcher, I will be using qualitative design to gain access to

details and information, which these stakeholders know much better than I do. All my three

participants are actually in charge of the administrative and syllabus design components of the

above-mentioned ESL pre-service teacher preparation program, which I am evaluating.

Therefore, they are knowledgeable enough to enrich the qualitative data for this evaluation. The

interview will be employed to elicit data about aspects of owning technology; using

technological tools; operating, downloading and uploading of programs and applications.

Ethical Principles/ Human Subject Compliance

This study will be conducted in an ethical, moral and responsible manner and the

participants will have the right to decide when, to what extent their participation is continued

Creswell (2012). According to the requirements of the Institutional Review Board (IRB), this

student will apply to the NIU office of compliance for permission to collect data and conduct the

study procedures. Moreover, the researcher will make sure that the participants know their rights

and a consent form will be administrated before conducting the study.


Evaluating The Libyan ESL Pre-service Teachers’ Preparation Program 22  

To ensure privacy during data collection, both the survey and the interview protocol will

be recorded and documented anonymously to the target population. Moreover, the data will be

stored on a password protected hard drive, which belongs to the researcher and can only be

accessible by him. Participation in the study is voluntary and all the participants will be given a

consent form to read and sign as an entry to the survey. The consent form will include an

explanation of the research purpose and that they are free to withdraw from the study at any time

without being held responsible for any commitments. As the survey is an electronic instrument,

the consent form will be placed at the beginning of the survey and the participants will have to

sign it in an electronic format before answering any survey questions. The proposed survey of

the study is an existing instrument and it will be used upon permission from the original authors

to use their survey after having modified it to the Libyan context.

Research Questions and Hypotheses

The data, which this evaluation is going to collect, will be used to answer the four

research questions of the study. Consequently, each question will be used to determine one

aspect of the research problem. For example, question number one Q1 will be used to identify

the relationship between the independent variables (i. e. ESL pre-service teachers’ pedagogical

content knowledge PCK and their technological content knowledge TCK) and Libyan ESL pre-

service teachers’ attitudes toward integrating technology in teaching practicum (the dependent

variable). Actually, questions Q1 and Q2 will be the focus of the survey; i.e. the quantitative

instrument of the study, and the data will be used to measure statistically the relationship

between these three variables and to test the following hypotheses:


Evaluating The Libyan ESL Pre-service Teachers’ Preparation Program 23  

1. There is a positive relationship between the pre-service teachers’ technological

knowledge (TCK) and their attitudes toward integrating technology.

2. There is not a relationship between the pre-service teachers’ technological knowledge

(TCK) and their attitudes toward integrating technology.

3. There is a positive relationship between the pre-service teachers’ pedagogical knowledge

(PCK) and their attitudes toward integrating technology.

4. There is not a relationship between the pre-service teachers’ pedagogical knowledge

(PCK) and their attitudes toward integrating technology.

On the other hand, the qualitative data will be used to answer the research question Q3 and

Q4. As for the instrument in the qualitative part of the study, an interview protocol will be

designed to ask questions about the current state of the English department’s syllabuses,

technology integration future plans, and the target goals and objectives of the English

Department.

Variables

In this proposed evaluation, the two constructs of PCK and TCK will be employed as the

independent variables that will predict the dependent variable. The dependent variable will be the

participants’ perceptions and attitudes toward integrating technology in the teaching practice

sessions and later in the real classroom.

Population and Sample

The accessible population is about (N = 80) Libyan pre-service ESL teachers to participate

in the quantitative phase, and three (N = 3) stakeholders to participate in the qualitative phase.
Evaluating The Libyan ESL Pre-service Teachers’ Preparation Program 24  

The ESL pre-service teachers are English department students, who are currently studying at

Janzour Education College in Tripoli University and who, at the time of the study, are enrolled in

the teaching practicum modules of teacher preparation programs. The sampling for the

quantitative procedures will be convenience random sampling and all the students will have

equal chances of participation in the study regardless for their age, gender or their technology

experience. The participants will receive the survey instrument electronically on their emails and

on the University of Tripoli Website. Before administrating the survey and the interviews, the

researcher will seek permission from the Northern Illinois University Institutional Review Board

(IRB) as well as officials and administrative staff at Tripoli University.

Instrumentation

The research design for this study will be mixed methods. The quantitative survey consists

of 32 items. The study participants will be asked to rate “each item on a seven-point Likert-type

scale where 1 - Strongly Disagree, 2 - Disagree, 3 - Slightly Disagree, 4 – Neither agree nor

disagree, 5 - Slightly Agree, 6 - Agree, 7 - Strongly Agree.” (Koh, Chai1, & Tsai, 2014, p. 188).

The 32 items of the questionnaire will be used for the collection of quantitative data, and its

internal consistency will be measured by using Cornbach’s coefficient alpha, which employs the

statistical formula (a = < .05). The sample will contain 80 Libyan ESL pre-service teachers. The

sampling will cover the ESL pre-service teachers in Janzour Education College in Tripoli

University, which is one of the many state universities in Libya. The sample will be

representative of the target population of the study since ESL pre-service teachers are part of the

target population that includes all the pre-service teachers in higher education institutions in

Libya. The quantitative findings will be used to reveal the predictors that control ESL pre-service

teachers’ attitudes toward using technology in the ESL classroom in Libyan pubic schools. Also
Evaluating The Libyan ESL Pre-service Teachers’ Preparation Program 25  

the findings from the qualitative phase will be used to identify stakeholders’ attitudes and

perceptions toward integrating technology in the teacher preparation programs of Libyan ESL

pre-service teachers.

Research Setting

The English Department in Janzour Education College of Tripoli University is part of a

national Libyan general program of preparing ESL teachers for becoming non-native teachers of

English in the Libyan education system. University of Tripoli is the largest university in Libya. It

is located in the capital city of Libya (Tripoli), and it includes many colleges that serve different

majors such as education, medicine, dentistry, science, languages, arts, economics, etc. The

university consists of 26 colleges for different academic majors such as science, math, languages,

medicine, and social sciences. It has four education colleges distributed geographically to cover

the largest area of student population. These colleges provide teacher preparation to Libyan

students of higher education to enable them to become teachers in different fields. Among the

field that education colleges serve is the ESL pre-service teachers preparation. In general,

University of Tripoli provides teacher preparation educational services to a student population of

about 40,000 Libyan students. All the students have to do teaching practicum as part of their

teaching preparation to be able to teach ESL in secondary and middle schools in Libya. The

researcher is familiar with this university because he has been working there as a lecturer for

more than ten years. This familiarity is one of the reasons behind the researchers choice of this

institution to be the setting of the study. The survey will be electronically sent to all individuals

identified as English department faculty working in the colleges and schools of University of

Tripoli, who will be asked to help administrate the survey to their ESL pre-service teachers.

However, the interview will be conducted online via Skype Communication Program. The
Evaluating The Libyan ESL Pre-service Teachers’ Preparation Program 26  

interviewer will negotiate the time for the interview to guarantee convenient times for the

participants (i. e. the 3 stakeholders).

Participants

The purpose of the study will be to investigate the factors that predict the pre-service

teachers’ perceptions and attitudes in relation to variables of technological knowledge (TCK),

pedagogical knowledge (PCK). Also to identify whether stakeholders’ attitudes and perceptions

toward integrating technology would enhance ESL pre-service teaching with technology in their

future careers. In total, 80 of Janzour Education College ESL pre-service teachers will be

surveyed as an accessible population that might represent the target population, which contains

all Libyan ESL pre-service teachers, who study at higher education colleges and institutions in

the state of Libya.

Validity Issues

Since this study is utilizing a survey, which will be adopted from an existing study, I will

conduct several procedures to insure validity and reliability. First, the researcher will insure

validity by piloting the survey to some Libyan ESL pre-service teachers in advance to relate the

instrument to the Libyan context and to avoid data collection bias. The sampling will be random

during the pilot practice in order to validate the survey through obtaining correct results and

valid data. Moreover, using a prior survey that is proved and validated in previous research will

strengthen both internal and external validity of the instrument.

Also, validity and reliability measures will be taken prior to administration of the

interviews. In order to do this, the instrument for the qualitative part has to be validated through

designing the research procedures such as preparing the time line for the interviews and

observations, writing the interview protocols, designing prompts, and administrating the
Evaluating The Libyan ESL Pre-service Teachers’ Preparation Program 27  

interviews so that the participants are put at ease. Also, as an interviewer, I have distant myself

from any personal or social relationships with the participants. My role is full of contact

scenarios with the participants so it is a double-edged role and I have to be very careful not to

confuse my role as a researcher with my identity as a friend and a colleague of the participants.

Confusing these roles might result in lack of seriousness and then it can affect the validity of the

instrument and thus reliability of the findings. For example, they might think I know about the

program as much as they do and so they do not need to tell me in details about something. If this

happens, then I need to adjust my interview questions and use the prompts to help obtain more

information.

This evaluation is aimed at identifying challenges and factors related to the Libyan ESL

pre-service teachers’ attitudes toward integrating technology in teacher preparation courses in

Libya. It also is an attempt to understand the English Department stakeholders’ viewpoints

regarding the current implementation and future improvement of the program. The evaluation

will examine how teacher technological knowledge might influence ESL pre-service teachers’

attitudes toward using technology in the ESL classroom. Also, the analyses will determine how

stakeholders’ perceptions, attitudes, and concerns might influence the implementation of the ESL

pre-service teachers’ preparation program.

The study incorporates the following variables: Libyan ESL pre-service attitudes toward

integrating technology in their program (as dependent variables) and factors of two specific

constructs of teachers’ TPACK, namely PCK and TCK (as independent variables). To ensure the

validity of the study several steps will be taken to limit threats to both external and internal

validity. According to Creswell (2012), in educational research studies there are several potential

threats, which can be an issue to external and internal validity. Therefore, the researcher will
Evaluating The Libyan ESL Pre-service Teachers’ Preparation Program 28  

conduct a validity test by applying pilot study and experts in the field of research will be asked to

check the interview protocol in order to eliminate any threats to validity. In addition, colleagues

and faculty at ETRA Department, College of Education, NIU will be asked to review the

questions control content validity. Finally, statistical analyses and SPSS procedures will also be

used in analyzing data to eliminate further threats to internal validity related to the quantitative

survey of the evaluation.

Data Collection

This is a proposal of a mixed-method design to investigate the development of Libyan pre-

service teachers’ preparation program. The study will incorporate a survey with 32 Likerit scale

items. This previously developed and used survey is designed based on a seven point Likert-type

scale that requires participants to select from seven scale options: “1 - Strongly Disagree, 2 -

Disagree, 3 - Slightly Disagree, 4 – Neither agree nor disagree, 5 - Slightly Agree, 6 - Agree, 7 -

Strongly Agree.” (Koh, Chai1, & Tsai, 2014, p. 188). The survey will measure the pre-service

teacher’s pedagogical content knowledge PCK and their technological content knowledge TCK.

After modification to meet the Libyan context variability, the survey will be prepared on

Qualtrics and will be distributed electronically using the participants’ emails and the Janzour

Education College’s Website as well as the English Department Facebook Page on the Internet.

The survey introduction section will include a brief introduction and the participants’ Consent

Forms and it will be prepared electronically to allow the participants to proceed with the survey

only after having accepted to participate by signing the consent form. Eventually, the participants

will have one month to respond, and a reminder email will be sent to the participants on their

emails to draw their attention to completion of the survey.


Evaluating The Libyan ESL Pre-service Teachers’ Preparation Program 29  

The qualitative part of the proposal includes a semi-structured interview protocol, which

will be used to collect data from three stakeholders. The interview questions will be based the

purpose that all the items and open-ended questions of the interview should seek data related to

integrating technology into the teacher preparation program. Each of the three participants will

be asked to share her/his experience of technology use in teaching and to respond to questions

related to their perceptions toward integrating technology into the future programs of the

department. Also, the stakeholders will be asked to participate their knowledge regarding

Description of Data Analysis Methods

For the quantitative the data analysis, the evaluation will utilize correlation matrices to

answer Q1 and simple regression analyses to answer Q2 and to analyze the data collected from

the survey. Conducting statistical analyses is necessary for the reliability measures, as they will

identify any inconsistencies that might occur due to deviant or missing data. Both descriptive and

correlational analysis will be applied to the data using the cloud SPSS system at NIU. SPSS

calculations of the mean, the standard deviation, Kurtosis, and Skewedness will be obtained to

maintain the reliability of the descriptive data of the study. In addition, normality will be

measured using the SPSS output distributions, frequencies, and normality of residuals and scatter

plots. Scores, outputs, and graphical analyses will be presented by means of histograms, tables,

scatterplots, and correlation matrices of SPSS.

A correlational approach is known for having more advantages as a statistical method to

describe the relationship between the study variables (i. e. TCK and PCK on one hand and

participants’ attitudes on the other). This method is statistically reliable because it allows

researchers to examine how independent variables are related to the dependent variable using the

difference scores to answer questions about the target population (Gravetter & Wallnau, 2014).
Evaluating The Libyan ESL Pre-service Teachers’ Preparation Program 30  

Furthermore, interview transcriptions, theme coding, and interpretation analysis of the

qualitative data is also necessary to provide deep and rich analyses for identifying the

stakegolders’ perceptions and attitudes toward integrating technology as a solution to the

difficulties related to the program. This technique will be useful in describing how the

interrelated factors of designing a new syllabus might be extracted fro the participants’

responses, and experience with technology.

Timeline

When
What How

February 2018 - April 2018


Prepare Proposal ETR 531 Class

June 2018 Obtain IRB & Location Complete required form


Permissions through ETRA Department

August 2018 - October


2018 Collect Data Electronic Survey &
Interviews

Nov 2018 -Jan 2019 Excel and SPSS &


Export and Analyze data qualitative analysis
procedures

Feb 2019 –March 2019 Use SPSS Tables and


Write discussion of results graphs & coded themes and
and conclusions interpretations

April 2019 Via emails


Submission of evaluation
results and suggestions

April 2019 Get feedback and further Stakeholders decision


stakeholders’ views

 
Evaluating The Libyan ESL Pre-service Teachers’ Preparation Program 31  

Program Evaluation Summary  

The purpose of this proposed mixed-method design is to evaluate the strengths and

weaknesses of the programs of ESL pre-service teacher preparation courses of Janzour Education

College of Tripoli University. The goal is to collected both quantitative and qualitative data to be

used in examining the factors that might control integrating technology to improve the outcomes

of the English Department’s teacher practicum of ESL pre-service teachers in Janzour Education

College.

A mixed-method approach is used to conduct research and evaluation when both

quantitative and qualitative data are needed to investigate a research problem that involves

complex settings such as educational environments and social science study projects. Therefore,

in this evaluation Q1 and Q2 will measure the relationship between more than two variables, so

the evaluation will use inferential statistics analysis to measure the regression between the study

variables. In addition, phenomenal descriptive data analyses will be used to address Q3 and Q4.

In this proposal, the procedures will utilize two instruments (i. e. a survey and an interview)

to help answer the questions and predict whether relationships exist among the Libyan ESL pre-

service teachers’ attitudes and perceptions toward using technology (as a dependent variable),

and the factors of PCK and TCK (as independent variables). Also, qualitative analysis will be

applied to determine what perceptions or concerns stakeholders have regarding the development

of the program to meet the ESL pre-service teachers expectations and technological knowledge,

as well as the stakeholders’ relative contributions to the improvement of the program in the

future. In addition, qualitative descriptions will be used to examine whether Libyan ESL pre-

service teachers’ attitudes and perceptions toward using technology are embraced by

stakeholders knowledge and experience of educational technology as suggested by the


Evaluating The Libyan ESL Pre-service Teachers’ Preparation Program 32  

constructivist theory. Results will be reported and presented as a final outcome of the evaluation

once all the descriptive calculations, qualitative interpretations, and mixed method analyses are

accomplished.
Evaluating The Libyan ESL Pre-service Teachers’ Preparation Program 33  

References

Bozdoğan, D. & Özen, R. (2014). Use of ICT Technologies and Factors Affecting Pre-service

ELT teachers’ Perceived ICT Self-Efficacy. TOJET: The Turkish Online Journal of

Educational Technology, 13(2), 186-196.

Creswell, J. W. (2012). Educational research: Planning, conducting, and evaluating quantitative

and qualitative research. (4th Edition), Boston, MA: Pearson Education.

Geng, G., & Disney, L. (2014). Exploring Pre-service Teachers’ Knowledge of and Ability to

Use Text Messaging. Australian Journal of Teacher Education, 39(7), 172-182.

Gravetter, F. J. & Wallnau, L. B. (2014). Essentials of Statistics for Behavioral Sciences. (8th

Edition), Belmont, CA. Wadsworth Cengage Learning.

Kablan, Z., & Kaya, S. (2014). Pre-service Teachers’ Constructivist Teaching Scores Based on

Their Learning Styles. Australian Journal of Teacher Education, 39(12), 65-77.

Koh, J. H. L., Chai1, C. S., & Tsai, C. C. (2014). Demographic Factors, TPACK Constructs, and

Teachers’ Perceptions of Constructivist-Oriented TPACK. Educational Technology &

Society, 17 (1), 185–196.

Manfra, M. M. & Hammond, T. C. (2006). Teachers’ Instructional Choices with Student-Created

Digital Documentaries: Case Studies. JRTE: Journal of Research on Technology in

Education, 41(2), 223–245.

Moll, I. (1991). The material and the social in Vygotsky’s Theory of cognitive development. 20p.

Retrieved from: http://files.eric.ed.gov/fulltext/ED352186.pdf


Evaluating The Libyan ESL Pre-service Teachers’ Preparation Program 34  

Park, S. H. &Peggy A. Ertmer, P. A. (2007). Impact of Problem-Based Learning (PBL) on

Teachers’ Beliefs Regarding Technology Use. JRTE: Journal of Research on Technology

in Education, 40(2), 247-267.

Rhema, A. & Miliszewska, I. (2010). Towards E-Learning in Higher Education in Libya. Issues

in Informing Science and Information Technology, 7, 423-437.

Smolucha, F. (1989). The relevaanc eof Vygotsky’s Theory of creative imagination for

comtemparary research on Play. Retrieved from:

http://files.eric.ed.gov/fulltext/ED314168.pdf