Вы находитесь на странице: 1из 120

2010:112

MASTER'S THESIS

2010:112 MASTER'S THESIS Behavioral Detection of Cheating in Online Examination Matus Korman Luleå University of

Behavioral Detection of Cheating in Online Examination

Matus Korman

Luleå University of Technology

D Master thesis Computer and Systems Sciences Department of Business Administration and Social Sciences Division of Information Systems Sciences

2010:112

- ISSN: 1402-1552 - ISRN: LTU-DUPP--10/112--SE

Acknowledgements

I would like to thank everyone, who contributed in, opposed to, assisted with, or otherwise helped me carrying out the study as well as writing this thesis – a result of the study. My thanks go to Dan Harnesk, PhD. (supervisor), S¨oren Samuelsson, PhD., and John Lindstr¨om, PhD., for the valuable advice and research guidance I was given; to Hugo Quisbert, PhD., Artjom Vassiljev and Viola Veiderpass for constructive op- position; to Lars Furberg for the ideas, which helped me to navigate to the research problem chosen and the interesting discussions we had; to Neil Costigan, PhD., for his inspiring work and presentations; to professor Ann H¨agerfors for managing is- sues also related to my study; and to my family for their mental support and advice. My further thanks go to Amir Molavi, Onur Yirmibesoglu, Marko Niemimaa, Elina Laaksonen, Nebojsa Mihajlovski, Vladimir Kichatov, Ali Fakhr, Darya Plankina, Anna Selischeva, Sana Rouis, Svante Edz´en, Peter Anttu, and others, who con- tributed to my thoughtflow through discussions, or supported me in different other ways.

Special thanks go to Behaviometrics AB and the people, efforts of whom relate to the study.

Also thanks to the contributions of all of you, the study has been done the way it has, and I feel having learned valuable knowledge and gained practice, for which there is use in the future.

Abstract

This thesis relates to studying possibilities of detecting online examination cheating through the measures of human-computer interaction dynamics.

The need for and use of online or computer-based examination seems to be growing, while this form of examination gives students a broader spectrum of opportunities including those for cheating, as compared to non-computerized ways of examination. The times are changing, there are many different reasons for examination dishonesty, many ways of performing it, and many ways of coping with it. Given an equilib- rium at this level, new ways of violation deserve new ways of prevention, or at least detection.

The study focuses on a method of computer-based examination cheating detec- tion based on measures of behavior and machine learning, and tries to link it to a broadly taken concept of academic dishonesty. The detection potential of this method is mainly indicated by cue leakage theory, subjects of which can be han- dled with use of pattern recognition and anomaly detection theory, all through a behavioral biometrics approach.

Contents

1 Introduction

 

1

1.1

Topic

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

3

1.2 Research goals and delimitation .

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

3

1.3 Significance of the study

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

4

1.4 Document structure

 

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

5

2 Background

 

7

2.1 Examination cheating

 

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

7

 

2.1.1 What’s wrong with cheating?

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

8

2.1.2 Why do students cheat?

 

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

9

2.1.3 The mission: preventing cheating

 

16

2.1.4 How do students cheat?

 

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

19

2.1.5 Detecting cheating as a means of prevention

 

21

2.1.6 Cheating review summary .

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

22

2.2 Specifics of distance operation .

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

26

3 Conceptual framework

 

29

3.1 Cue leakage theory

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

29

3.2 Pattern recognition theory .

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

30

3.3 Anomaly detection

 

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

31

3.4 Behaviometrics

 

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

32

 

3.4.1 Biometrics in general .

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

33

3.4.2 Specifics of behaviometrics

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

38

3.4.3 Keystroke dynamics

 

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

41

3.4.4 Mouse dynamics

 

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

43

3.4.5 Linguistic dynamics

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

44

3.4.6 ‘Special purpose’ behaviometrics

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

44

3.5 Vision of a behavioral cheating detection approach

.

.

.

.

.

.

.

.

.

.

48

 

3.5.1 The angle of attack .

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

49

3.5.2 Behavioral characteristics as the cheating detection unifier

50

3.5.3 The detection mechanism

 

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

50

4 Methodology

 

53

4.1 My setting and the research method

 

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

53

4.2 Validity of a research design .

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

55

4.3 Reliability and validity of a measure

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

56

4.4 Research design and research process

 

57

 

4.4.1

Empirical inputs

 

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

58

 

4.4.2 Observations

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

59

4.4.3 Questionnaire

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

62

4.4.4 Analysis

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

63

5

Analysis and observations

 

65

5.1 Analysis

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

65

 

5.1.1 Quantitative molecular level

 

65

5.1.2 Qualitative molecular level

 

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

66

5.1.3 Qualitative molar level .

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

66

 

5.2 Observations

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

67

5.3 Observation 1

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

67

 

5.3.1 General highlights

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

67

5.3.2 Session-specific highlights

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

68

 

5.4 Observation 2

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

70

 

5.4.1 General highlights

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

70

5.4.2 Session-specific highlights

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

70

 

5.5 Observation 3

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

73

 

5.5.1 General highlights

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

73

5.5.2 Session-specific highlights

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

73

 

5.6 Triangulative analysis remarks

 

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

75

6 Results and findings

 

77

 

6.1 Behavioral anomaly indication

 

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

77

6.2 Indicating cheating

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

77

6.3 Indication difficulties

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

77

7 Conclusion and discussions

 

79

 

7.1 Conclusion

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

79

7.2 Cheating detection and prevention approach discussion

 

80

 

7.2.1 Behaviometric aspects

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

80

7.2.2 Cheating aspects

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

82

7.2.3 Psychological aspects

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

83

 

7.3 Research approach discussion

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

84

7.4 Outlooks for further research

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

84

Appendices

 

97

A Subjects of automated observation

 

99

 

A.1

Basic structure of the analytics

 

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

99

A.2

Keystroke dynamics features

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

100

A.3

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

100

A.4

A.5

Mouse dynamics features Silence dynamics features

. Linguistic dynamics features .

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

100

101

B Subjects of manual observation

 

105

C Questionnaire and observation task content

C.1

Questionnaire

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

107

107

 

C.2

. Authentic writing and formulating

 

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

108

C.3

C.4

. Verbatim copying by listening .

Verbatim copying by reading

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

109

109

C.5

Copying by reading and reformulating

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

110

List of Figures

2.1

A cheating-extended model of Ajzen’s theory of planned behavior

 

10

2.2

Model of student cheating decision based on internal (personal) and

 

external factors

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

11

2.3

Model of cheating causation

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

15

2.4

Graphical overview of cheating and counter-cheating relations

 

24

2.5

Overview of a cheating and counter-cheating process

 

.

.

.

.

.

.

.

.

.

25

3.1

A classification example

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

30

3.2

Biometric system error rates .

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

36

3.3

A typical architecture of a biometric system

36

3.4

Fusion of biometric systems

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

37

3.5

The biometric menagerie .

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

39

3.6

An example process of mouse dynamics analysis

.

.

.

.

.

.

.

.

.

.

.

43

3.7

Deterrence mechanism of cheating detection

49

3.8

Model of the cheating detection approach

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

52

4.1

Research process overview

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

58

4.2

The observation design used in the study

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

60

4.3

The observation process (including questionnaire)

62

4.4

Data flow and control relations of the data gathering and analysis .

processes

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

63

7.1

The cheating prevention approach

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

82

A.1

A.2

. Context and process of the automated analysis part

Analytics structure

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

99

100

C.1

Example free diagram

.

.

.

.