International Indexed & Refferred Research Journal, October 2012, ISSN 0975-3486, RNI- RAJBAL 2009/30097: VoL IV *ISSUE- 37

Research PaperMathematics

Calculating and Interpreting Reliability Estimates for Achievement Test in Graph Theory - (A modern branch of Mathematics)
October,2012 * Pankit S. Gandhi * Asst. Prof. of Mathematics, Kishinchand Chellaram College, Churchgate, Mumbai A B S T R A C T
A test is a measuring tool or instrument developed to measure the level of attainment of specified instructional objectives. Achievement tests are tests designed to determine how much the learner has learned from specified content via systemic and controlled instruction. They represent a terminal evaluation of the learner's status on the completion of a course of study or training. Instrument reliability is the degree of consistency or dependability with which an instrument measures what it is supposed or designed to measure.The present study aims at analyzing the data (scores) obtained from the achievement tests written by students in the subject of Graph Theory by calculating and interpreting the test - retest and internal consistency reliability estimates. Keywords: Achievement test, Reliability, Graph Theory, Multimedia Package.

Introduction Reliability is the consistency of a measure. In educational testing, reliability refers to the confidence that the test score will be the same across repeated administrations of the test. There are four general classes of reliability estimates, each of which estimates reliability in a different way. They are: i. Inter - Rater or Inter - Observer Reliability ii. Test - Retest Reliability (also called Stability) iii. Parallel - Forms Reliability iv. Internal Consistency Reliability may be estimated through a variety of methods that generally fall into two types: Single-administration and multiple-administration. Multiple-administration methods require that two assessments be administered. In the test-retest method, reliability is estimated as the Pearson product-moment correlation coefficient between two administrations of the same measure. Single-administration methods include splithalf and internal consistency. The most common internal consistency measure is Cronbach's alpha, which is usually interpreted as the mean of all possible split-half coefficients. Cronbach's alpha is a generalization of an earlier form of estimating internal consistency, KuderRichardson Formula 20. About Graph Theory Graph theory is an important branch of modern Mathematics that is used throughout the physical, social and artificial sciences. The past few decades have witnessed a spectacular growth of Graph Theory mainly due to its applications in discrete optimization problems and due to the advent of computers. Methodology In this study, the sample consisted of a group of 38 students comprising of S.Y.B.Sc. Mathematics class of a degree college affiliated to Mumbai University. Graph

Theory constitutes two units in the syllabus of Mathematics Paper III (Computational Mathematics) at the Second Year of Bachelor of Science Degree course offered at Mumbai University for students opting for Mathematics at this level. For the purpose of the study a Multimedia package on the first unit of Graph Theory along with an Achievement test was developed by the researcher. The package and the test were validated by experts before being used for the study. Thereafter, the Multimedia package was demonstrated to the students and the same was made available to the students for self study in the computer laboratory of the college. The students were given four sessions of two and a half our each for this self study through the Multimedia package.After the study sessions the achievement test was administered to the students to determine their level of achievement. The test was of one and a half hour duration and comprised of 18 multiple choice questions of 1 mark each and 6 descriptive questions of 2 marks each, thus the total marks of the test summed up to 30. After the achievement test the answer sheets and the question papers were immediately collected back from the students. After two weeks a Retest was administered to the students. The test questions and the time allotted for the retest was exactly the same as the achievement test. The data (scores) obtained was used to find the stability and reliability of the test using SPSS v 17. Calculations In the test-retest method, reliability was estimated as the Pearson product-moment correlation coefficient r between the achievement test and retest. The following value was obtained: Pearson product moment correlation coefficient = r = 0.765. Cronbach's

36

RESEARCH ANALYSIS AND EVALUATION

International Indexed & Refferred Research Journal, October 2012, ISSN 0975-3486, RNI- RAJBAL 2009/30097: VoL IV *ISSUE- 37

alpha was calculated to determine the internal consistency or average correlation of items in the achievement to gauge its reliability. The following value was obtained: Cronbach's alpha = = 0.810. Findings (Interpretation) The value of r always falls within the range -1 to +1. An r of -1 means a perfect negative correlation and an r of +1 means a perfect positive correlation. An r of 0 means zero correlation. Guilford (1982) offers an interpretation of the value r, as shown in table below. Table 1: Interpretation of Pearson product - moment correlation coefficient (r) Considering Guilford's values, the correlation between achievement test and retest the value of 0.765 can be considered as high, indicating a strong relationship and hence high stability of the achievement test instrument. Value of r Interpretation < 0.2 Slight, almost no relationship. 0.2 - 0.4 Low correlation, definite but small relationship. 0.4 - 0.7 Moderate correlation, substantial relationship. 0.7 - 0.9 High correlation, strong relation ship. 0.9 - 1.0 Very high correlation, very de pendable relationship.

Cronbach's alpha Internal consistency < 0.5 Unacceptable 0.5 - 0.6 Poor 0.6 - 0.7 Questionable 0.7 - 0.8 Acceptable 0.8 -0.9 Good 0.9 - 1.0 Excellent Table 2: Interpretation of Cronbach's alpha (?) A commonly accepted rule of thumb for describing internal consistency using Cronbach's alpha is as given in the table below .Thus, from the Cronbach's alpha value of 0.810 it can be concluded that the measuring instrument of achievement test was internally consistent. Suggestions The following suggestions are made from the findings of the present study: * Reliability of test instruments should be important considerations for teachers while planning test to measure student achievement. * Educational policy makers should push for qual ity test item writing and insist on reporting reli ability estimates across test instruments to make the prevalent examination system more meaningful.

R E F E R E N C E
[1] Mehrens, W. A., and LehmaN, I. J. (1991) Measurement and Evaluation in Education and Psychology, (4th ed.). Rinehart and Winston Inc: Orlando. [2] Oluwatayo, James Ayodele. (2012). Validity and Reliability Issues in Educational Research, Journal of Educational and Social Research. Vol. 2 (2) May 2012. [3] Cronbach, L. J., & Meehl, P. E. (1955) Construct Validity in Psychological Tests, Psychological Bulletin, 52. [4] Berk, R., (1979). Generalizability of Behavioral Observations: A Clarification of Interobserver Agreement and Interobserver Relability. American Journal of Mental Deficiency, Vol. 83, No. 5. [5] George, D., & Mallery, P. (2003). SPSS for Windows step by step: A simple guide and reference. (4th ed.). Boston: Allyn & Bacon.

RESEARCH ANALYSIS AND EVALUATION

37