The Information Systems Assessment Examination

Center For Computing Education Research

A Division of the Education Foundation of the Institute for Certification of Computing Professionals
Dr. Herbert E. Longenecker, Jr. Director

To Our Current and Future University Partners: The Center for Computing Education Research has provided a method for direct assessment for programs of Information Systems based on the IS2002 Model Curriculum. This is considered to be a direct measure of the effectiveness of your courses because of the structuring of the IS2002 curriculum, and because of the use of the structure by the CCER.

A map may be constructed between IS2002 learning units and your course outcomes. This result links your University Curriculum to the national model. IS2002 has already been mapped both to ABET Program Objectives, as well as to Skills identified within IS2002 as being entry level skills for employment. Exam Question Objectives have been developed which focus simultaneously on Learning Units (LUs) of IS2002 and Skills. Questions developed form this Skill-LU pairing can be used to estimate Skill attainment, and LU performance. Because the ICCP regards the exam a very thorough review of Skills, I has agreed to Certify exam takers with

the Information Systems Analyst (ISA) CertificateStudents greatly appreciate this product. At the same time, the questions provide a direct measure of LUs and, because of the mappings Program and outcome measures. These are nationally normed outcome measures.

The Instruction-Outcome-Assessment Trinity

Instruction is designed to achieve an outcome. The question is, how do you know if the instruction was successful, rather, how is success measured. The answer is through assessment. The idea is, if you know how you are doing, you can target specific improvements. Opinions can be collected from students, work can be collected, and even employers can be asked how are my students doing. Such measurements are important for assessment. The ICCP has established that direct measures of skills may be obtained using well constructed Scenario-Based questions that are well written as judged by experts. The CCER has adopted this philosophy in developing its exams. Experts must agree that the question actually measures the objectives, and that the measurement is accurate and reasonable. Psychometrically, questions can be neither too easy, or too hard to answer. The questions must be fair yet discriminating. Faculty who have taken the ISA exam describe it as very difficult and totally comprehensive of the Information Systems discipline. Faculty have achieved the highest scores on the ISA exam! Because at least 4 questions are mapped to each of the higher level learning units, the exam, when mapped to university course outcomes, gives a reliable direct measure for each mapped learning unit, and therefore for each University course outcome. Because of the national norming, your performance relative to others can be measured. Faculty have used these scores to isolate areas of weakness, and have shown significant improvement in multi-year serial scores. Exam Scheduling The ISA exam consists of 258 questions. It may be given in a single 3 hour block, or it may be given in two parts of 1- hours. The CCER is currently working to find a psychometrically reliable method to shorten the exam, yet still maintain the same precision of estimating both LU and Skill values. Students must pre-register for the exam several days in advance to give time for verification of the accuracy of student information. At the time of the exam, you must provide a proctor who will enter a secrete password of 3 characters for each student to initiate the exam. The exam is given at regular work-stations. The proctor must check the system-tray to ensure that no other programs are running. This is done to assure the safety of the exam. At the conclusion of the exam, the exam is automatically graded, and students will find out how well they have done. ISA Qualification If the student achieves a score of 50 or more, they have passed the exam, and are immediately qualified to apply for the ISA certification. They must pay the ICCP fee, agree to live by the ICCP code of ethics, and agree to report proof of their computing degree within a year to the

ICCP office. The ICCP offers the students a very reduced fee for this exam of $300. Normally, for a professional certificate from the ICCP, three separate exams would have to be completed each for the same amount. In addition, there is no fee to take the exam. Normally the fee would be charged regardless of passing. Relevance of Certification It has been studied that certified graduates compete for better jobs, make more money, and receive recognition and promotion faster than their peers. In many aspects of the workplace immediate multi-thousand dollar raises are offered just for proof of competency. Anecdotally, many faculty report that their students who hold the ISA certificate are well rewarded. Most faculty encourage their students to achieve this high level of distinction. Students like the certificates: Robert Brown, ISA A few students qualify with scores greater than 70 for the master level of this certificate. Students who have reached this level are excellent, and have done very well in the job market. Becoming Involved with the CCER It is very easy to join the CCER. Send an email request to hlongenecker@usouthal.edu requesting to be involved. Most Deans we have talked with immediately recognize the value of the CCER service, and are willing to pay the $2000 annual fee to have the service. For each student who registers with the ICCP for the ISA certificate, the ICCP has agreed to provide a $70 credit to be applied to your assessment fee to underscore the value the ICCP places on the UniversityCCER relationship. The CCER began as a volunteer effort, but the increasing work-load exceeded the time and emotional load placed on faculty. So, the CCER has taken on professionals to assist with maintenance and development of the infrastructure. There is still room for volunteer effort. The Director, Psychometrics Expert, Question writers and reviewers are faculty who are uncompensated. The Board of the ICCP Education Foundation is uncompensated. Involvement in Research There are many research questions that have come from studies based on work with the CCER. If you are interested in assessment and curriculum research, please speak the CCER Director to explore opportunities. What is expected of a University Participant? The following is expected of all participating universities: 1. You must provide a contact person who will learn the system and work with the CCER staff. The contact person will register other faculty and assign roles.

2. Once students are encouraged to register for an exam, there must be a University member who knows the student information well enough to verify the data entered by the student. 3. Usually the university contact person arranges to recruit exam proctors. These are usually faculty members, who are designated as Procter to the system. They enter a pass word for each student to begin the exam, and monitor to ensure no extra activity takes place that would threaten exam security. 4. The university contact person must schedule the exam within the system, and coordinate students showing up for the exam, as well as pre-registering the students, and having their data entry verified. 5. To utilize the assessment value of the exam, courses must be entered into the system, and then they must be mapped. 6. Mapping is best done by faulty involved with the course in question. Each learning unit is inspected. If it is relevant to the course, it is mapped to the course. Then, an outcome statement is generated for each mapping. This effort takes time, but the effort is well rewarded by mapping the exam results directly to the course. 7. After the exam period elapses reports are made available for review: Exam Results Exam summary Exam taker performance summary IS Model Curriculum Summary: All IS Model Curriculum Summary: Logical Course and LU only Local Course Learning Unit Analysis IS Model Curriculum Skill Analysis IS Core Area Exam Summary IS Curriculum Analysis Skill thread report Curriculum: Summary Curriculum: Local Courses Curriculum: Core Area Summary Curriculum: Core Area By Learning Unit Curriculum: Core Area By Learning Unit By Local Objectives Curriculum: Core Area and Local Courses Summary Curriculum: IS 2002 Summary Curriculum: IS 2002 By Learning Unit Curriculum: IS 2002 By Learning Unit By Local Objectives Curriculum: Local Course Objectives Curriculum: Local Course By IS 2002 Curriculum: Local Course By Core Area Summary Curriculum: Local Course By Core Area Detail Curriculum: Missing LUs by Core Area Curriculum: Missing LUs by IS 2002

The University contact person should get to know the CCER Director. Limited telephone consulting service is available to assist with aspects of the CCER service. It is possible to coordinate more elaborate arrangements, if necessary and desirable.