Assessment & Learning

Assessment Handbook

Principles of Assessment

• Assessment links to student learning objectives
• Test methods and items are developmentally appropriate for the learner.
• Assessment is integrated into the student’s daily learning experience.
• Self reflection and self assessment are continuously reinforced to promote life long learning.
• Frequent formative assessments are provided to facilitate learning.
• High-stakes summative examinations are milestones for students to demonstrate learning occur less infrequently.
• Grading standards are criterion referenced and are established prior to the examination.
• Grading processes are transparent and consistent across the curriculum.
• Multiple choice questions adhere to USMLE formats.
• Examinations are criterion referred competency-based and reflect all aspects of the curriculum (e.g., medical knowledge, clinical skills (history and physical examination), communication skills, and ethics & professionalism.
• Assessment score and grade reporting is timely.
• Innovations in assessment reflect developments in educational theory and research.

Value of Assessments

"The evaluation of student achievement must employ a variety of measures of knowledge, competence and performance, systematically and sequentially applied throughout medical school. There should be specific attention to the development of problem-solving and clinical reasoning abilities." Liaison Committee on Medical Education (LCME) Accreditation Standards, June 2002.

Educational assessments (e.g., examinations, tests) are tools for learning as well as measures learning. Effective examinations assess students’ grasp of the learning objectives. Ideally, assessments prepare students for life long self assessment and self directed learning. Assessment help guide faculty teaching efforts and inform curricular change and development. UNM/SOM assessments assure future patients that our School of Medicine graduates are qualified to continue the next steps in their medical training.

The University of New Mexico School of Medicine has developed an integrated series of examinations. These are given frequently enough that students and faculty can measure progress but not so frequently that short-term memorization of details is inadvertently reinforced. The exams are closely related to recent curricular content. They are also designed to pose problems that integrate previously learned material and to encourage review, knowledge retention, concept formation, and synthetic thinking. Examinations also test various clinical skills, professional conduct, and ethical behaviors essential to effective clinical practice.

Professional Conduct and Test Ethics

The Honor Code you signed when you entered medical school describes UNM/SOM expectations which you have committed to uphold. Behaviors or actions, which undermine or attempt to subvert the examination process for oneself or others, are unacceptable.

•    Integrity                        •    Discipline                       •    Effort
•    Honesty                        •    Professionalism
•    Diligence

Examples of unprofessional and unacceptable behaviors include :

• Before an Examination
  1. Unauthorized access to examination materials prior to test administration.
  2. Unauthorized entry into the area where test materials are prepared or maintained.
  3. Unauthorized copying, reproducing and/or disseminating of test materials including old test materials which were not made officially available.
• During an Examination
  1. Sharing information about any test materials with anyone other than faculty or test administrators. (This includes content from standardized patient or paper cases.)
  2. Possessing or utilizing unauthorized materials (in written or electronic form)
  3. Possessing or utilizing unauthorized equipment during an examination (for example: recording devices, cell phones, cameras)
  4. Leaving the testing area without authorization.
  5. Disturbing other students, standardized patients, or staff. Talking must be kept to a quiet minimum in hallways near testing rooms. Pagers and cell phones must be turned off.
  6. All written notes made during the examination, must be deposited in the designated area before moving forward to the next testing activity.
  7. Unauthorized copying, reproducing and or disseminating test materials.
• After an Examination
  1. Unauthorized sharing of information about any of the test materials including simulated or paper cases.
  2. Altering or misrepresenting examination scores.
  3. Unauthorized copying, reproducing and/or disseminating test materials.

Any irregular behavior will be investigated and evaluated for breeches in ethical conduct.

Types of Assessments

Written examinations tests cognitive knowledge

• Multiple choice questions

Examples: Single-best answer and extended matching
• Free response questions

Examples: Short answer and essay questions

Performance examinations tests knowledge application

• Laboratory practical
• Skills station
• O.S.C.E. (Objective Structured Clinical Examination) The student demonstrates his or her clinical skills in a simulated clinical encounter with a standardized patient.
• History taking, physical examination, and communication skills are assessed in the context of the standardized patient encounter.

Formative Assessments

Purpose: to provide feedback to the teacher and student about the learning progress. These “diagnostics” help faculty adjust course content to the needs of the learners. A formal grade is not recorded; however, this is an opportunity for students to gauge the depth of their knowledge and target areas for more effective study. Formative assessments are encouraged in every learning setting and are frequent, sometimes less formal assessments.

Different learning settings use different formative assessment formats. Some examples of formative assessment formats include:

1. In class, I-clicker questions about lecture content;
2. Weekly online quizzes where the score is for your information; and,
3. Coaching after an Attending has observed you evaluate a patient.

Formative assessment prepares you to perform successfully on summative assessments.

Summative Assessments

Purpose:  to examine cumulative knowledge in a particular domain. The scores from these examinations are recorded, reported to the registrar, and ultimately appear on your transcript. Like formative examinations, summative assessments are administered in a variety of formats.

Standardized National Exams

Comprehensive Basic Science Examination (CBSE)

USMLE Step 1: Practice Test
The Comprehensive Basic Science Exam (CBSE) is a practice exam for USMLE Step 1. It is administered during Phase I. This exam consists of 200 questions which you have 4 hours to complete.

Comprehensive Clinical Science Examination (CCSE)

USMLE Step 2: Practice Test
The Comprehensive Clinical Science Exam is a practice test for USMLE Step 2 Clinical Knowledge (CK). It is administered toward the end of Phase II.

Clinical Subject Examinations “Shelf boards”
Shelf boards are standardized, discipline-based exams that assess the 7 clinical clerkship areas: Neurology, Family Medicine, Internal Medicine, Obstetrics and Gynecology, Pediatrics, Psychiatry and Surgery. Brief content outlines are available from National Board of Medical Examiners.

United States Medical Licensing Examinations (USMLE)

Step 1: Basic science content.

Step 2 CK: Integrated clinical content (Best scheduled after Phase 2 clerkship rotations and OSCEs have been completed).

Step 2 CS: Clinical Skills performance assessment consisting of 12 standardized patient cases. Students are assessed on their history taking, physical examination, patient communication skills, and clinical note writing. This examination is conducted at a national testing center and is best scheduled after Phase 2 clerkship rotations and OSCEs have been completed.

UNM/SOM Assessment Schedule

2013 Curricular Map

2012 Curricular Map

2011 Curricular Map

2010 Curricular Map

How to Prepare for a Test

• Review all orientation materials which are available on the web or which have been presented in class.
• Pick up your test code from the Assessment & Learning office located on the first floor of the HSC library in room 116, 272-8028.
• Prepare your diagnostic kit (stethoscope, sphygmomanometer, ophthalmoscope, otoscope, and reflex hammer). Make sure batteries are charged and working.
• Wear your badge, white coat, and neat, professional dress for clinical skills encounters.
• Beepers, cell phones, Blackberries, and other communication devices must be turned off.
• Tests begin and end on time. Be prompt.
• Get a good night’s sleep.

Competencies

The six UNM competencies defined by the SOM education council are:

1. Medical Knowledge
2. Patient Care
3. Practice Based Learning and Improvement
4. Ethics and Professionalism
5. Interpersonal and Communication Skills
6. Systems and Community Based Practice

Patient Care

Clinical skills are assessed in standardized patient interactions that involve history taking, physical exam, and clinical note writing. Cases are based on common clinical problems. Thoughtful evaluation, with adequate breadth and depth to formulate a competent patient assessment and plan of care are expected.

Ethics and Professionalism

This competency is intrinsic to sound clinical practice, and is assessed in a developmentally attuned manner using a variety of methods such as videotaped vignettes, standardized patient encounters, essays, short-answer tests, and multiple-choice questions. Exams may include core definitions of ethics principles. Clinical ethics decision-making skills and attunement to attitudinal and values issues in medicine are tested systematically with respect to a variety of content areas.

Interpersonal and Communication Skills

Communication skills are assessed during standardized patient encounters and through various written exercises. The standardized patients have extensive training in evaluating your performance using the essential elements of communication Global Ratings Scale. This is a behaviorally anchored global rating scale from which your score is derived. The interactions with the SPs are videotaped for student to review and learn from. Videos are available for you to view after grades are released.

 

Testing Hints

Standardized Patient Encounters at UNM/SOM

Demonstrate best practice clinical performance

• Review your Foundations of Clinical Practice syllabus and practice your clinical and communication skills on each other, family members, and friends.
• You will do your best if you respond as if these stations are actual clinical situations.
• Read the instruction on the door sign carefully.
• Remember the value of open-ended questions when you conduct your interview.
• Treat the standardized patient with same courtesy and respect you would any patient.
• You may have seen the SP in different patient roles. Respond to the current role of the standardized patient.
• Stay in role.
• Wash your hands: All the exam rooms have either sinks or waterless sanitizers. In this clinic-like test setting it is best to wash your hands immediately before beginning the physical examination and again at the end of the physical examination.

Test Accommodations

Contact OASIS (Office of Academic Support & Information Services), 272-5192, or, Cheri Koinis, Student Success Manager, 272-5042.

Appeals

If you think there is an error in your performance assessment score, you have 5 business days to appeal the grade after exam results are posted.  You must comply with the following procedure.

1. Contact Assessment & Learning, 272-8129, to schedule a time to view the recording of your standardized patient encounter.
2. Review all the exam documents related to the encounter. These will be in your file in Assessment & Learning.
3. If after viewing your encounter and reviewing your file, you still think your performance was scored in erroneously, you may request a review of your grade.
4. Submit an email request to review your score to Paul Perea, Data Services Manager, Assessment & Learning. He will coordinate your request.
5. Specify part of the score you want reviewed.
6. Keep in mind that sometimes errors in scoring favor the student, but, sometimes errors do not favor the student. The entire encounter will be rescored by standardized patient trainers and faculty, and any revisions to the score (either positively or negatively) will be recorded as your final grade for the encounter.

Remediation

Learning Prescription

A learning prescription is a statement of educational intent. It serves as an action plan to help you focus your efforts, toward your learning goals.

Learning objectives are best stated by defining what you will have learned when you complete your plan. They should include areas for growth identified from tutorials, examinations, standardized patient encounters, independent review, clinical experiences, feedback from faculty, as well as areas in which you seek new knowledge.

Format

1. Statement: State your personal learning objectives. Be specific.
2. Plan: Describe the concrete steps you will take to accomplish your objectives. Delineate the specific learning methods you will use. Make sure they are feasible. Develop a timeline to keep yourself on track. Know when you plan to accomplish your steps. Be realistic.
3. Outcome: Describe how you will know that you met your objectives.

Frequently Asked Questions

What if I’m really sick (e.g. high fever and can’t keep anything down) or have a family emergency (e.g. serious illness or death) at the time I’m supposed to take a final exam?
We know you want to be able to perform at your best. Pencil and paper exams are relatively flexible and can be made up when you are feeling better. Some laboratory exams and standardized patient performance examinations are very difficult to reschedule. It is possible that you will need to wait until a particular performance exam is administered again to make up your excused absence. If you are uncertain about what to do, please call so we can help you problem solve. If you are unable to be present for an examination, you should notify the Office of Education at 272-4823 and Assessment & Learning at 272-8028 prior to the start of the exam.

What if it snows?
On days when there is bad weather, call UNM's Snow Hotline at 277-SNOW or check their website. Phase I, II, and III students who are not on clinical rotations should follow the instructions for main campus students, e.g. if UNM is on a two hour delay. Phase II and III students who are on clinical rotations should check with their clinical team as they may be expected to follow their usual schedule regardless of weather.

May I expect to receive feedback from the standardized patient?
Sometimes yes and sometimes no. While standardized patients may give feedback in teaching settings, do not ask a standardized patient for verbal feedback during or after an examination. When feedback is formally scheduled you will receive directions. Standardized patients often write comments about your test performance on the checklists. These comments can be very helpful and are available for you to review once the exams are graded and filed.

How do I introduce myself to a standardized patient at the beginning of a clinical skills test station?
Introduce yourself by your full name and title. For example, “Hello, my name is Michelle Smith. I’m a 2^nd year medical student.”