Health Sciences Center UNM A–Z Directory FastInfo myUNM

• Critical Thinking Home
• What is Critical Thinking?
• Research in Education
• Research in Nursing Practice
• Teaching and Learning Strategies
• Evaluation and Measurement
• Informational Resources

• Applications
• Calendar
• Library
• Facilities
• Faculty & Staff
• Financial Aid
• Technical Support
• UNM Links
• Request Information
• Contact Info
• Map

Evaluation and Measurement

The purpose of evaluating critical thinking is to:

• provide students with feedback on specific strengths and areas needing improvement

• assist faculty in identifying which skills and dispositions to target courses and clinical experiences

• assess specific critical thinking course and clinical objectives

• assess specific course assignments and teaching/learning strategies aimed at developing students’ critical thinking skills and dispositions

• direct the inclusion of critical thinking skills throughout the curricula

• establish local norms for program evaluation

• evaluate critical thinking criteria for program accreditation

Critical thinking is measured by:

• standardized critical thinking tests

• critical thinking indicators or criteria as components of rubrics or grids for use in evaluating discussions, oral presentations, written papers, journals

• Socratic questioning of thinking related to knowledge and actions or practice

• evaluation of use of mind maps or graphic diagrams used to clarify concepts or connections of substantive areas related to nursing

• multimodal means and methods of assessment overtime to provide the most accurate picture of an individual’s critical thinking ability

Types of tools available to measure critical thinking include:

• Watson-Glaser Critical Thinking Appraisal

  • 80 item tool measures general critical thinking ability with 5 subscales: inference, recognition of assumptions, deductions, interpretation, evaluation of arguments, split-half reliability coefficients range from .69-.85 (Bondy, Koenigseder, Ishee, & Williams, 2001)

  • frequently used in nursing education studies, well constructed, but based on narrow scope and content

  • a short form S with 40 updated test items; internal consistency reliability alpha .81 (Geisinger, 1998)

• Ennis-Weir Argumentation Test Level X: An Essay Test of Rational Thinking Ability

  • no norms or strong evidence of reliability. Recommmended as a discussion generator (Rudman, 1985)

• The Cornell Critical Thinking Test

  • general measure of critical thinking ability, considered sound. no national normative data for different levels of test, divides skills intoseparate aspects, lack of attitude or dispositional assessment (Lawrenz & Orton, 1992).

• The California Critical Thinking Skills Test (CCTST)

  • measures 5 core skills: analysis, evaluation, inference, deductive reasoning and inductive reasoning.

  • norm referenced, theory based, alternative forms (internal consistency Kuder-Richardson (KR)20 = .70 form A, .71 form B (Facione, P.A. & Facione, N.C., 1994a)

  • 34 multiple choice short problem statements and scenarios that are discipline neutral

  • questions about ability to assess individual abilities and stability reliability especially in analysis subscale (Bondy, Koenigseder, Ishee, & Williams, 2001

  • concurrent, content, construct validity reported (Facione, P.A. & Facione, N.C., 1994a)

  • strong positive correlation between CCTST scores and

  • grade point average (gpa), NCLEX, SAT scores verbal and math, ACT scores, GRE and its scale scores (Facione, N.C., 1997) \

• The California Critical Thinking Disposition Inventory (CCTDI)

  • developed specifically to measure disposition to use critical thinking as separate from cognitive skills

  • Measures 7 traits: truth-seeking, open-mindedness, analyticity, systematicity, critical thinking self-confidence, inquisitiveness, and maturity

  • Theoretically generated and evolved from empirical analysis process

  • 75 item tool, 6 point Likert-scale questionnaire with “agree-disagree” responses

  • construct validity highly significant correlation with CCTST (r=.66, .67 p<.001) (Facione, P.A. & Facione, N.C., 1992)

  • believed to show acceptable level of reliability, need for more validity evidence (Bondy, Koenigseder, Ishee, & Williams, 2001)

• The Holistic Critical Thinking Rubric

  • Four level rubric incorporates statements of skills (“accurately interprets evidence”) with dispositions (“fair-mindedly follows where evidence and reasons lead”) (Facione, P.A. & Facione, N.C., 1994b)

  • Represents different levels and degrees of critical thinking which support construct and theory of critical thinking used in the CCTST and CCTDI

  • Form was adapted with permission from authors to combine levels into two levels of positive and negative indicators or critical and uncritical thinking (Duphorne, 2000).

Measurement of critical thinking at the College of Nursing includes:

• Use of the California Critical Thinking Skills Test (CCTST) and The California Critical Thinking Disposition Inventory (CCTDI) on entrance of junior students and on exit as seniors from the undergraduate program o The skills and dispositions measured by these tools are represented in a critical thinking framework which is translated into practice by problem assessment, problem solving, decision making and professional judgment and reflected in our undergraduate program.

• Use of critical thinking outcomes for course and clinical objectives related to nursing knowledge and application, and clinical reasoning and professional judgment

  Use of critical thinking criteria or indicators of cognitive skills and dispositions in evaluating assignments, i.e. discussions, journals, concept maps, portfolios, debates, critiques, research papers

  Use of different types of examinations (multiple choice, short answer, essay, case studies) designed to measure nursing knowledge and application through higher levels of critical thinking skills and dispositions

Interpreting the results: What do scores for critical thinking mean

Looking at scores

• CCTST (skills) results for individual students indicate exact scores for total critical thinking skills and the 5 subscale scores for analysis, evaluation, inference, deductive reasoning, and inductive reasoning

• CCTST (skills) total scores range from 1-34 but scores of less than 10 or more than 20 are rare. The modal (most frequent) score for undergraduates in the aggregate study is 16 with a standard deviation of 4.457 and minimum score of 2 and maximum score of 29 (Facione, N.C., 1997). Specific skills are scored individually with scores based on number of items represented by each skill.

• CCTDI (disposition) total scores of over 350 indicate a consistent overall strength in disposition, over 280 an average strength, and less than 280 a serious overall weakness. A specific disposition score of 50 or more indicates a strength, a score of 40 is average and 30 or below is considered a weakness.

• CCDI (disposition) results for individual students indicate exact scores for total critical thinking disposition and the 7 subscale scores for truth-seeking, open-mindedness, inquisitiveness, systematicity, maturity, confidence and analyticity.
  
  Comparing scores with students in similar classes or levels in other nursing programs

• Suggested norms and calculated percentile norms are used for comparing our class and individual student scores with classes in other nursing programs.

• These norms were developed based on aggregate data from 145 predominantly undergraduate samples (Total N= 7,926 cases) from 50 nursing education programs throughout the United States using the same CCTST and CCTDI instruments (Facione, N.C, 1997).

• Comparison of class total mean scores and subscale skills mean scores with a norm for a junior, senior, or BSN-RN nursing class can assist faculty in recognizing areas for increased focus in courses and emphasis throughout the curricula.

• Comparison of individual total mean scores and subscale skills mean scores with class norms can assist students in recognizing areas of strength and areas needing improvement.

Viewing results

Below is a hypothetical example showing an individual’s scores on the skills test (CCTST) and the disposition inventory (CCTDI) and how they compare with the student’s class means norm sample for a similar level class. Looking at the class means compared with norms for that class, it appears that the class has exceeded or is very close in means for total and specific skills. Looking at the individual student’s scores, it appears that the student has met or exceeded the class means in all areas except analysis, evaluation, induction, open-mindedness, and inquisitiveness. Scores for the student indicate above average total critical thinking skills and particularly strong deductive reasoning and very positive across the board critical thinking disposition with a particularly strong disposition for confidence in one’s critical thinking.

How can you develop your critical thinking skills?

Assess your critical thinking skills and dispositions

1. Identify your strengths and areas needing improvement

2. Target those skills and habits you need to develop

3. Focus on learning opportunities to foster those skills

Take advantage of opportunities to practice critical thinking skills

1. “Think aloud” while working through a problem or complex patient

2. situation using clinical reasoning and judgment

3. journal to focus your thinking and reflect on what you have

4. done, why, and what you could do differently

5. develop the art of “questioning” at higher levels of analysis, synthesis and evaluation to increase your knowledge and understanding

6. raise social, policy, financial, ethic issues related to health care and practice to challenge your thinking

7. actively participate in discussions both in class and out to stimulate

8. thinking outside the “box” and “thinking aloud”

9. find evidence and nursing knowledge to support your thinking and actions in practice

Seek out others that are creative and critical thinkers

1. associate with others who look for creative ideas and solutions

2. find teachers, mentors, and coaches who can guide you in this process

3. share your thinking, feelings, and values with others surrounding dilemmas, problems, complex care situations

Check out the descriptions of critical thinking skills and dispositions found in What is critical thinking? and more suggestions for developing your skills in Teaching Learning Strategies

References

Brody, K.N., Koenigseder, L.A , Ishee, J.H. & Williams, B.G. ( 2001). Psychometric properties of the California Critical Thinking Tests. Journal of Nursing Measurement, 9 (3), 309-328.

Duphorne, P.O.(2000). The effect of three computer conferencing designs on critical thinking skills of nursing students at a distance. Ph.D. diss., University of New Mexico, Albuquerque.

Facione, N.C. (1997). Critical thinking assessment in nursing education programs: An aggregate data analysis. Millbrae, CA: California Academic Press.

Facione, P.A., & Facione, N.C. (1992). The California Critical Thinking Disposition Inventory (CCTDI). Test administration manual. Millbrae, CA: California Academic Press.

Facione, P.A. & Facione, N.C. (1994a). The California Critical Thinking Skills Test. CCTST, Form A and Form B—Test Manual. Millbrae, CA: California Academic Press.

Facione, P.A. & Facione, N.C. (1994b). The Holistic Critical Thinking Scoring Rubric. Millbrae, CA: California Academic Press.

Geisinger, K. (1998). The thirteenth mental measurements yearbook. Lincoln, NE: The Burrows Institute of Mental Measurements.

Lawrenz, F. & Orton, R.E. (1992). Cornell Critical Thinking Tests—LevelX and Level A. In D.J. Keyser & R. C. Sweetland, (Eds). Test Critiques Vol.IX (pp.123-131). Austin TX:PRO-ED.

Rudman, H.C. (1985). Review of Ennis-Weir Argumentation Test, Level X: An essay test of rational thinking ability. In J.V. Mitchell, Jr. Ed. The ninth mental measurements yearbook, Vol. I. Lincoln, NE: University of Nebraska Press.