Overcoming Assessment Illiteracy
There are some who equate illiteracy with ignorance. But, there is a difference. Ignorance is a general lack of knowledge. Each of us is ignorant in our own way. One person may be unfamiliar with the rules of a game, another unclear about the difference between ultraviolet and infrared radiation, or unable to distinguish valid from reliable research.
In general, illiteracy is the inability to read. People may also be functionally illiterate meaning that their reading and writing skills are inadequate for completing the daily routines of life and work; just as some can be color blind and others have color deficiency where they see colors differently. When someone has a red-green vision deficiency, they can learn the sequence of traffic lights and then determine whether the top or bottom light is shining brighter. There are also some people who are color sufficient yet are unable to correctly identify which of these traffic light patterns is right side up.
EACH OF US AS IS, MAL-, MIS-, DYS/DIS- OR ILL-, SOMETHING OR OTHER.
What’s your mal-? Are you mal-content when things do go as expected? Do you mal-function when overwhelmed with educational mandates or are mal-adaptive when too many changes take place at once.
What’s your mis-? Are you easy to mid-lead, do you have students who seem to be mis-guided in conforming to norms? Who hasn’t been mis-taken or taken mis-steps either physically or socially?
What’s your dis-? Are you dis-tracted when there are too many demands on your time and energy? Do you feel dis-comfort when things don’t seem to be as expected, or dis-quiet when you head in for your annual review?
These situations require responsive guidance that is appropriate and practical. Overcoming any types of ill-, mal-, dis, or dys- requires insight into the situation as well as the recognition that it may take longer to restore or repair than it took to develop the problem.
COMPARING ASSESSMENT ILLITERACY to ASSESSMENT LITERACY
Whether it is called ignorance or illiteracy, the lack of knowledge about assessment can be challenging for teachers, harmful to students, and result in inaccurate inferences about learning. Consider Kai, who plays music by ear, but can’t make sense of mathematical ratios and intervals. Or Izzy who speaks clearly and eloquently when explaining a concept to a peer but struggles to put the same words on paper.
Assessment Illiteracy refers to a broad range of skills and knowledge that can be confusing or elusive. In addition, external decision makers demand accountability in the form of standardized test scores. There’s no denying that the vocabulary of measurement and statistics can be confusing such as norm vs. criterion or correlation and causation. In practice, illiteracy is evident when students are given only requisite and authorized tests, when the majority of classroom measures are based on selected choice questions, or when reporting is summative, relying primarily on final scores.
It is possible to flip these practices by embedding assessment throughout teaching and learning, engaging students as assessors, facilitating student accountability for learning outcomes, emphasizing growth measures, and accentuating progress.
Assessment Literate teachers and leaders understand the role of assessment in a learning culture, can design and utilize multiple types of assessment, and effectively make use of assessment results. Assessment literate educators have the skills to:
- Design and select valid assessment instruments.
- Align assessment with standards and learning intentions.
- Utilize multiple methods to assess diverse levels of complexity for multiple purposes.
- Monitor student progress toward learning targets.
- Interpret assessment data in relation to strengths, gaps, growth, and outcomes.
- Use reliable and consistent assessment results to inform decisions about individual students, teaching routines, instructional planning, and curriculum design.
- Communicate assessment outcomes to students, parents, and other constituents.
- Recognize and minimize detrimental assessment practices and inappropriate uses of assessment information.
Assessment illiteracy is no longer acceptable anywhere in our education system: From legislators who mandate frequent testing of students to teachers and students who are overwhelmed by it. Assessment literacy requires a collective voice in assessment practices that comes from a widespread understanding of comprehensive assessment as a multidimensional process that is routinely integrated throughout teaching and learning. When literacy is evident, assessments increase opportunities for students and support meaningful accountability.
In brief, best practice in assessment is:
- Purposeful, informative, and coherent.
- Intentional, balanced, and practical.
- Focused yet flexible.
- Rigorous and responsive.
- Cogent, credible, and technically sound.
- Inclusive and accessible for all learners.
For students to be successful the assessment content and process meet these criteria:
A. Wording is clear and understandable for all learners.
B. esponses align with and display evidence of progress towards mastery.
C. There may be more than one single right answer.
D. Students have opportunities to elaborate and explain their answers.
E. Alternative responses are considered in relation to learning intentions.
F. Helps students to apply and transfer learning.
G. Assessment results are visible, explicit and useful in improving learning.
PUTTING INTO PRACTICE
Scenarios: Questions 1 to 4 are given to students in a college class on assessing student learning. Analyze each question in relation to the criteria for constructing assessment questions. (You may rely on the indicators of best practice in 1 to 6 or the student success indicators in A to G.) Explain your analysis in your own words. Make recommendations for improvement.
- Describe the relative merits of selected choice questions and essay questions for measuring learning outcomes at the understanding level of the taxonomy.
Sample Response to question 1:
“I can see how B is evident in that the student’s depth and clarity of their response can provide insight into their learning. C and D, are also incorporated in the task by asking students to “describe” rather than simply select a response. A is questionable because “relative” can have relatively different meanings.
I do not see F because the questions and response are constrained by the word “Describe,” which is typically at the understanding level of the taxonomy. I think the question could be clearer, (per best practice #5 and A) but I’m not sure how to do that.”
- T F The true-false item, which is favored by test experts, may also be called an alternative-response item.
- _________What is a six-sided polygon known as? (The student says it is a stop sign)
- Which 2 people discovered the structure of DNA. Write the correct answers here: ________
A. James Watson B. Rosalind Franklin C. Sheldon Cooper D. Francis Crick E. A & B F. A & C
For questions 5 and 6 below, discuss your responses and analysis with your team.
- Choose and explain which is a more valid assessment strategy.
A. Ms. Tau chooses test questions from the teacher’s version of the classroom text
B. Ms. See uses a test blueprint to be sure that questions align with each of the learning intentions: Learn more about blueprints and table of specifications at ODE and PARE.
- Which strategy helps students stretch learning beyond recall?
A. Students write selected choice questions for an upcoming test. These may include true/false, matching, or multiple choice.
B. Student uses their new learning to write a letter to someone in a position of power, for example, a school leader, expert on the topic, or representative in public office. The letter is scored for connections to learning intentions such as accuracy, clarity, use of facts, organization, and persuasiveness.
Questions: Contact me at https://www.assessmentnetwork.net