Authentic Assessment

[Md. Mostafa Rashel]

Guidelines for Writing Standards

GUIDELINE #1:  For a standard to be amenable to assessment, it must be observable and measurable.  For example, a standard such as
"Students will correctly add two-digit numbers"
is observable and measurable.  However, a standard such as
"Students will understand how to add two-digit numbers"
is not observable and measurable.  You cannot observe understanding directly, but you can observe performance.  Thus, standards should include a verb phrase that captures the direct demonstration of what students know and are able to do.

Some bad examples:
Students will develop their persuasive writing skills.
Students will gain an understanding of pinhole cameras.
Rewritten as good examples:
Students will write an effective persuasive essay.
Students will use pinhole cameras to create paper positives and negatives.

GUIDELINE #2: A standard is typically more narrow than a goal and broader than an objective. (See the section on Standards for a fuller discussion of this distinction.)

Too Broad
Of course, the line between goals and standards and objectives will be fuzzy. There is no easy wasy to tell where one begins and another one ends. Similarly, some standards will be broader than others. But, generally, a standard is written too broadly if
•   it cannot be reasonably assessed with just one or two assessments
•   (for content standards) it covers at least half the subject matter of a course or a semester

For example, the Illinois Learning Standards for social science lists "Understand political systems, with an emphasis on the United States" as a goal. That is a goal addressed throughout an entire course, semester or multiple courses. The goal is broken down into six standards including "Understand election processes and responsibilities of citizens." That standard describes what might typically be taught in one section of a course or one unit. Furthermore, I feel I could adequately capture a student's understanding and application of that standard in one or two assessments. However, I do not believe I could get a full and rich sense of a student's grasp of the entire goal without a greater number and variety of classroom measures. On the other hand, the standard, "understand election processes and responsibilities of citizens," would not typically be taught in just one or two lessons, so it is broader than an objective. Hence, it best fits the category of a standard as that term is commonly used.

Another tendency to avoid that can inflate the breadth of a standard and make it more difficult to assess is the coupling of two or more standards in a single statement. This most commonly occurs with the simple use of the conjunction "and." For example, a statement might read
Students will compare and contrast world political systems and analyze the relationships and tensions between different countries.

Although these two competencies are related, each one stands alone as a distinct standard. Additionally, a standard should be assessable by one or two measures. Do I always want to assess these abilities together? I could, but it restricts my options and may not always be appropriate. It would be better to create two standards.

Students will compare and contrast world political systems.
Students will analyze the relationships and tensions between different countries.
In contrast, the use of "and" might be more appropriate in the following standard:
Students will find and evaluate information relevant to the topic.
In this case, the two skills are closely related, often intertwined and often assessed together.

Too Narrow
A possible objective falling under the social science standard mentioned above that a lesson or two might be built around would be "students will be able to describe the evolution of the voter registration process in this country." This statement would typically be too narrow for a standard because, again, it addresses a relatively small portion of the content of election processes and citizen responsibilities, and because it could be meaningfully assessed in one essay question on a test. Of course, you might give the topic more attention in your government course, so what becomes an objective versus a standard can vary. Also, it is important to note that standards written for larger entities such as states or districts tend to be broader in nature than standards written by individual teachers for their classrooms. A U.S. government teacher might identify 5-15 essential ideas and skills for his/her course and voter registration might be one of them.
As you can see, each of these distinctions and labels are judgment calls. It is more important that you apply the labels consistently than that you use a specific label.

Note: You may have noticed that the Illinois Learning Standard that I have been using as an example violates Guideline #1 above -- it uses the verb understand instead of something observable. The Illinois Standards avoids this "problem" in most cases. However, the State addresses it more directly by writing its "benchmark standards" in more observable language. For example, under the general standard "understand election processes and responsibilities of citizens" it states that by early high school (a benchmark) students will be able to "describe the meaning of participatory citizenship (e.g., volunteerism, voting) at all levels of government and society in the United States."

GUIDELINE #3:  A standard should not include mention of the specific task by which students will demonstrate what they know or are able to do.

For example, in a foreign language course students might be asked to
Identify cultural differences and similarities between the student's own culture and the target culture using a Venn diagram.

The statement should have left off the last phrase "using a Venn diagram." Completing a Venn diagram is the task the teacher will use to identify if students meet the standard. How the student demonstrates understanding or application should not be included with what is to be understood or applied. By including the task description in the standard, the educator is restricted to only using that task to measure the standard because that is what the standard requires. But there are obviously other means of assessing the student's ability to compare and contrast cultural features. So, separate the description of the task from the statement of what the student should know or be able to do; do not include a task in a standard.

GUIDELINE #4:  Standards should be written clearly.
GUIDELINE #5: Standards should be written in language that students and parents can understand.

Share your expectations with all constituencies. Students, parents and the community will feel more involved in the process of education. Standards are not typically written in language that early elementary students can always understand, but the standards (your expectations) can be explained to them.


Dr. Mueller

Enduring ..........

[Md. Mostafa Rashel]

Workshop: Writing a Good Standard

In the "workshops" sprinkled throughout this website I will attempt to capture (and model) the process I follow when assisting someone or some group in developing standards or authentic tasks or rubrics. For this workshop, I will begin with an initial draft of a standard and work with an imaginary educator towards a final product. You can "play along at home" by imagining how you would respond to the educator or to me.

Somewhere in the Smoky Mountains .... (hey, it's my workshop; I'll host it where I like!)
Educator: How is this for a standard:
I will teach my students what the main themes of Romeo and Juliet are.
Me: First, standards describe what students should know and do, not what the teacher will do. So, standards typically begin, "Students will ...."

Educator: So, I could change it to Students will know the main themes of Romeo and Juliet.
Me: Yes, that would be a more appropriate way to begin your standard. Standards also should describe observable and measurable behavior on the student's part so that we can assess it. "Knowing" is not something you can directly observe. So, ask yourself "how could they show me they know?"

Educator: Well, I could have them write a paper explaining the main themes. Maybe I could write a standard saying
Students will write a paper explaining the main themes of Romeo and Juliet.
Me: Can you observe "explaining"?
Educator: Yes, I think so.

Me: Yes, so that verb is a good one for a standard. Are there other ways a student could explain the themes to you besides in a paper?
Educator: Sure. They could do it in a speech, or a poster or on an exam.

Me: Good. You don't want to limit yourself in how you might assess this understanding. So, you usually want to avoid including an assignment or task in your standard. Otherwise, you always have to assign a paper to meet that standard.
Educator: I could say
Students will explain the themes of Romeo and Juliet.

Me: Yes, that is observable and clear. It effectively describes the student learning you said you wanted at the beginning. But let's go back to the main question. You always want to ask yourself "why would I want my students to meet this standard?" Why do you want them to be able to explain the themes of Romeo and Juliet?
Educator: Well, I want my students to be able to pick up a piece of literature and be able to tell what the author's main ideas are, and to find some meaning in it for them.

Me: So, you would like them to do that for literature other than Romeo and Juliet as well?
Educator: Yes, we just always teach Romeo and Juliet.
Me: So, you want to identify what really matters to you, what you really want the students to come away with. Typically, that will go beyond one piece of literature or one author. So, you want to write a standard more generically so that you can choose from a variety of literature and still develop the same knowledge and skills in your students.

Educator: I see. That makes sense. I could say
Students will be able to identify themes across a variety of literature.
Me: Very good. But now I am going to be tough on you. I imagine there are some fourth grade teachers who would tell me they have that same standard for their readers. Is the skill of "identifying a theme" really something your ninth and tenth grade students are learning in your classes or do they come to you with that ability?

Educator: Well, they should have it when they get to me, but many of them still can't identify a theme very well. And, now I am asking them to do it with a more sophisticated piece of literature than fourth graders read.
Me: So, it is certainly appropriate that your students continue to review and develop that skill. But would you hope that your students understanding of theme goes beyond simply being able to identify it in a piece?

Educator: Sure. I would like my students to understand the relationship now between theme and character development and plot and setting and how all of those work to shape the piece.
Me: And why does any of that matter? Why should they learn that?
Educator: Well, like I said before, I want them to be able to pick up a play or story and make sense of what the author is trying to communicate so they can make some personal connections to it and hopefully make some more sense of their lives. Also, I hope they realize that literature is another way they can communicate with others. So, by learning the techniques of Shakespeare and others they can learn how to express themselves effectively and creatively. Maybe those should be my standards, making sense of the world and communicating effectively, or are those too broad?

Me: Those are too broad for standards. Those sound like your overall goals for your course. But you could not easily assess such goals in one or two measures. You want to break them down into several standards that capture the key components of your goals and that are amenable to assessment. So, let's go back to your statement about the relationship of theme to the other elements of literature. It's not that being able to identify a theme is a useless skill. But you want your students to go beyond that. How can we frame what you said as a standard?
Educator: How about Students will explain the relationships between theme, character, setting ...
Do I need to list all the literary elements I cover?

Me: You could. Or, if that might change from one year to another you could say something like
Students will explain the relationships between several literary elements (e.g., theme, character, setting, plot) ....
Educator: You can do that in a standard?

Me: Yes, you can do anything you want in writing a standard as long as it captures significant learning you value and is written in a manner that can be assessed.
Educator: But there are some elements, like theme, that I would always want them to understand.

Me: Then you can say "several literary elements including theme, character, setting, and plot ...."
Educator: That's better. So, how about this?

Students will explain relationships between and among literary elements including character, plot, setting, theme, conflict and resolution and their influence on the effectiveness of the literary piece.
Me: Very nice! Is it realistic?

Educator: Yes, I think so.
Me: Is it something worth learning?

Educator: Definitely.
Me: Can you assess it?

Educator: Oh yes, there would be a lot of ways. So.... are we done?
Me: Yes. You have developed an excellent standard.

Educator: That was a lot of work.
Me: Yes. It is not easy to write good standards. But, after you have done a few the rest will come more easily.

Educator: (with a touch of sarcasm) Oh, sure.

Dr. Mueller

Enduring ..........

[Md. Mostafa Rashel]

S

tep 2: Select an Authentic Task
________________________________________

[/b]
Note: Before you begin this section I would recommend you read the section onAuthentic Tasks to learn about characteristics and types of authentic tasks.

 Starting from Scratch: Look at Your Standards
 Starting from Scratch: Look at the Real World
 Workshop: Creating an Authentic Task

If you completed Step 1 (identify your standards) successfully, then the remaining three steps, particularly this one, will be much easier. With each step it is helpful to return to your goals and standards for direction. For example, imagine that one of your standards is
Students will describe the geographic, economic, social and political consequences of the Revolutionary War.

In Step 2, you want to find a way students can demonstrate that they are fully capable of meeting the standard. The language of a well-written standard can spell out what a task should ask students to do to demonstrate their mastery of it. For the above standard it is as simple as saying the task should ask students to describe the geographic, economic, social and political consequences of the Revolutionary War. That might take the form of an analytic paper you assign, a multimedia presentation students develop (individually or collaboratively), a debate they participate in or even an essay question on a test.

"Are those all authentic tasks?"

Yes, because each one a) asks students to construct their own responses and b) replicates meaningful tasks found in the real world.

"Even an essay question on a test? I thought the idea of Authentic Assessment was to get away from tests."

First, authentic assessment does not compete with traditional assessments like tests. Rather, they complement each other. Each typically serves different assessment needs, so a combination of the two is often appropriate. Second, if you read the section on Authentic TasksI mentioned above (and I am beginning to doubt you did :-), then you will recall that essay questions fall near the border between traditional and authentic assessments. Specifically, essay questions are constructed-response items. That is, in response to a prompt, students construct an answer out of old and new knowledge. Since there is no one exact answer to these prompts, students are constructing new knowledge that likely differs slightly or significantly from that constructed by other students. Typically, constructed response prompts are narrowly conceived, delivered at or near the same time a response is expected and are limited in length. However, the fact that students must construct new knowledge means that at least some of their thinking must be revealed. As opposed to selected response items, the teachers gets to look inside the head a little with constructed response answers. Furthermore, explaining or analyzing as one might do in an essay answer replicates a real-world skill one frequently uses. On the other hand, answering a question such as

Which of the following is a geographical consequence of the Revolutionary War?
a.
b.
c.
d.
requires students to select a response, not construct one. And, circling a correct answer is not a significant challenge that workers or citizens commonly face in the real world.
So, yes, it can be that easy to construct an authentic assessment. In fact, you probably recognize that some of your current assessments are authentic or performance-based ones. Moreover, I am guessing that you feel you get a better sense of your students' ability to apply what they have learned through your authentic assessments than from your traditional assessments.


Dr. Mueller

Enduring ..........

[Md. Mostafa Rashel]

Starting from Scratch?: Look at your Standards

What if you do not currently have an authentic assessment for a particular standard? How do you create one from scratch? Again, start with your standard. What does it ask your students to do? A good authentic task would ask them to demonstrate what the standard expects of students. For example, the standard might state that students will solve problems involving fractions using addition, subtraction, multiplication and division.

Teachers commonly ask students to do just that -- solve problems involving fractions. That is an authentic task.

 
Starting from Scratch?: Look at the Real World

But what if you want a more engaging task for your students? A second method of developing an authentic task from scratch is by asking yourself "where would they use these skills in the real world?" For computing with fractions teachers have asked students to follow recipes, order or prepare pizzas, measure and plan the painting or carpeting of a room, etc. Each of these tasks is not just an instructional activity; each can also be an authentic assessment.
See more examples of authentic tasks.

Dr. Mueller

Enduring ..........

[Md. Mostafa Rashel]

Step 3: Identify the Criteria for the Task
_______________________________
 
Examples of Criteria
Characteristics of a Good Criterion
How Many Criteria do you Need for a Task?
Time for a Quiz!
 
Criteria: Indicators of good performance on a task
In Step 1, you identified what you want your students to know and be able to do. In Step 2, you selected a task (or tasks) students would perform or produce to demonstrate that they have met the standard from Step 1. For Step 3, you want to ask "What does good performance on this task look like?" or "How will I know they have done a good job on this task?" In answering those questions you will be identifying the criteria for good performance on that task. You will use those criteria to evaluate how well students completed the task and, thus, how well they have met the standard or standards.

Examples
Example 1: Here is a standard from the Special Education collection of examples:
The student will conduct banking transactions.
The authentic task this teacher assigned to students to assess the standard was to
make deposits, withdrawals or cash checks at a bank.
To identify the criteria for good performance on this task, the teacher asked herself "what would good performance on this task look like?" She came up with seven essential characteristics for successful completion of the task:
•   Selects needed form (deposit, withdrawal)
•   Fills in form with necessary information
•   Endorses check
•   Locates open teller
•   States type of transaction
•   Counts money to be deposited to teller
•   Puts money received in wallet

If students meet these criteria then they have performed well on the task and, thus, have met the standard or, at least, provided some evidence of meeting the standard.

Example 2: This comes from the Mathematics collection. There were six standards addressed to some degree by this authentic task. The standards are: Students will be able to
•   measure quantities using appropriate units, instruments, and methods;
•   setup and solve proportions;
•   develop scale models;
•   estimate amounts and determine levels of accuracy needed;
•   organize materials;
•   explain their thought process.


Rearrange the Room
You want to rearrange the furniture in some room in your house, but your parents do not think it would be a good idea. To help persuade your parents to rearrange the furniture you are going to make a two dimensional scale model of what the room would ultimately look like.

Procedure:
1) You first need to measure the dimensions of the floor space in the room you want to rearrange, including the location and dimensions of all doors and windows. You also need to measure the amount of floor space occupied by each item of furniture in the room. These dimensions should all be explicitly listed.
2) Then use the given proportion to find the scale dimensions of the room and all the items.
3) Next you will make a scale blueprint of the room labeling where all windows and doors are on poster paper.
4) You will also make scale drawings of each piece of furniture on a cardboard sheet of paper, and these models need to be cut out.
5) Then you will arrange the model furniture where you want it on your blueprint, and tape them down.
6) You will finally write a brief explanation of why you believe the furniture should be arranged the way it is in your model.

Your models and explanations will be posted in the room and the class will vote on which setup is the best.

Finally, the criteria which the teacher identified as indicators of good performance on the Rearrange the Room task were:
•   accuracy of calculations;
•   accuracy of measurements on the scale model;
•   labels on the scale model;
•   organization of calculations;
•   neatness of drawings;
•   clear explanations.

(But how well does a student have to perform on each of these criteria to do well on the task? We will address that question in Step 4: Create the Rubric.)

You may have noticed in the second example that some of the standards and some of the criteria sounded quite similar. For example, one standard said students will be able to develop scale models, and two of the criteria were accurary of measurements on the scale model andlabels on the scale model. Is this redundant? No, it means that your criteria are aligned with your standards. You are actually measuring on the task what you said you valued in your standards.


The authentic task used to assess these standards in a geometry class was the following:

Characteristics of a Good Criterion
So, what does a good criterion (singular of criteria) look like? It should be
•   a clearly stated;
•   brief;
•   observable;
•   statement of behavior;
•   written in language students understand.

Additionally, make sure each criterion is distinct. Although the criteria for a single task will understandably be related to one another, there should not be too much overlap between them. Are you really looking for different aspects of performance on the task with the different criteria, or does one criterion simply rephrase another one? For example, the following criteria might be describing the same behavior depending on what you are looking for:
•   interpret the data
•   draw a conclusion from the data

Another overlap occurs when one criterion is actually a subset of another criterion. For example, the first criterion below probably subsumes the second:
•   presenter keeps the audience's attention
•   presenter makes eye contact with the audience

Like standards, criteria should be shared with students before they begin a task so they know the teacher's expectations and have a clearer sense of what good performance should look like. Some teachers go further and involve the students in identifying appropriate criteria for a task. The teacher might ask the students "What characteristics does a good paper have?" or "What should I see in a good scale model?" or "How will I (or anyone) know you have done a good job on this task?"


Dr. Mueller

Enduring ..........

[Md. Mostafa Rashel]

How Many Criteria do you Need for a Task?

Of course, I am not going to give you an easy answer to that question because there is not one. But, I can recommend some guidelines.
• Limit the number of criteria; keep it to the essential elements of the task. This is a guideline, not a rule. On a major, complex task you might choose to have 50 different attributes you are looking for in a good performance. That's fine. But, generally, assessment will be more feasible and meaningful if you focus on the important characteristics of the task. Typically, you will have fewer than 10 criteria for a task, and many times it might be as few as three or four.
• You do not have to assess everything on every task. For example, you might value correct grammar and spelling in all writing assignments, but you do not have to look for those criteria in every assignment. You have made it clear to your students that you expect good grammar and spelling in every piece of writing, but you only check for it in some of them. That way, you are assessing those characteristics in the students' writing and you are sending the message that you value those elements, but you do not take the time of grading them on every assignment.
• Smaller, less significant tasks typically require fewer criteria. For short homework or in-class assignments you might only need a quick check on the students' work. Two or three criteria might be sufficient to judge the understanding or application you were after in that task. Less significant tasks require less precision in your assessment than larger, more comprehensive tasks that are designed to assess significant progress toward multiple standards.

Ask. Ask yourself; you have to apply the criteria. Do they make sense to you? Can you distinguish one from another? Can you envision examples of each? Are they all worth assessing?

Ask your students. Do they make sense to them? Do they understand their relationship to the task?
Do they know how they would use the criteria to begin their work? To check their work?

Ask your colleagues. Ask those who give similar assignments. Ask others who are unfamiliar with the subject matter to get a different perspective if you like.

If you have assigned a certain task before, review previous student work. Do these criteria capture the elements of what you considered good work? Are you missing anything essential?

Dr. Mueller

Enduring ..........

[Md. Mostafa Rashel]

Time for a Quiz!
Do you think you could write a good criterion now? Do you think you would know a good one when you saw one? Let's give you a couple small tasks:

Task 1: Write three criteria for a good employee at a fast-food restaurant. (There would likely be more than three, but as a simple check I do not need to ask for more than three. Assessments should be meaningful and manageable!)

Task 2: I have written three criteria for a good employee below. I intentionally wrote two clear criteria (I hope) and one vague one. Can you find the vague one among the three? Are the other two good criteria? (Yes, I wrote them so of course I think they are good criteria. But I will let you challenge my authority just this once :-)
•   the employee is courteous
•   the employee arrives on time
•   the employee follows the sanitary guidelines
What do you think? In my opinion, the first criterion is vague and the latter two are good criteria. Of course, evaluating criteria is a subjective process, particularly for those you wrote yourself. So, before I explain my rationale I would reiterate the advice above of checking your criteria with others to get another opinion.

To me, the statement "the employee is courteous" is too vague. Courteous could mean a lot of different things and could mean very different things to different people. I would think the employer would want to define the behavior more specifically and with more clearly observable language. For example, an employer might prefer:
•   the employee greets customers in a friendly manner

That is a more observable statement, but is that all there is to being courteous? It depends on what you want. If that is what the employer means by courteous then that is sufficient. Or, the employer might prefer:
•   the employee greets customers in a friendly manner and promptly and pleasantly responds to their requests

"Is that one or two criteria?" It depends on how detailed you want to be. If the employer wants a more detailed set of criteria he/she can spell out each behavior as a separate criterion. Or, he/she might want to keep "courteous" as a single characteristic to look for but define it as two behaviors in the criterion. There is a great deal of flexibility in the number and specificity of criteria. There are few hard and fast rules in any aspect of assessment development. You need to make sure the assessment fits your needs. An employer who wants a quick and dirty check on behavior will create a much different set of criteria than one who wants a detailed record.

The second criterion above, the employee arrives on time, is sufficiently clear. It cannot obviously name a specific time for arriving because that will change. But if the employer has identified the specific time that an employee should arrive then "arrive on time" is very clear. Similarly, if the employer has made clear the sanitary guidelines, then it should be clear to the employees what it means to "follow the guidelines."

"Could I include some of that additional detail in my criteria or would it be too wordy?" That is up to you. However, criteria are more communicable and manageable if they are brief. The employer could include some of the definition of courteous in the criterion statement such as
•   the employee is courteous (i.e., the employee greets customers in a friendly manner andpromptly and pleasantly responds to their requests)

However, it is easier to state the criterion as "the employee is courteous" while explaining to the employees exactly what behaviors that entails. Whenever the employer wants to talk about this criterion with his/her employees he can do it more simply with this brief statement. We will also see how rubrics are more manageable (coming up in Step 4) if the criteria are brief.

"Can I have sub-criteria in which I break a criterion into several parts and assess each part separately?" Yes, although that might be a matter of semantics. Each "sub-criterion" could be called a separate criterion. But I will talk about how to handle that in the next section "Step 4: Create the Rubric."



Dr. Mueller

Enduring ..........

[Md. Mostafa Rashel]

Step 4: Create the Rubric
________________________________________
 Creating an Analytic Rubric
 Creating a Holistic Rubric
 Final Step: Checking Your Rubric
 Workshop: Writing a Good Rubric
 
Note: Before you begin this section I would recommend that you read the section on Rubrics to learn about the characteristics of a good rubric.

In Step 1 of creating an authentic assessment, you identified what you wanted your students to know and be able to do -- your standards.

In Step 2, you asked how students could demonstrate that they had met your standards. As a result, you developed authentic tasks they could perform.

In Step 3, you identified the characteristics of good performance on the authentic task -- the criteria.
Now, in Step 4, you will finish creating the authentic assessment by constructing a rubric to measure student performance on the task. To build the rubric, you will begin with the set of criteria you identified in Step 3. As mentioned before, keep the number of criteria manageable. You do not have to look for everything on every assessment.

Once you have identified the criteria you want to look for as indicators of good performance, you next decide whether to consider the criteria analytically or holistically. (See Rubrics for a description of these two types of rubrics.)


Dr. Mueller

Enduring ..........

[shamsi]

Dear Sir:

Thanks for introducing such a practical assessment system.The examples you mentioned-I found it very much helpful to use in my classes.

Regards

Shamsi

[Antara11]

Dear sir,

It's really helpful for every teacher to assess their students.

Regards,

[Md. Mostafa Rashel]

Thanks for reply and study. I hope it will work more in the class room assessment

[Antara11]

Some of these are really helpful to know. Thanks, sir.

[sushmita]

Well done Boss. Is that talking about Rubric? Or the Rubric is located in another page.

[Tahsina]

This is indeed an area getting expanded more... I am currently working on Performance Based Learning which is very close to authentic assessment. Can you please mention the specific link/source? I just found 'Dr Mueller'...

[sushmita]

Thank you Sir for discussing about Rubric.