Authentic Assessment

[Md. Mostafa Rashel]

Descriptors

The above rubric includes another common, but not a necessary, component of rubrics --descriptors. Descriptors spell out what is expected of students at each level of performance for each criterion. In the above example, "lots of historical inaccuracies," "can tell with difficulty where information came from" and "all relevant information is included" are descriptors. A descriptor tells students more precisely what performance looks like at each level and how their work may be distinguished from the work of others for each criterion. Similarly, the descriptors help the teacher more precisely and consistently distinguish between student work.

Many rubrics do not contain descriptors, just the criteria and labels for the different levels of performance. For example, imagine we strip the rubric above of its descriptors and put in labels for each level instead. Here is how it would look:

Criteria                                  Poor (1)             Good (2)                        Excellent (3)

Number of Sources        x1           
Historical Accuracy        x3           
Organization                x1           
Bibliography                x1   
        
It is not easy to write good descriptors for each level and each criterion. So, when you first construct and use a rubric you might not include descriptors. That is okay. You might just include the criteria and some type of labels for the levels of performance as in the table above. Once you have used the rubric and identified student work that fits into each level it will become easier to articulate what you mean by "good" or "excellent." Thus, you might add or expand upon descriptors the next time you use the rubric.

Dr. Mueller

Enduring ..........

[Md. Mostafa Rashel]

Why Include Levels of Performance?

Clearer expectations
As mentioned in Step 3, it is very useful for the students and the teacher if the criteria are identified and communicated prior to completion of the task. Students know what is expected of them and teachers know what to look for in student performance. Similarly, students better understand what good (or bad) performance on a task looks like if levels of performance are identified, particularly if descriptors for each level are included.

More consistent and objective assessment
In addition to better communicating teacher expectations, levels of performance permit the teacher to more consistently and objectively distinguish between good and bad performance, or between superior, mediocre and poor performance, when evaluating student work.

Better feedback
Furthermore, identifying specific levels of student performance allows the teacher to provide more detailed feedback to students. The teacher and the students can more clearly recognize areas that need improvement.

Dr. Mueller

Enduring ..........

[Md. Mostafa Rashel]

Analytic Versus Holistic Rubrics

For a particular task you assign students, do you want to be able to assess how well the students perform on each criterion, or do you want to get a more global picture of the students' performance on the entire task? The answer to that question is likely to determine the type of rubric you choose to create or use: Analytic or holistic.

Analytic rubric
Most rubrics, like the Research rubric above, are analytic rubrics. An analytic rubric articulates levels of performance for each criterion so the teacher can assess student performance on each criterion. Using the Research rubric, a teacher could assess whether a student has done a poor, good or excellent job of "organization" and distinguish that from how well the student did on "historical accuracy."

Holistic rubric
In contrast, a holistic rubric does not list separate levels of performance for each criterion. Instead, a holistic rubric assigns a level of performance by assessing performance across multiple criteria as a whole. For example, the analytic research rubric above can be turned into a holistic rubric:

3 - Excellent Researcher
•   included 10-12 sources
•   no apparent historical inaccuracies
•   can easily tell which sources information was drawn from
•   all relevant information is included

2 - Good Researcher
•   included 5-9 sources
•   few historical inaccuracies
•   can tell with difficulty where information came from
•   bibliography contains most relevant information

1 - Poor Researcher
•   included 1-4 sources
•   lots of historical inaccuracies
•   cannot tell from which source information came
•   bibliography contains very little information

In the analytic version of this rubric, 1, 2 or 3 points is awarded for the number of sources the student included. In contrast, number of sources is considered along with historical accuracy and the other criteria in the use of a holistic rubric to arrive at a more global (or holistic) impression of the student work. Another example of a holistic rubric is the "Holistic Critical Thinking Scoring Rubric" (in PDF) developed by Facione & Facione.

When to choose an analytic rubric
Analytic rubrics are more common because teachers typically want to assess each criterion separately, particularly for assignments that involve a larger number of criteria. It becomes more and more difficult to assign a level of performance in a holistic rubric as the number of criteria increases. For example, what level would you assign a student on the holistic research rubric above if the student included 12 sources, had lots of inaccuracies, did not make it clear from which source information came, and whose bibliography contained most relevant information? As student performance increasingly varies across criteria it becomes more difficult to assign an appropriate holistic category to the performance. Additionally, an analytic rubric better handles weighting of criteria. How would you treat "historical accuracy" as more important a criterion in the holistic rubric? It is not easy. But the analytic rubric handles it well by using a simple multiplier for each criterion.
When to choose a holistic rubric

So, when might you use a holistic rubric? Holistic rubrics tend to be used when a quick or gross judgment needs to be made. If the assessment is a minor one, such as a brief homework assignment, it may be sufficient to apply a holistic judgment (e.g., check, check-plus, or no-check) to quickly review student work. But holistic rubrics can also be employed for more substantial assignments. On some tasks it is not easy to evaluate performance on one criterion independently of performance on a different criterion. For example, many writing rubrics (see example) are holistic because it is not always easy to disentangle clarity from organization or content from presentation. So, some educators believe a holistic or global assessment of student performance better captures student ability on certain tasks. (Alternatively, if two criteria are nearly inseparable, the combination of the two can be treated as a single criterion in an analytic rubric.)

Dr. Mueller

Enduring ..........

[Md. Mostafa Rashel]

How Many Levels of Performance Should I Include in my Rubric?

There is no specific number of levels a rubric should or should not possess. It will vary depending on the task and your needs. A rubric can have as few as two levels of performance (e.g., a checklist) or as many as ... well, as many as you decide is appropriate. (Some do not consider a checklist a rubric because it only has two levels -- a criterion was met or it wasn't. But because a checklist does contain criteria and at least two levels of performance, I include it under the category of rubrics.) Also, it is not true that there must be an even number or an odd number of levels. Again, that will depend on the situation.

To further consider how many levels of performance should be included in a rubric, I will separately address analytic and holistic rubrics.

Analytic rubrics
Generally, it is better to start with a smaller number of levels of performance for a criterion and then expand if necessary. Making distinctions in student performance across two or three broad categories is difficult enough. As the number of levels increases, and those judgments become finer and finer, the likelihood of error increases.

Thus, start small. For example, in an oral presentation rubric, amount of eye contact might be an important criterion. Performance on that criterion could be judged along three levels of performance: never, sometimes, always.

             makes eye contact with audience            never                  sometimes           always

Although these three levels may not capture all the variation in student performance on the criterion, it may be sufficient discrimination for your purposes. Or, at the least, it is a place to start. Upon applying the three levels of performance, you might discover that you can effectively group your students' performance in these three categories. Furthermore, you might discover that the labels of never, sometimes and always sufficiently communicates to your students the degree to which they can improve on making eye contact.

On the other hand, after applying the rubric you might discover that you cannot effectively discriminate among student performance with just three levels of performance. Perhaps, in your view, many students fall in between never and sometimes, or between sometimes and always, and neither label accurately captures their performance. So, at this point, you may decide to expand the number of levels of performance to include never, rarely, sometimes, usually and always.

                makes eye contact          never   rarely   sometimes   usually   always

There is no "right" answer as to how many levels of performance there should be for a criterion in an analytic rubric; that will depend on the nature of the task assigned, the criteria being evaluated, the students involved and your purposes and preferences. For example, another teacher might decide to leave off the "always" level in the above rubric because "usually" is as much as normally can be expected or even wanted in some instances. Thus, the "makes eye contact" portion of the rubric for that teacher might be

                  makes eye contact   never   rarely   sometimes   usually

So, I recommend that you begin with a small number of levels of performance for each criterion, apply the rubric one or more times, and then re-examine the number of levels that best serve your needs. I believe starting small and expanding if necessary is preferable to starting with a larger number of levels and shrinking the number because rubrics with fewer levels of performance are normally
•   easier and quicker to administer
•   easier to explain to students (and others)
•   easier to expand than larger rubrics are to shrink

The fact that rubrics can be modified and can reasonably vary from teacher to teacher again illustrates that rubrics are flexible tools to be shaped to your purposes. To read more about the decisions involved in developing a rubric, see the chapter entitled, "Step 4: Create the Rubric."
Holistic rubrics

Much of the advice offered above for analytic rubrics applies to holistic rubrics as well. Start with a small number of categories, particularly since holistic rubrics often are used for quick judgments on smaller tasks such as homework assignments. For example, you might limit your broad judgments to
•   satisfactory
•   unsatisfactory
•   not attempted
or
•   check-plus
•   check
•   no check
or even just
•   satisfactory (check)
•   unsatisfactory (no check)

Of course, to aid students in understanding what you mean by "satisfactory" or "unsatisfactory" you would want to include descriptors explaining what satisfactory performance on the task looks like.

Even with more elaborate holistic rubrics for more complex tasks I recommend that you begin with a small number of levels of performance. Once you have applied the rubric you can better judge if you need to expand the levels to more effectively capture and communicate variation in student performance.

To read more about the decisions involved in developing rubrics, see

Dr. Mueller

Enduring ..........

[Md. Mostafa Rashel]

Descriptors
The above rubric includes another common, but not a necessary, component of rubrics --descriptors. Descriptors spell out what is expected of students at each level of performance for each criterion. In the above example, "lots of historical inaccuracies," "can tell with difficulty where information came from" and "all relevant information is included" are descriptors. A descriptor tells students more precisely what performance looks like at each level and how their work may be distinguished from the work of others for each criterion. Similarly, the descriptors help the teacher more precisely and consistently distinguish between student work.

Many rubrics do not contain descriptors, just the criteria and labels for the different levels of performance. For example, imagine we strip the rubric above of its descriptors and put in labels for each level instead. Here is how it would look:

Criteria                                       Poor (1)                  Good (2)                 Excellent (3)
Number of Sources              x1           
Historical Accuracy               x3           
Organization                        x1           
Bibliography                        x1           


It is not easy to write good descriptors for each level and each criterion. So, when you first construct and use a rubric you might not include descriptors. That is okay. You might just include the criteria and some type of labels for the levels of performance as in the table above. Once you have used the rubric and identified student work that fits into each level it will become easier to articulate what you mean by "good" or "excellent." Thus, you might add or expand upon descriptors the next time you use the rubric.

Dr. Mueller

Enduring ..........

[Md. Mostafa Rashel]

Why Include Levels of Performance?

Clearer expectations
As mentioned in Step 3, it is very useful for the students and the teacher if the criteria are identified and communicated prior to completion of the task. Students know what is expected of them and teachers know what to look for in student performance. Similarly, students better understand what good (or bad) performance on a task looks like if levels of performance are identified, particularly if descriptors for each level are included.

More consistent and objective assessment
In addition to better communicating teacher expectations, levels of performance permit the teacher to more consistently and objectively distinguish between good and bad performance, or between superior, mediocre and poor performance, when evaluating student work.
Better feedback

Furthermore, identifying specific levels of student performance allows the teacher to provide more detailed feedback to students. The teacher and the students can more clearly recognize areas that need improvement.


Dr. Mueller

Enduring ..........

[Md. Mostafa Rashel]

Analytic Versus Holistic Rubrics

For a particular task you assign students, do you want to be able to assess how well the students perform on each criterion, or do you want to get a more global picture of the students' performance on the entire task? The answer to that question is likely to determine the type of rubric you choose to create or use: Analytic or holistic.

Analytic rubric

Most rubrics, like the Research rubric above, are analytic rubrics. An analytic rubric articulates levels of performance for each criterion so the teacher can assess student performance on each criterion. Using the Research rubric, a teacher could assess whether a student has done a poor, good or excellent job of "organization" and distinguish that from how well the student did on "historical accuracy."

Holistic rubric

In contrast, a holistic rubric does not list separate levels of performance for each criterion. Instead, a holistic rubric assigns a level of performance by assessing performance across multiple criteria as a whole. For example, the analytic research rubric above can be turned into a holistic rubric:

3 - Excellent Researcher
•   included 10-12 sources
•   no apparent historical inaccuracies
•   can easily tell which sources information was drawn from
•   all relevant information is included

2 - Good Researcher
•   included 5-9 sources
•   few historical inaccuracies
•   can tell with difficulty where information came from
•   bibliography contains most relevant information

1 - Poor Researcher
•   included 1-4 sources
•   lots of historical inaccuracies
•   cannot tell from which source information came
•   bibliography contains very little information

In the analytic version of this rubric, 1, 2 or 3 points is awarded for the number of sources the student included. In contrast, number of sources is considered along with historical accuracy and the other criteria in the use of a holistic rubric to arrive at a more global (or holistic) impression of the student work. Another example of a holistic rubric is the "Holistic Critical Thinking Scoring Rubric" (in PDF) developed by Facione & Facione.

When to choose an analytic rubric
Analytic rubrics are more common because teachers typically want to assess each criterion separately, particularly for assignments that involve a larger number of criteria. It becomes more and more difficult to assign a level of performance in a holistic rubric as the number of criteria increases. For example, what level would you assign a student on the holistic research rubric above if the student included 12 sources, had lots of inaccuracies, did not make it clear from which source information came, and whose bibliography contained most relevant information? As student performance increasingly varies across criteria it becomes more difficult to assign an appropriate holistic category to the performance. Additionally, an analytic rubric better handles weighting of criteria. How would you treat "historical accuracy" as more important a criterion in the holistic rubric? It is not easy. But the analytic rubric handles it well by using a simple multiplier for each criterion.

When to choose a holistic rubric
So, when might you use a holistic rubric? Holistic rubrics tend to be used when a quick or gross judgment needs to be made. If the assessment is a minor one, such as a brief homework assignment, it may be sufficient to apply a holistic judgment (e.g., check, check-plus, or no-check) to quickly review student work. But holistic rubrics can also be employed for more substantial assignments. On some tasks it is not easy to evaluate performance on one criterion independently of performance on a different criterion. For example, many writing rubrics (see example) are holistic because it is not always easy to disentangle clarity from organization or content from presentation. So, some educators believe a holistic or global assessment of student performance better captures student ability on certain tasks. (Alternatively, if two criteria are nearly inseparable, the combination of the two can be treated as a single criterion in an analytic rubric.)





Dr. Mueller

Enduring ..........

[Md. Mostafa Rashel]

How Many Levels of Performance Should I Include in my Rubric?

There is no specific number of levels a rubric should or should not possess. It will vary depending on the task and your needs. A rubric can have as few as two levels of performance (e.g., a checklist) or as many as ... well, as many as you decide is appropriate. (Some do not consider a checklist a rubric because it only has two levels -- a criterion was met or it wasn't. But because a checklist does contain criteria and at least two levels of performance, I include it under the category of rubrics.) Also, it is not true that there must be an even number or an odd number of levels. Again, that will depend on the situation.

To further consider how many levels of performance should be included in a rubric, I will separately address analytic and holistic rubrics.

Analytic rubrics
Generally, it is better to start with a smaller number of levels of performance for a criterion and then expand if necessary. Making distinctions in student performance across two or three broad categories is difficult enough. As the number of levels increases, and those judgments become finer and finer, the likelihood of error increases.

Thus, start small. For example, in an oral presentation rubric, amount of eye contact might be an important criterion. Performance on that criterion could be judged along three levels of performance: never, sometimes, always.

makes eye contact with            audience                never            sometimes     always

Although these three levels may not capture all the variation in student performance on the criterion, it may be sufficient discrimination for your purposes. Or, at the least, it is a place to start. Upon applying the three levels of performance, you might discover that you can effectively group your students' performance in these three categories. Furthermore, you might discover that the labels of never, sometimes and always sufficiently communicates to your students the degree to which they can improve on making eye contact.

On the other hand, after applying the rubric you might discover that you cannot effectively discriminate among student performance with just three levels of performance. Perhaps, in your view, many students fall in between never and sometimes, or between sometimes and always, and neither label accurately captures their performance. So, at this point, you may decide to expand the number of levels of performance to include never, rarely, sometimes, usually and always.

makes eye contact   never   rarely   sometimes   usually   always

There is no "right" answer as to how many levels of performance there should be for a criterion in an analytic rubric; that will depend on the nature of the task assigned, the criteria being evaluated, the students involved and your purposes and preferences. For example, another teacher might decide to leave off the "always" level in the above rubric because "usually" is as much as normally can be expected or even wanted in some instances. Thus, the "makes eye contact" portion of the rubric for that teacher might be

makes eye contact   never   rarely   sometimes   usually

So, I recommend that you begin with a small number of levels of performance for each criterion, apply the rubric one or more times, and then re-examine the number of levels that best serve your needs. I believe starting small and expanding if necessary is preferable to starting with a larger number of levels and shrinking the number because rubrics with fewer levels of performance are normally
•   easier and quicker to administer
•   easier to explain to students (and others)
•   easier to expand than larger rubrics are to shrink

The fact that rubrics can be modified and can reasonably vary from teacher to teacher again illustrates that rubrics are flexible tools to be shaped to your purposes. To read more about the decisions involved in developing a rubric, see the chapter entitled, "Step 4: Create the Rubric."

Dr. Mueller

Enduring ..........

[Md. Mostafa Rashel]

Holistic rubrics
Much of the advice offered above for analytic rubrics applies to holistic rubrics as well. Start with a small number of categories, particularly since holistic rubrics often are used for quick judgments on smaller tasks such as homework assignments. For example, you might limit your broad judgments to
•   satisfactory
•   unsatisfactory
•   not attempted
or
•   check-plus
•   check
•   no check
or even just
•   satisfactory (check)
•   unsatisfactory (no check)
Of course, to aid students in understanding what you mean by "satisfactory" or "unsatisfactory" you would want to include descriptors explaining what satisfactory performance on the task looks like.
Even with more elaborate holistic rubrics for more complex tasks I recommend that you begin with a small number of levels of performance. Once you have applied the rubric you can better judge if you need to expand the levels to more effectively capture and communicate variation in student performance.

Dr. Mueller

Enduring ..........

[Md. Mostafa Rashel]

Step 4: Create the Rubric
________________________________________
 Creating an Analytic Rubric
 Creating a Holistic Rubric
 Final Step: Checking Your Rubric
 Workshop: Writing a Good Rubric
 
Note: Before you begin this section I would recommend that you read the section on Rubrics to learn about the characteristics of a good rubric.

In Step 1 of creating an authentic assessment, you identified what you wanted your students to know and be able to do -- your standards.

In Step 2, you asked how students could demonstrate that they had met your standards. As a result, you developed authentic tasks they could perform.

In Step 3, you identified the characteristics of good performance on the authentic task -- the criteria.

Now, in Step 4, you will finish creating the authentic assessment by constructing a rubric to measure student performance on the task. To build the rubric, you will begin with the set of criteria you identified in Step 3. As mentioned before, keep the number of criteria manageable. You do not have to look for everything on every assessment.
Once you have identified the criteria you want to look for as indicators of good performance, you next decide whether to consider the criteria analytically or holistically. (See Rubrics for a description of these two types of rubrics.)
 
Creating an Analytic Rubric
In an analytic rubric performance is judged separately for each criterion. Teachers assess how well students meet a criterion on a task, distinguishing between work that effectively meets the criterion and work that does not meet it. The next step in creating a rubric, then, is deciding how fine such a distinction should be made for each criterion. For example, if you are judging the amount of eye contact a presenter made with his/her audience that judgment could be as simple as did or did not make eye contact (two levels of performance), never, sometimes or always made eye contact (three levels), or never, rarely, sometimes, usually, or always made eye contact (five levels).

Generally, it is better to start small with fewer levels because it is usually harder to make more fine distinctions. For eye contact, I might begin with three levels such as never, sometimes and usually. Then if, in applying the rubric, I found that some students seemed to fall in between never and sometimes, and never or sometimes did not adequately describe the students' performance, I could add a fourth (e.g., rarely) and, possibly, a fifth level to the rubric.

In other words, there is some trial and error that must go on to arrive at the most appropriate number of levels for a criterion. (See the Rubric Workshop below to see more detailed decision-making involved in selecting levels of performance for a sample rubric.)

Do I need to have the same number of levels of performance for each criterion within a rubric?
No. You could have five levels of performance for three criteria in a rubric, three levels for two other criteria, and four levels for another criterion, all within the same rubric. Rubrics are very flexible Alaskan Moose. There is no need to force an unnatural judgment of performance just to maintain standardization within the rubric. If one criterion is a simple either/or judgment and another criterion requires finer distinctions, then the rubric can reflect that variation.
Here are some examples of rubrics with varying levels of performance......

Do I need to add descriptors to each level of performance?
No. Descriptors are recommended but not required in a rubric. As described in Rubrics, descriptors are the characteristics of behavior associated with specific levels of performance for specific criteria. For example, in the following portion of an elementary science rubric, the criteria are 1) observations are thorough, 2) predictions are reasonable, and 3) conclusions are based on observations. Labels (limited, acceptable, proficient) for the different levels of performance are also included. Under each label, for each criterion, a descriptor (in brown) is included to further explain what performance at that level looks like.

See the attached file

As you can imagine, students will be more certain what is expected to reach each level of performance on the rubric if descriptors are provided. Furthermore, the more detail a teacher provides about what good performance looks like on a task the better a student can approach the task. Teachers benefit as well when descriptors are included. A teacher is likely to be more objective and consistent when applying a descriptor such as "most observations are clear and detailed" than when applying a simple label such as "acceptable." Similarly, if more than one teacher is using the same rubric, the specificity of the descriptors increases the chances that multiple teachers will apply the rubric in a similar manner. When a rubric is applied more consistently and objectively it will lead to greater reliability and validity in the results.


Dr. Mueller

Enduring ..........

[Md. Mostafa Rashel]

Assigning point values to performance on each criterion

As mentioned above, rubrics are very flexible tools. Just as the number of levels of performance can vary from criterion to criterion in an analytic rubric, points or value can be assigned to the rubric in a myriad of ways. For example, a teacher who creates a rubric might decide that certain criteria are more important to the overall performance on the task than other criteria. So, one or more criteria can be weighted more heavily when scoring the performance. For example, in a rubric for solo auditions, a teacher might consider five criteria: (how well students demonstrate) vocal tone, vocal technique, rhythm, diction and musicality. For this teacher, musicality might be the most important quality that she has stressed and is looking for in the audition. She might consider vocal technique to be less important than musicality but more important than the other criteria.So, she might give musicality and vocal technique more weight in her rubric. She can assign weights in different ways. Here is one common format:
Rubric 1: Solo Audition

                                    0   1   2   3   4   5   weight
vocal tone                           
vocal technique                                    x2
rhythm                           
diction                           
musicality                                                    x3

In this case, placement in the 4-point level for vocal tone would earn the student four points for that criterion. But placement in the 4-point box for vocal technique would earn the student 8 points, and placement in the 4-point box for musicality would earn the student 12 points. The same weighting could also be displayed as follows:


Rub

ric 2: Solo Audition

                                  NA   Poor      Fair   Good   Very Good   Excellent
vocal tone                 0   1      2              3             4                    5
vocal technique         0   2      4              6             8                    10
rhythm                         0   1      2              3             4                    5
diction                         0   1      2              3             4                    5
musicality                         0   3      6              9             12                    15

In both examples, musicality is worth three times as many points as vocal tone, rhythm and diction, and vocal technique is worth twice as much as each of those criteria. Pick a format that works for you and/or your students. There is no "correct" format in the layout of rubrics. So, choose one or design one that meets your needs.

Yes, but do I need equal intervals between the point values in a rubric?
No. Say it with me one more time -- rubrics are flexible tools. Shape them to fit your needs, not the other way around. In other words, points should be distributed across the levels of a rubric to best capture the value you assign to each level of performance. For example, points might be awarded on an oral presentation as follows:

Rubric 3: Oral Presentation

Criteria                           never   sometimes   always
makes eye contact           0               3                   4
volume is appropriate   0               2                   4
enthusiasm is evident   0               2                   4
summary is accurate   0               4                   8


In other words, you might decide that at this point in the year you would be pleased if a presenter makes eye contact "sometimes," so you award that level of performance most of the points available. However, "sometimes" would not be as acceptable for level of volume or enthusiasm.

Here are some more examples of rubrics illustrating the flexibility of number of levels and value you assign each level.

Rubric 4: Oral Presentation

Criteria                           never   sometimes   usually
makes eye contact           0           2                   4
volume is appropriate   0                              4
enthusiasm is evident   0                              4
summary is accurate   0           4                    8
In the above rubric, you have decided to measure volume and enthusiasm at two levels -- never or usually -- whereas, you are considering eye contact and accuracy of summary across three levels. That is acceptable if that fits the type of judgments you want to make. Even though there are only two levels for volume and three levels for eye contact, you are awarding the same number of points for a judgment of "usually" for both criteria. However, you could vary that as well:

Rubric 5: Oral Presentation

Criteria                           never   sometimes   usually
makes eye contact           0                  2            4
volume is appropriate   0                               2
enthusiasm is evident   0                               2
summary is accurate   0                  4            8

In this case, you have decided to give less weight to volume and enthusiasm as well as to judge those criteria across fewer levels.

So, do not feel bound by any format constraints when constructing a rubric. The rubric should best capture what you value in performance on the authentic task. The more accurately your rubric captures what you want your students to know and be able to do the more valid the scores will be.


Dr. Mueller

Enduring ..........

[Md. Mostafa Rashel]

Creating a Holistic Rubric

In a holistic rubric, a judgment of how well someone has performed on a task considers all the criteria together, or holistically, instead of separately as in an analytic rubric. Thus, each level of performance in a holistic rubric reflects behavior across all the criteria. For example, here is a holistic version of the oral presentation rubric above.
Rubric 6: Oral Presentation (Holistic)

Oral Presentation Rubric

Mastery
•   usually makes eye contact
•   volume is always appropriate
•   enthusiasm present throughout presentation
•   summary is completely accurate

Proficiency
•   usually makes eye contact
•   volume is usually appropriate
•   enthusiasm is present in most of presentation
•   only one or two errors in summary

Developing
•   sometimes makes eye contact
•   volume is sometimes appropriate
•   occasional enthusiasm in presentation
•   some errors in summary

Inadequate
•   never or rarely makes eye contact
•   volume is inappropriate
•   rarely shows enthusiasm in presentation
•   many errors in summary

An obvious, potential problem with applying the above rubric is that performance often does not fall neatly into categories such as mastery or proficiency. A student might always make eye contact, use appropriate volume regularly, occasionally show enthusiasm and include many errors in the summary. Where you put that student in the holistic rubric? Thus, it is recommended that the use of holistic rubrics be limited to situations when the teacher wants to:

•   make a quick, holistic judgment that carries little weight in evaluation, or
•   evaluate performance in which the criteria cannot be easily separated.

Quick, holistic judgments are often made for homework problems or journal assignments. To allow the judgment to be quick and to reduce the problem illustrated in the above rubric of fitting the best category to the performance, the number of criteria should be limited. For example, here is a possible holistic rubric for grading homework problems.

Rubric 7: Homework Problems

Homework Problem Rubric

++ (3 pts.)
•   most or all answers correct, AND
•   most or all work shown

+ (1 pt.)
•   at least some answers correct, AND
•   at least some but not most work shown

- (0 pts.)
•   few answers correct, OR
•   little or no work shown

Although this homework problem rubric only has two criteria and three levels of performance, it is not easy to write such a holistic rubric to accurately capture what an evaluator values and to cover all the possible combinations of student performance. For example, what if a student got all the answers correct on a problem assignment but did not show any work? The rubric covers that: the student would receive a (-) because "little or no work was shown." What if a student showed all the work but only got some of the answers correct? That student would receive a (+) according to the rubric. All such combinations are covered. But does giving a (+) for such work reflect what the teacher values? The above rubric is designed to give equal weight to correct answers and work shown. If that is not the teacher's intent then the rubric needs to be changed to fit the goals of the teacher.

All of this complexity with just two criteria -- imagine if a third criterion were added to the rubric. So, with holistic rubrics, limit the number of criteria considered, or consider using an analytic rubric.

Dr. Mueller

Enduring ..........

[Md. Mostafa Rashel]

Final Step: Checking Your Rubric

As a final check on your rubric, you can do any or all of the following before applying it.
•   Let a colleague review it.
•   Let your students review it -- is it clear to them?
•   Check if it aligns or matches up with your standards.
•   Check if it is manageable.
•   Consider imaginary student performance on the rubric.

By the last suggestion I mean to imagine that a student had met specific levels of performance on each criterion (for an analytic rubric). Then ask yourself if that performance translates into the score that you think is appropriate. For example, on Rubric 3 above, imagine a student scores
•   "sometimes" for eye contact (3 pts.)
•   "always" for volume (4 pts.)
•   "always" for enthusiasm (4 pts.)
•   "sometimes" for summary is accurate (4 pts.)

That student would receive a score of 15 points out of a possible 20 points. Does 75% (15 out of 20) capture that performance for you? Perhaps you think a student should not receive that high of a score with only "sometimes" for the summary. You can adjust for that by increasing the weight you assign that criterion. Or, imagine a student apparently put a lot of work into the homework problems but got few of them correct. Do you think that student should receive some credit? Then you would need to adjust the holistic homework problem rubric above. In other words, it can be very helpful to play out a variety of performance combinations before you actually administer the rubric. It helps you see the forest through the trees.

Of course, you will never know if you really have a good rubric until you apply it. So, do not work to perfect the rubric before you administer it. Get it in good shape and then try it. Find out what needs to be modified and make the appropriate changes.

Okay, does that make sense? Are you ready to create a rubric of your own? Well, then come into my workshop and we will build one together. I just need you to wear these safety goggles. Regulations. Thanks.

(For those who might be "tabularly challenged" (i.e., you have trouble making tables in your word processor) or would just like someone else to make the rubric into a tabular format for you, there are websites where you enter the criteria and levels of performance and the site will produce the rubric for you.)


Dr. Mueller

Enduring ..........

[Md. Mostafa Rashel]

 Workshop: Writing a Good Rubric

 Step 1: Identify the Standards
 Step 2: Select an Authentic Task
 Step 3: Identify the Criteria for the Task
 Step 4: Create the Rubric

Step 1: Identify the Standards
________________________________________

[/b]

For any type of assessment, you first must know where you want to end up.  What are your goals for your students?  An assessment cannot produce valid inferences unless it measures what it is intended to measure.  And it cannot measure what it is intended to measure unless the goal(s) has been clearly identified.  So, completing the rest of the following steps will be unproductive without clear goals for student learning.

Standards, like goals, are statements of what students should know and be able to do. However, standards are typically more narrow in scope and more amenable to assessment than goals.  (Before going further, I would recommend that you read the section on Standards for a fuller description of standards and how they are different from goals and objectives.)

 What Do Standards Look Like?
 How do you get Started Writing Standards?
 What are Some Guidelines to Follow in Developing Standards?
 Workshop: Writing a Good Standard
 
What Do Standards Look Like?
Standards are typically one-sentence statements of what students should know and be able to do at a certain point. Often a standard will begin with a phrase such as "Students will be able to ..." (SWBAT). For example,
Students will be able to add two-digit numbers.
Or, it might be phrased
Students will add two-digit numbers.
A student will add two-digit numbers.
Or just
Identify the causes and consequences of the Revolutionary War.
Explain the process of photosynthesis.
More examples:

Extensive set of links to standards organized by subject and state
Standards examples (this site)
Also, read the section on types of standards to see how standards can address course content, or process skills or attitudes towards learning.



Dr. Mueller

Enduring ..........

[Md. Mostafa Rashel]

How Do You Get Started?
I recommend a three-step process for writing standards:
1. REFLECT
2. REVIEW
3. WRITE

1. REFLECT
As I will discuss below, there are many sources you can turn to to find examples of goals and standards that might be appropriate for your students. There are national and state standards as well as numerous websites such as those above with many good choices. It is unnecessary to start from scratch. However, before you look at the work of others, which can confine your thinking, I would highly recommend that you, as a teacher or school or district, take some time to examine (or REFLECT upon) what you value. What do you really want your students to know and be able to do when they leave your grade or school?

Here is a sample of questions you might ask yourself:

•   What do you want students to come away with from an education at _______?
•   What should citizens know and be able to do?
•   If you are writing standards for a particular discipline, what should citizens know and be able to do related to your discipline?
•   What goals and standards do you share with other disciplines?
•   What college preparation should you provide?
•   Think of a graduate or current student that particularly exemplifies the set of knowledge and skills that will make/has made that student successful in the real world. What knowledge and skills (related and unrelated to your discipline) does that person possess?
•   Ask yourself, "above all else, we want to graduate students who can/will ........?
•   When you find yourself complaining about what students can't or don't do, what do you most often identify?

As a result of this reflection, you might reach consensus on a few things you most value and agree should be included in the standards. You might actually write a few standards. Or, you might produce a long list of possible candidates for standards. I do not believe there is a particular product you need to generate as a result of the reflection phase. Rather, you should move on to Step 2 (Review) when you are clear about what is most important for your students to learn. For example, reflection and conversation with many of the stakeholders for education led the Maryland State Department of Education to identify the Skills for Success it believes are essential for today's citizens. Along with content standards, the high school assessment program in Maryland will evaluate how well students have acquired the ability to learn, think, communicate, use technology and work with others.

2. REVIEW
Did you wake up this morning thinking, "Hey, I'm going to reinvent the wheel today"? No need. There are many, many good models of learning goals and standards available to you. So, before you start putting yours down on paper, REVIEW what others have developed. For example, you can Look at
•   your state goals and standards
•   relevant national goals and standards
•   other state and local standards already created
o   check out the site mentioned above - Putnam Valley
•   your existing goals and standards if you have any
•   other sources that may be relevant (e.g., what employers want, what colleges want)
Look for
•   descriptions and language that capture what you said you value in Step 1 (REFLECT)
•   knowledge and skills not captured in the first step -- should they be included?
•   ways to organize and connect the important knowledge and skills
Look to
•   develop a good sense of the whole picture of what you want your students to know and to do
•   identify for which checkpoints (grades) you want to write standards

3. WRITE
The biggest problem I have observed in standards writing among the schools and districts I have worked with is the missing of the forest for the trees. As with many tasks, too often we get bogged down in the details and lose track of the big picture. I cannot emphasize enough how important it is to periodically step back and reflect upon the process. As you write your standards, ask yourself and your colleagues guiding questions such as
•   So, tell me again, why do we think this is important?
•   Realistically, are they ever going to have to know this/do this/use this?
•   How does this knowledge/skill relate to this standard over here?
•   We don't have a standard about X; is this really more important than X?
•   Can we really assess this? Should we assess it?
•   Is this knowledge or skill essential for becoming a productive citizen? How? Why?
•   Is this knowledge or skill essential for college preparation?

Yes, you may annoy your colleagues with these questions (particularly if you ask them repeatedly as I would advocate), but you will end up with a better set of standards that will last longer and provide a stronger foundation for the steps that follow in the creation of performance assessments.
Having said that, let's get down to the details. I will offer suggestions for writing specific standards by a) listing some common guidelines for good standards and b) modeling the development of a couple standards much as I would if I were working one-on-one with an educator.

Dr. Mueller

Enduring ..........