Evaluation: assessing student achievement of learning outcomes

  1. Introduction
  2. Key to successful learning: aligning assessment with learning outcomes
  3. Reviewing your assessment strategy
  4. Designing assessments which allow students to demonstrate their achievement of the learning outcomes
  5. Key assessment terms
  6. Common assessment tasks
  7. Assessing group work
  8. Using rubrics to communicate assessment standards
  9. For further assistance
  10. Further reading and resources

The following items (Questions 2 and 3) in the Learner Experience of Unit survey ask students to indicate their level of agreement in relation to assessments:

  • The assessment criteria were clearly defined.
  • The grading standards or grade descriptors were clear to me.

This document aims to assist teachers to ensure that assessment tasks and practices are effective for student learning. Additional assistance is available from the LTC's Assessment Toolkit.

 

1. Introduction

An effective assessment task is one which assesses students' attainment of the learning outcomes. Unit learning outcomes are what students are expected to know, understand or be able to do in order to be successful in a unit. They begin with an action verb and describe something observable and measurable. You can find out more about learning outcomes in other resources on the Learning and Teaching Centre website, and in the Evaluation Resource: Developing Your Unit - Clear Goals and Standards.

Return To Top

 

2. Key to successful learning: aligning assessment with learning outcomes

One of the keys to successful learning is the aligned curriculum (Biggs, 2003): this means that learning outcomes are clear, learning experiences are designed to assist student achievement of those outcomes, and carefully designed assessment tasks allow students to demonstrate achievement of those outcomes. This concept is illustrated in Figure 1:

1. The learning outcomes are clear.

Blue arrow pointing down

2. The learning experiences (face-to-face and virtual) are designed to help students achieve those learning outcomes.

Blue arrow pointing down

3. The assessment tasks allow the students to demonstrate their achievement of those learning outcomes.

 

More information on curriculum alignment can be found in the Evaluation Resource: Developing your unit - Organising Your Unit.

Return To Top

 

3. Reviewing your assessment strategy

Begin by thinking about your current assessment practice. The questions below will help guide your reflection.

  • How do you currently assess your students? List the methods you use.
  • Is each assessment worth doing and can/do you explain to your students why?
  • Can you explain how the assessment methods you currently use are matched to the expected learning outcomes?
  • What skills and capabilities do you want your students to leave your unit/course with?
  • Approximately how much does each assessment process cost students and staff in terms of time taken and resources used?
  • Do you feel you might be over assessing? How do you know?
  • What criteria do you use? Are they yours, or can you involve students themselves in formulating them?
  • Do the students know the criteria? Do they really understand them?
  • Is the feedback you give your students clearly related to your assessment criteria?
  • How well does the feedback students receive on assessed work help them to know how they are doing?
  • How much practice and guidance do students get in the chosen assessment methods?
  • What assessments do students enjoy and why?
  • How do you know that the students find your assessments useful?
  • In what ways do the assessments help your student learn?

Review your assessment strategy regularly. It can be even more productive when done in partnership with your students and colleagues.

Return To Top

 

4. Designing assessments which allow students to demonstrate their achievement of the learning outcomes

Assessing learning can profoundly shape the educational experiences of students. One of the challenges of effective assessment is to ensure that there is a close alignment between the learning goals, the teaching and learning activities aimed at meeting learning goals and the assessment tasks used to assess whether learning goals have been met. Current best practice includes assessment which is aligned to learning goals which focus not only on content knowledge but also on process and capabilities.

Return To Top

 

5. Key assessment terms

Table 5.1: Key assessment terms
AssessmentThe collection of information about the nature and extent of learning outcomes/any procedure used to estimate learners learning. The term is derived from the Latin (assidere meaning "to sit beside").
MeasurementRepresentation of assessment information by a number or grade on a scale of some kind. Answers the question, How much?
EvaluationThe making of judgments about the value of a grade and/or the nature and extent of learning outcomes. Answers the question, How well?
Assessment taskAn instrument or systematic procedure by which assessment information is collected.
Formative assessmentUngraded assessment task used before or during learning to support planning and/or diagnosis and/or to provide feedback about learning progress/offers advice and feedback which does not contribute grades towards the final result.
Summative assessmentGraded assessment task used following learning which counts towards the final result.
ValidityDegree to which the assessment task measures what it is intended to measure.
ReliabilityDegree to which the assessment task consistently yields the same result.
Norm-referencedUses the performance of a group of learners to rank order learners or 'grading on the curve'. Number of learners who can receive distinctions, credits, passes or fails is set.
Criterion-referencedEstablishes the criteria for performance and any learner meeting the criteria receives the associated grade. Every student can potentially achieve the highest grade.
Standards-basedEstablishes the criteria for performance as well as articulates the various levels of quality in performance that is associated with a grade. Grades are awarded to students based on the level of performance they have achieved.
Authentic assessmentAssessment tasks which test whether a learner is able to demonstrate their learning outcomes in a situation which is as close as possible to a real world context.

Return To Top

 

6. Common assessment tasks

There are many different assessment tasks you can use to assess your students. The following will help you choose the right assessment task. Remember that whatever assessment task you decide to use, it should be clearly aligned with the learning outcomes and teaching and learning activities in your course.

6.1 Short form test

Short form tests are also known as objective tests. They include multiple choice, completion (or cloze), true-false and matching types, of which multiple choice is the most commonly used. A multiple choice test item consists of a statement, called the stem, and several alternative statements, one of which is the correct answer, while the others are distracters.

For example:

Formative assessment refers to:
(a) a practice test;
(b) a test used to determine a grade;
(c) a test used to determine prior knowledge; or
(d) a test to monitor learning progress.

Table 6.1: Pros and cons of the Short Form Test
ProsCons
Measures wide sample of contentDifficult to set items which assess more than recall
Easy and quick to scoreTime consuming to produce
Good for reviewing contentEncourages guessing
Provides formative feedbackRestricts creative students
Provides fast feedbackNourishes illiteracy
Items can be reusedPoorly constructed questions can give clues to students
Marker reliability highDifficult to interpret wrong answers

Hints for writing multiple choice items:

  • The stem should consist of a single, clear idea. It should make sense independent of the rest of the question.
  • Avoid stems stated in negative terms as these are more difficult to understand and may cause confusion.
  • Make sure that all the alternatives are grammatically consistent with the stem and similar in form and length to one another.
  • Make the distracters plausible by using common misconceptions and typical student errors.
  • If you use the alternatives 'none of the above' and 'all of the above' include them as the incorrect answer about 75% of the time.
  • The correct answer should appear without pattern and equally often in each of the alternative positions.

6.2 Short answer test

Short answer questions require a brief answer consisting of a phrase, sentence or short paragraph.

For example: Write a brief definition of formative assessment.

Table 6.2: Pros and cons of the Short Answer Test
ProsCons
Measures relatively wide sample of contentRelatively difficult to set compared to short form
Reasonably easy and quick to scoreDifficult to establish criteria
Encourages clear and concise expressionScoring may be subjective
Encourages literacyMay encourage guessing
Good for reviewing contentLittle opportunity to display argument and originality
Items can be reused 

Hints for writing short answer questions:

  • Be clear about what you are asking.
  • Avoid using phrases straight from the text book.

6.3 Essay

Essays require students to select, organise and integrate material on a given topic. They also test writing skill and the ability to develop an argument and use evidence to support it. Essays may vary from a single page (about 300 typed words) to major assignments of ten pages (3000 words). Essays may be written under timed exam conditions or set as research assignments.

For example:

Compare summative and formative assessment with reference to assessing higher order learning as defined by Bloom's taxonomy for the cognitive domain (Bloom, 1956).

Table 6.3: Pros and cons of the Essay
ProsCons
Helps students to develop writing skillsMay not sample a wide range of content
Can reveal errors in understanding or misconceptionsQuestions may not be well thought out
Takes less time to set than short form questions such as multiple choiceHand written responses may be graded on factors other than the content, such as legibility
Requiring students to write can improve understanding of a topicTime consuming to grade
Helps students to develop information literacy skillsConsistency in assigning marks is difficult to maintain
 Subjectivity may affect fair grading

6.4 Performance test

Performance tests involve either a hands on activity, such as using microscope correctly or taking a patient history, or the development of products, such as developing a building design or software package.

Table 6.4: Pros and cons of a Performance Test
ProsCons
Encourages students to take ownership of the learning processMay be time consuming to set, present and assess
Replicates real world conditions or contextsCan be difficult to determine assessment criteria
Students have the opportunity to demonstrate mastery of what they have learnedCan prompt performance anxiety in students
Can assess a range of skills or outcomes including generic skillsMay require additional resources
Allows for a variety of tasksMakes comparison between students difficult since the products may be very different
Encourages active learningSubjectivity may affect fair grading

Hints for using performance assessments:

  • Ensure that the task is clearly related to unit goals.
  • Choose tasks which can be completed within the time allowed.
  • Check that any special resources needed are available to all students.
  • Make sure that students have the knowledge and skills needed to carry out the task.
  • Describe the task and its requirements clearly in writing. Allow time in class for discussion and clarification.
  • Provide clear criteria for assessment when the task is set. Consider involving students in deciding on the criteria.
  • Require students to provide progress reports or to submit material at various intervals before the final submission date.
  • Check on student progress regularly and have contingency plans if something goes wrong.
  • If possible involve others in the assessment process (e.g. another instructor, peers or the participants themselves).

6.5 Written report

The report is a common way of presenting information and recommendations or conclusions related to a specific purpose. Reports are often used as assessment task because well developed report writing skills are important in many professional contexts. Reports are written based on gathering and analysing information using a discipline specific methodology and format. They can be used to assess laboratory, field work or case studies.

Table 6.5: Pros and cons of the Written Report
ProsCons
Replicates real world activityMay encourage students to fabricate data to make the report 'look good'
Marking for grading using a template is relatively fastInconsistent marking between multiple markers can arise
Gives students practice in writing using a standard formatCan prompt performance anxiety in students
Can assess generic skills such as information and computer literacyMarks allocated may not reflect time and effort needed to complete the task
Allows for a range topics and fociProducing a report as an assessment task may not align with the intended learning outcomes
May encourage reflection and problem solvingSubjectivity may affect fair grading

Hints for using written reports:

  • Be clear about how marks are allocated to each section of the report.
  • Weight marks according to the learning objectives which the task is assessing.
  • State clearly what the report format is, include exemplars of good and poor reports.
  • Tell students whether and how language (grammar, spelling, punctuation) will be assessed.
  • Keep the number of reports required in a semester to a reasonable number and match the percentage of overall marks to the time and effort needed to produce a good quality report.
  • Explicitly teach students effective report writing skills before using the report as an assessment task.

6.6 Project

Projects are an extended piece of work involving inquiry based activities. Project may be small or large, undertaken by individuals or in groups and have outcomes such as a report, design, art work, working product.

Table 6.6: Pros and cons of the Project
ProsCons
Allows for greater student involvement in and responsibility for learningMay be time consuming to develop and mark
Assesses in-context student learningCan be difficult to determine assessment criteria
Encourages initiative, independence and problem solvingSubjectivity may affect fair grading
Can assess a wide range of skills or outcomes including generic skills particularly time and task managementMay require additional resources
Provides an opportunity to showcase skills and achievementsMakes comparison between students difficult since the projects may be very different
Is comprehensive, multidimensional and flexibleAllows students to explore a topic in depth

Hints for using projects:

  • Provide clear criteria for assessment when the task is set. Consider involving students in deciding on the criteria.
  • Require students to provide progress reports or to submit material at various intervals before the final submission date.
  • Check on student progress regularly and have contingency plans if something goes wrong.
  • If possible involve others in the assessment process (e.g. another instructor, peers or the participants themselves).

6.7 Presentation

Presentations are usually made orally to a class on a prepared topic and may include the use of presentation aids such as PowerPoint, handouts or audiovisuals. This assessment may be undertaken individually or as a group. Presentations may take different forms such as role plays, facilitating group activities, debating, presenting a product, question and answer time, and formal speeches.

Table 6.7: Pros and cons of the Presentation
ProsCons
Can assess a range of skills or outcomes including generic skillsCan be very time consuming of limited class meetings
Marking using criteria is relatively fast and reliableSubjectivity may affect fair grading
Allows for immediate feedback to the studentCan prompt performance anxiety in students
Allows for a variety of topicsProvides an opportunity to display argument and originality

Hints for using presentation tasks:

  • Design and use an assessment pro-forma with weightings for each aspect of the presentation.
  • Inform students of the requirements and criteria.
  • Set minimum and maximum time limits for each presentation.
  • If group presentations are used work out beforehand what is expected from each member of the group and how marks will be distributed among group members.
  • With group presentations, include some assessment of the working of the group as well as of the presentation.
  • Provide students with opportunities to develop and practice oral skills.

6.8 Poster

A poster is a visual representation of a topic or the outcomes of learning activity. They can use different media, including online technology, and can be created individually or in groups.

Table 6.8: Pros and cons of the Poster
ProsCons
Provides an opportunity to display creativity and originalityCan focus unduly on presentation rather than content or understanding
Can assess a range of skills or outcomes including generic skillsMay require additional resources
Marking using criteria is relatively fast and reliableMakes comparison between students difficult since the posters may be very different
Allows for a variety of topicsSubjectivity may affect fair grading
Has potential for peer assessment 
Encourages active learning 

Hints for using poster presentations:

  • Ensure that the task is clearly related to unit goals.
  • Check that any special resources or materials needed are available to all students.
  • Make sure that students have the knowledge and skills needed to carry out the task.
  • Describe the task and its requirements clearly in writing. Allow time in class for discussion and clarification.
  • Provide clear criteria for assessment when the task is set. Consider involving students in deciding on the criteria.
  • If the task is large, require students to provide progress reports or to submit material at various intervals before the final submission date.
  • Check on student progress regularly and have contingency plans if something goes wrong.
  • If possible involve others in the assessment process (e.g. another instructor, peers or the participants themselves).

6.9 Journal

Journals (also called learning logs or learning diaries) are written by students over a period of time, such as a semester, in order to record and reflect on their personal learning experiences and outcomes. They provide an opportunity for students to express their feelings, thoughts and beliefs about the content and process of learning and themselves as learners using an informal writing style and structure.

Table 6.9: Pros and cons of the Journal
ProsCons
Allows for greater student involvement in and responsibility for learningMay be time consuming to develop and assess
Encourages self-assessment and reflectionCan be difficult to determine assessment criteria
Provides valuable insight into student feelings, thoughts and beliefsSubjectivity may affect fair grading
Is comprehensive, multidimensional and flexibleRequires time to establish the required high-trust, low risk environment
Encourages regular and extended writingMay raise issues of privacy and confidentiality
Students may resist undertaking regular writing and fabricate or sanitise journal entries 

Hints for using journals:

  • Describe the task and its requirements clearly in writing. Allow time in class for discussion and clarification.
  • Allow class time for journal writing.
  • Suggest areas for students to focus on, possibly using guide questions or statements.
  • Provide frequent feedback, especially in the early stages.
  • Consider keeping a journal yourself and share entries with your students.
  • Acknowledge the value of student comments by responding to journal items.
  • Take time to establish a high-trust low-risk learning environment.
  • Provide examples of journal entries.

6.10 Portfolio

A portfolio is "a purposeful collection of student work that exhibits the student's efforts, progress and achievements in one or more areas. The collection must include student participation in selecting contents, the criteria for judging merit and evidence of student self-reflection" (Paulson, Paulson & Meyer, 1991, p60).

Table 6.10: Pros and cons of the Portfolio
ProsCons
Allows for greater student involvement in and responsibility for learningMay be time consuming to develop and assess
Assesses in-context student learningCan be difficult to determine assessment criteria
Encourages self-assessment and reflectionSubjectivity may affect fair grading
Can assess a range of skills or outcomes including generic skillsMay require additional resources
Provides an opportunity to showcase skills and achievementsMakes comparison between students difficult since the portfolios may be very different
Comprehensive, multidimensional and flexible 

Hints for using portfolios:

  • Describe the task and its requirements clearly in writing. Allow time in class for discussion and clarification.
  • Provide examples of completed portfolios.
  • Involve students in selecting portfolio items.
  • Include compulsory items which show student learning activities, self-reflection and self-evaluation.
  • Use portfolios for different functions at different times of the year.
  • Ensure that the length of a portfolio is the equivalent of a written assignment you would normally set, such as an essay or report.
  • Provide clear criteria for assessment when the task is set. Consider involving students in deciding on the criteria.
  • Require students to provide progress reports or to submit material at various intervals before the final submission date.
  • Check on student progress regularly and have contingency plans if something goes wrong.
  • If possible involve others in the assessment process (e.g. another instructor, peers or the participants themselves).

Return To Top

 

7. Assessing group work

Group work can help develop specific skills sought by employers such as:

  • teamwork skills (skills in working within team dynamics; leadership skills);
  • analytical and cognitive skills (analysing task requirements; questioning; critically interpreting material; evaluating the work of others);
  • collaborative skills (conflict management and resolution; accepting intellectual criticism; flexibility; negotiation and compromise); and
  • organisational and time management skills.

Group work may reduce the workload involved in assessing, grading and providing feedback to students. Provided the model of group work adopted is workable and effective, group/groupwork assessment can reduce/streamline assessment by reducing the number of pieces of work to be assessed. However, the tasks of monitoring and assisting students through the process might also add to overall work load.

7.1 Dimensions of assessing group work

There are three basic dimensions to assessment of group work:

  • Students' demonstrated abilities of working effectively as a team member (process);
  • Students demonstrated knowledge and application of the skills of successful task completion (process); and
  • The quality of the output or product of the group assessment task (product).

Groups can produce tangible products such as posters, models or artefacts, formal reports and electronic or other forms of media product. They can also submit records of meetings, planning sheets or other monitoring documents as evidence of their progress. The output can also include (or consist of) a performance or seminar presentation.

The assessment plan for any unit must be designed to assess students' achievement of the unit learning outcomes. If the skills of teamwork are stated explicitly in the learning outcomes, these skills must be learnt, demonstrated and assessed. If the outcomes state that student groups produce a certain type and/or quality of product (presentation, poster or report), students must also be assessed on this according to specified criteria. Staff must consider carefully how and why they are assessing students - and be prepared to provide them with a written justification for each assessment task. This is most important for group-based assessment.

If group work is being used solely to aid learning, to motivate students or to reduce staff marking loads, the rationale for it should be made clear to the students and it should not require student learning that detracts from their achievement of other stated learning outcomes.

Students will also need explicit guidelines on what is acceptable and what is not acceptable in terms of their collaborative versus individual work and assessment. They need to be alerted as to how incidents of academic misconduct, such as plagiarism or collusion, might occur in the exercise and how such incidents can be avoided. More information on this is available in the Macquarie policy on Plagiarism. This type of information should be included in the written guidelines for the assigned work.

Assessment can be conducted by external assessors (experts in the field), lecturer or tutor, groups of peers, through self-assessment or by some combination of these. The advantage of a combination of assessors is that it allows multiple perspectives of the work to be judged and it reduces the chance of bias that might be inherent in a single form of assessment.

Students also need to know what will happen should one or more group members withdraw or if one or more contributes so little that it jeopardises the likelihood that the group can complete its task. They also need to be aware of how to deal with conflict within groups and what is the appeal process should students not be able to successfully negotiate among themselves about distribution of marks.

7.2 Planning group assessment

There are different ways of allocating marks to individual students for the quality of their contribution to the group-working process or the group work product.

Which assessment model is appropriate for the tasks you set for your students? In the first instance, decisions about the model of assessment need to be made with respect to the following:

  • What will be assessed - the product of the group work, the process of the group work, or both (and if the latter, what proportion of the total mark will be allocated to each)?
  • What assessment criteria will be used and who will determine them - you, your students or both?
  • Who will apply the assessment criteria and determine marks - you, your students (peer and/or self assessment), an external assessor or a combination?
  • If groups are to be given a total mark to 'share' according to individual contributions, how will the shared mark be determined and distributed?

7.3 Group assessment criteria: Models and examples

Students need to have the assessment criteria specified before they start work on the assessment task. They should know what outcomes they will be expected to demonstrate and how the evidence of their work needs to be documented. Where students are to collaboratively decide the criteria, this process should occur at the beginning and all students then receive a written copy of the agreed criteria.

The four marking models described below represent different ways of recognising the contribution of the individual student. In each case, it is assumed that students have written criteria that enable them to award marks and that, where relevant, they are given an appropriate algorithm to determine the distribution of marks.

Model 1: All students get the same mark or grade regardless of individual effort, contribution or ability

If teams in authentic working environments in your particular area customarily succeed or fail on the basis of team performance alone, and the contribution of individuals is of little importance, you may want to assess students this way. The marks can be determined by the quality of the product or the process or both. All students in the group are awarded the same mark. Staff must recognise that awarding equal marks regardless of individual input is likely to be contentious. If proceeding this way, staff must make their expectations explicit in the unit outline. An outcome should be that students learn to accept the (stated) rationale for the type of group assessment and work within the (explicit) guidelines or rules. Most students have only experienced educational environments where assessment is competitive and they are unlikely to understand that it can, or should, be anything but competitive. Staff will need to address students' beliefs and concerns about this.

Example 1:

The marks are determined by quality of the product and each student gets the same mark. For example, a group of students prepares a business plan for a company. The business plan is awarded a mark of 16/20. Each student thus gets 16 marks.

Example 2:

Where there is to be a separate group process mark, all students get an equal share. For example, the group preparing the business plans submits documents that provide evidence about how the group went through the process of researching and assembling the business plan. The group is awarded a mark of 7/10 for the process, so each student gets 7 marks.

Model 2: Students get a proportion of their marks according to individual contribution to the process

Where students' individual contributions to the process are to be assessed, there needs to be a way of doing this reliably and fairly. Students themselves are probably in the best position to make such judgments and can do so quite reliably provided they are given the opportunity and understand the criteria (Brown, Bull & Pendlebury, 1997; James, McInnis & Devlin, 2002b; Race, 1999). The process can be either by self-assessment or peer-assessment or both. The teacher or tutor may also have input. Various examples are outlined below - but each has limitations or associated problems.

A more comprehensive account of how this model might be implemented forms Appendix 1, at the end of this document.

There are two ways of determining the marks for individual contribution to process:

  • Each group gets a mark, determined by the quality of the product, to share or distribute among themselves (see Example 3); or
  • Each group is given a pre-determined mark, irrespective of the quality of the product, to distribute among themselves (see Example 4).

In both of these examples, there is a further variation in the determination of each student's mark - either a fixed number of marks is distributed among the students (variation A) or each student gets a proportion of a fixed number of marks (variation B).

Example 3(A): Distribution of a fixed number of marks

A group of four students prepares an environmental impact report. They get 24/30 marks for the report. They therefore get 96 marks (= 24 x 4) to distribute among themselves for individual contribution to the process of researching and preparing the report. Some students get more than 24 marks, and others less. Specified criteria and a distribution algorithm are needed.

The problems with this method are that:

  • there is effectively no maximum mark for a given student (unless it is capped at the maximum - 30 in this example);
  • high marks for one student reduces the maximum possible marks for the others;
  • it is unlikely that students from different groups, who make equal contribution to their respective groups, will get equal reward for effort; and
  • the tendency is for groups to allocate the marks equally, regardless of differing contributions.
Example 3(B): Proportional marks method

In the example above (3A), each member of the group would have available to them the full 24 marks i.e. they would each get a mark out of 24 for the quality of their contribution to the process, regardless of the mark allocated to the others in the group. A mark above the maximum is not possible. Specified criteria are needed (but no distribution algorithm).

The problems with this method are that:

  • it is unlikely that students in different groups, who make equal contribution to their respective groups, will get equal reward for effort; and
  • the tendency is for groups to allocate the marks equally, regardless of differing contributions.
Example 4(A): Distribution of a fixed number of marks

Groups of five students each prepare materials for, and deliver, a seminar. All groups are given 100 marks (20x5) to distribute according to individual contribution to the preparation process, regardless of the quality of the actual seminar. Thus some students will get more than 20 and some will get less. Specified criteria and a distribution algorithm are needed.

The problems with this method are that:

  • there is effectively no maximum mark for a given student (unless it is capped at some arbitrary maximum, which in this example might be 25);
  • high marks for one student reduces the maximum possible marks for the others; and
  • the tendency is for groups to allocate the marks equally, regardless of differing contributions.
Example 4(B): Proportional marks method

In example above (4A), each member of a group would have available 20 marks i.e. they would each get a mark out of 20, regardless of the mark allocated to the others in the group, for their contribution to the process. A mark above the maximum is not possible. Specified criteria are needed (but no distribution algorithm).

The problem with this method is the tendency for groups to allocate the marks equally, regardless of differing contributions. Each of these methods implies that students have criteria on which they know that they are to be judged, and that they use these criteria when peer- and/or self-assessing. Students will have a better understanding of the criteria if they are involved in deciding what they should be.

Model 3: Students get a proportion of their marks according to individual contribution to the product

This method is likely to be more controversial depending on how students have allocated (or how they have been directed to allocate) the roles or tasks that directly contribute to the product quality. If teams are working effectively, they will make use of the diverse skills and abilities of the members to maximise the quality of their product. To then penalise any one member because their role is not seen as important as another can be seen as discriminatory. One might also argue that this goes against the philosophy of 'teamwork'.

The assignment context must be one for which it is possible to differentiate between the different roles/tasks that students are required to undertake. The allocation of students to particular roles must be fair, and there must be performance criteria for each role. Students are then judged against the criteria for the role that they performed.

While there might be a number of possible ways of allocating marks, only the one described in example 5 is recommended. In this example, the contribution of each student can clearly be seen and verified.

Example 5:

A group of three students prepares an educational website. One student has the role of website designer/manager, another is the instructional designer and the third researches the content for the site. Each student has criteria for the quality of their aspect of the completed website, and is required to explain or justify their contribution.

The website is awarded 18/20 for web design, 15/20 for instructional design and 7/20 for content. Thus the students get 18, 15 and 17 marks respectively.

Model 4: Students get a proportion of their marks according to their individual knowledge about the process or the product.

An assessment variation, perhaps in addition to one of the previous models, might be that students provide further written evidence of their learning. They might be asked to write an account of their understanding of the process and/or their learning in a separate submission or in response to an examination question. They might also be asked questions that test their individual knowledge about the product. This can serve to support (or otherwise) students' claims about their contribution to processes or product. These will result in extra marking.

Evidence of an individual student's knowledge, can also take the form of reflective journals, purposeful portfolios, global or itemised criterion-based reports (usually proformas), documents (such as minutes of meeting) or working reports. Where portfolios of evidence or learning journals are used, students might need to learn how to collect and archive relevant evidence.

Example 6:

A group of six students undertakes a six-week research project on the geomorphology of a particular region. They will produce a final group report. To ascertain the contribution and learning of each member of the group, the students are required to submit a research diary recording their progress, relevant diagrams and printouts and findings at weekly intervals throughout the six weeks. In addition, there is a question in the exam that asks them to briefly describe the geomorphology of their area and to explain how their research method could have been improved.

7.4 Implementing group assessment

Whichever model is chosen, careful thought is required. It might be worth trialling a model with a small class before implementing large-scale changes. There are a number of ways and algorithms for apportioning marks for individual effort and input where this is deemed appropriate.

Three other issues that may arise are:

  • Allocating students to groups strategically;
  • Providing support for groups to keep them on-track; and
  • Dealing with situations in which groups are of unequal size

The most significant recent publication to support assessment in Australian higher education, including group assessment, can be found online at the Centre for Studies in Higher Education website or in hard copy (James et al., 2002b). This publication provides many further suggestions for pre-empting and managing problems related to group-based assessment.

Group work provides many benefits for students and for staff but it is a process that must be planned and implemented carefully. In making the decision to implement group-based assessment, consider the following:

  • Why are students to work and be assessed in groups in this unit?
  • What are the learning outcomes?
  • How will the groups be constituted?
  • What are the assessment criteria for meeting the learning outcomes?
  • How will the quality of each student's work be judged?
  • What other processes will be put in place to ensure that the assessment process is fair, equitable and manageable?

Return To Top

 

8. Using rubrics to communicate assessment standards

8.1 What is a rubric?

A scoring rubric is a table that makes clear to students the criteria against which their work will be assessed. Students can use it in developing, revising and judging the quality of their own work. Assessors can use it to assess students' performance either analytically or holistically (see p 21). Perhaps the most important function of a scoring rubric, however, is in providing both formative (ongoing) and summative (after marking) feedback to students and feedback to staff on students' learning and thus the effect of their teaching (Huba & Freed, 2000).

Assessment is a process that should be integral with student learning and not just something tacked on at the end of a period of learning. If assessment is viewed this way, well-constructed scoring rubrics can play a key role implementing an effective learning and assessment program.

In many instances, the use of a scoring rubric enables students to hand in work that is better in quality than they might otherwise have done. Students do not have to guess or infer what the assessor wants; and assessors are forced to articulate and, if necessary, quantify the most valued outcomes of students' learning (Stevens & Levi, 2005).

Well-constructed rubrics increase inter-rater reliability through improving comparability among a number of assessors, an important factor to consider when using more subjective assessment tasks. A rubric may be designed to be re-used-the task that the student undertakes, or the content of the task, may change but the scoring rubric may be the same. This ultimately can save a lot of time.

Rubrics are consistent with outcomes-focused education and support a learner-centred approach to teaching, and thus should help in changing our thinking about assessment at Macquarie in the future. The use of scoring rubrics is also consistent with the principle of making assessment processes open and accountable. The proviso is that the rubric is well-constructed, valid and reliable, and subject to ongoing evaluation.

Table 8.1: Example of a rubric
 Levels of Achievement
CriteriaAdvancedProficientFunctionalDeveloping
Accurately identifies constraints of obstaclesAccurately and thoroughly describes the relevant constraints or obstaclesAddresses obstacles or constraints that are not immediately apparent.Accurately identifies the most important constraints or obstacles.Identifies some constraints or obstacles that are accurate along with some that are not accurate. Omits the most significant constraints or obstacles.
Identifies viable and important alternatives for overcoming the constraints or obstaclesIdentifies creative but plausible solutions to the problem under consideration. The solutions address the central difficulties posed by the constraint of obstacle.Proposes alternative solutions that appear plausible and that address the most important constraints or obstacles.Presents alternative solutions for dealing with the obstacles or constraints, but the solutions do not all address the important difficulties.Presents solutions that fail to address crticial parts of the problem.

The dimensions of performance may also have sub-categories.

For the rubric to be complete, each aspect of quality must be elaborated with a clear description of defining features of work at each level of mastery. Sometimes this is just represented as the extremes of a continuum from 'unacceptable' to 'excellent', which is less informative since it leaves the student unaware of exactly what constitutes unacceptable, acceptable, good or excellent work.

8.2 How to develop a rubric

Table 8.2: Stages in the process of developing a rubric
StagesComment
Decide the Dimensions of Performance or essential elements that must be evident in high quality work [rows].Rule of thumb: If a student can score highly on all dimensions but not score well overall, you have the wrong dimensions. Revise. Discuss with colleagues.
Decide the levels of achievement - number and type [columns].Ways of describing various levels of mastery include:
  • advanced, proficient, functional, developing
  • sophisticated, competent, partly competent, not yet competent
  • exemplary, proficient, marginal, unacceptable
  • or others that you choose (between 3 and 5).
A number of unit conveners at Macquarie have decided to use the standard Macquarie grades (HD, D, C, P, PC, F) in their rubrics.
For each Dimension of Performance, distinguish between acceptable and unacceptable (failing) performanceWrite the criteria for acceptable performance clearly and unambiguously.
For each Dimension of Performance, write clear performance descriptors (criteria) at each achievement level.Try to determine qualitative and quantitative differences that characterise work or performance at the different levels. More on this below...
Include (if possible) the consequences of performing at each level.For example, the standard of the work would (or would not) be accepted by the profession or a business (as in a charter) or a professional journal (as in publication guidelines), etc.
Add the rating scheme you will use and apply any weightings.Consider:
  • awarding marks (analytical scheme) or not (holistic scheme). There are arguments for each of these approaches (see below). It's useful to discuss this with your teaching team to ensure a consistent approach.
  • including weighting criteria if required.
  • whether marks should be awarded for work below the minimum standard
  • the criteria for 'failure'.
Evaluate and revise accordingly.Few rubrics will be constructed perfectly the first time. They are developmental tools and need to be critically evaluated after use.

(Huba & Freed, 2000)

8.3 Differentiating performance levels

The hardest part about constructing a rubric is formulating and clearly articulating standards for different levels of performance (or levels of achievement).

Try to determine objective, qualitative or quantitative differences that characterise work or performance at the different levels. Avoid:

  • different grades of the same character (good, better, best etc)
  • undefined terms e.g. trivial work, good use of, significant work
  • value-laden terms e.g. excellent or poor work

These terms may have meaning for the assessor but don't tell the student the standard expected nor what they can do to improve.

We can educate students simply by using rubrics.

  • We reveal to students the standards of the discipline and the profession (and help to inculcate them into professional practice).
  • We inform students about the different qualities that comprise good (and poor) work.
  • We can involve students in setting the standards that they agree to be judged by.
  • We can involve students in using their own words to describe the criteria.
  • They provide a means of opening the channels of communication between lecturer/tutor and students.

It is possible for a number of people to give feedback to the students on his/her work - peers, other staff etc - emphasising the different points of view about a piece of work.

One of the more sophisticated but effective uses of a scoring rubric is to have the students collaboratively decide the assessment criteria. This makes them much more aware of the criteria and what they mean, as well as giving students greater 'ownership' of the process.

8.4 Holistic versus analytic rubrics

Holistic rubrics are a form of qualitative assessment while analytic rubrics are a form of quantitative assessment. Holistic rubrics allocate a grade or performance level to a student simply on the basis of performance against explicit standards (without resorting to marks). Analytic rubrics allocate a specific number of marks for each criterion and/or performance level. Holistic rubrics are becoming more accepted because they "accommodate more than concrete facts" (Dunn, Morgan, O'Reilly, & Parry, 2004) but are often undervalued by those from a scientific measurement background because they may be seen to be too subjective. Provided, however, that the quality criteria for qualitative assessment are applied, they can be as valid an assessment tool as are analytic rubrics (see end of this document for quality criteria).

The following is a simple quantitative approach to assigning marks to each criterion.

Score all items on a 4-point scale:

  • 0 = task or element not done
  • 1 = task or element done, but clearly incorrect
  • 2 = task or element done, but only partially correct
  • 3 = task or element done, and clearly correct
  • 0s, 1s and 3s will be clearly identifiable; anything else is a 2.

One issue is whether or not to award any marks to 'unsatisfactory performance'. It might be possible to accumulate enough marks on unsatisfactory performance to tip a student over the magical 50%. An alternative is to set a criterion for passing that states there must be no element for which performance is unsatisfactory (regardless of the overall mark). Another way to limit this is to allocate an 'overall performance' Dimension on top of all other Dimensions of Performance.

An example of a rubric from Engineering

The following is part of a rubric for an 'Engine Design' project in an Engineering course. The content of the rubric is somewhat independent of the content of the task (see Huba & Freed, 2000, p.159-160)

Table 8.4a: Example of a rubric from an Engineering course
 Levels of Achievement
CriteriaExcellent (A)

4 points
Good (B)

3 points
Needs Improvement (C, D)

2 points
Unacceptable (F)

1 point
Formulation of Design problem
Formulation and scope of problemDesign problem formulation is clear and well thought out. The problem scope is well definedThe problem formulation is clear, but the scope is not well-definedThe problem formulation is unclear in some respects and does not appear to be well thought outThe design problem is not formulated clearly
SignificanceThe problem chosen represents a current challenge facing the engine industry. The potential market is large and clearly identifiedThe problem represents a current challenge in the engine industry, but the potential market is small or is not clearly definedThe problem does not represent a current challenge in the engine industry, the market is small or is not clearly definedThe problem does not represent a current challenge in the engine industry. There is no explanation about who would be interested in the product or why. There is no evidence of the background work (eg market analysis) that is needed to design an engine.
Engineering skill utilisation
AnalysisEngineering analysis is detailed and challenging and is used at every stage of the design process.Engineering analysis is detailed and challenging but some steps do not appear to be supported by calculations.Some analysis is included, but it is not very detailed or challenging. Many steps are not supported by calculationsEngineering anlaysis is used infrequently. When used, appears trivial and leads to obvious conclusions.

An example of a generic (re-usable) rubric for Problem-Solving

Table 8.4b: Generic rubric for problem-solving
 Levels of Achievement
Criteria4 points3 points2 points1 point
Accurately identifies constraints of obstaclesAccurately and thoroughly describes the relevant constraints or obstaclesAddresses obstacles or constraints that are not immediately apparent.Accurately identifies the most important constraints or obstacles.Identifies some constraints or obstacles that are accurate along with some that are not accurate. Omits the most significant constraints or obstacles.
Identifies viable and important alternatives for overcoming the constraints or obstaclesIdentifies creative but plausible solutions to the problem under consideration. The solutions address the central difficulties posed by the constraint of obstacle.Proposes alternative solutions that appear plausible and that address the most important constraints or obstacles.Presents alternative solutions for dealing with the obstacles or constraints, but the solutions do not all address the important difficulties.Presents solutions that fail to address crticial parts of the problem.
Selects and adequately tries out alternativesEngages in effective, valid, and exhaustive trials of the selected alternatives. Trials go beyond those required to solve the problem and show a commitment to an in-depth understanding of the problem.Puts the selected alternatives to trials adequatel to determine their utilities.Tries out the alternatives, but the trials are incomplete and important elements are omitted or ignored.Does not satisfactorily test the selected solutions.
If other alternatives were tried, accurately articulates and supports the reasoning behind the order of their selection, and the extent to which each overcame the obstacles or constraintsProvides a clear, comprehensive summary of the reasoning that led t the selection of secondary soution. The descirption includes a review of the decisions that produced the order of selection and how each alternatives fared as a solution.Describes the process that led to the ordering of secondary solutions. The description offers a clear, defensible rationale for the ordering of the alternatives and the final selection.Describes the process that led to the ordering of secondary solutions. The description does not provide a clear rationale for the ordering of the alternatives, or th student does not address all the alternatives that were tried.Describes an illogical method for determining the relative value of the alternatives. The student does not present a reasonable review of the strengths and weaknesses of the alternative solutions that were tried and abandoned.

8.5 Evaluating a rubric

Rubrics should be reviewed each time they are used, and then revised accordingly. Questions that can be used to do this are:

  • Does it measure the learning outcome(s) that you want measured?
  • Does it measure ALL the important outcomes?
  • Does it measure unimportant/extraneous outcomes?
  • Does the rubric cover the important dimensions of student performance?
  • Are the performance levels and scales well-defined - does everyone understand them?
  • Is there a clear basis for assigning scores at each scale point?
  • Do the 'excellent' descriptors describe a high enough performance standard?
  • Is there sufficient distinction between each dimension
  • Can different scorers apply the rubric consistently?
  • Is the rubric fair and free from bias?
  • Is the rubric useful, feasible, manageable and practical?

8.6 Performance criteria

The following are examples of performance criteria drawn from selected scoring rubrics (all cited in Huba and Freed (2000)):

Criterion: Team skills-Group functioning

Table 8.6a: Performance criteria relating to Team skills - Group Functioning
ExcellentThe group functions well. Peer review indicates good distribution of effort. All members are challenged and feel their contributions are valued.
GoodThe group functions fairly well. Some people in the group believe they are working harder (or less hard) than others, but everyone is contributing.
Needs improvementThe group is still functioning, but each individual is doing his/her own work and ignoring the efforts of others. There are frequent episodes where one person's design will not fit with another's due to lack of communication.
UnacceptableThe group functions poorly. All work is the product of individual effort.

Criterion: Practising ethical standards of the chosen discipline

Table 8.6b: Performance criteria relating to Practicing Ethical Standards
ExemplaryActs congruently with, and advocates for, the ethical standards of chosen discipline.
ProficientActs congruently with the ethical standards of chosen discipline.
MarginalActs congruently with the ethical standards of the chosen discipline. Any violations are relatively minor.
UnacceptableViolates ethical standards of chosen profession. Violations are serious.

Criterion: Knowledge of past economic decisions

Table 8.6c: Performance criteria relating to Knowledge of Past Economic Decisions
ExemplaryYour bill reflects economic decisions which have worked in the past, is based on research involving comparisons of previous eras, and makes reference to actual events.
ProficientYour bill reflects economic decisions which have worked in the past, but your examples are limited. More examples would make your bill more easily understood by the reader.
MarginalYour bill makes little reference to precedents from the past. You need to add some to help the reader decide if the bill would work and should be passed into law.
UnacceptableYour bill makes no reference to precedents from the past. The reader is left wondering if you know what past economic and political practice has been.

8.7 Quantitative Assessment Standards

Validity

Assessment of a student's knowledge and skills usually results in an empirical 'indicant' i.e. a number (mark) or a grade that represents the knowledge and skills being assessed. Validity refers to the extent to which the indicant measures what it claims to measure. In educational terms, however, it is often not the indicant/measure that is validated but the purpose for which it is being used that is usually submitted to validation processes. Hence, a mark based on a student's recall of content knowledge cannot be used as a measure of the student's higher order skills, such as knowledge application.

Reliability

Reliability refers to both the accuracy and precision of measurement. Different tests of a student's particular knowledge or skills, if administered independently of one another, should all give the same result. Two different assessors should arrive at the same conclusion about a student's learning. The three facets of reliability refer to the basic rationale (can the knowledge and skills be translated into a measurement?), the procedures for data collection (the assessment tool, its administration and marking) and the statistical procedures following (what we subsequently do with the numbers).

Qualitative Assessment Standards

Good qualitative assessments are credible, dependable and confirmable. Assessment is credible when the form of assessment is aligned closely with its corresponding form of learning outcome. Assessment is dependable when subjective methods are applied consistently and consensually. Assessment is confirmable when an audit trail (documentary or electronic) is maintained to enable back-tracking to original criteria-based judgments (adapted from Guba & Lincoln's (1989) Fourth Generation Evaluation standards).

Return To Top

 

9. For further assistance

Your first point of contact should be your faculty's Associate Dean and Director in Learning and Teaching.

 

10. Further reading and resources

LTC resources

Other Resources

Policies and procedures

Reading

Biggs, J. (2003). Teaching for quality learning at university: What the student does (2nd ed.). Buckingham: Society for Research into Higher Education and Open University Press.

Bloom, B. S., Ed. (1956). Taxonomy of educational objectives: The classification of educational goals. Book 1: Cognitive domain. London, Longman.

Brown, G., Bull, J., & Pendlebury, M. (1997). Assessing student learning in higher education. London: Routledge.

Guba, E. G. and Lincoln, Y. S. (1989). Fourth generation evaluation. Newbury Park: Sage. Huba, M. E. & Freed, J. E. (2000). Learner-centered assessment on college campuses: shifting the focus from teaching to learning. Boston: Allyn and Bacon.

James, R., McInnis, C. & Devlin, M. (2002). Assessing learning in Australian universities: Ideas, strategies and resources for quality in student assessment. Melbourne: Centre for the Study of Higher Education.

Paulson, L. F., Paulson P. R., & Meyer C. (1991). What makes a portfolio a portfolio? "Educational Leadership," 48(5), 60-63.

Stevens, D. D and Levi, A. J. (2005). Introduction to rubrics. Virginia, Stylus.

Acknowledgement

The Learning and Teaching Centre thanks Curtin University for their permission to adapt this resource for Macquarie University. The original resource may be found at http://evaluate.curtin.edu.au/improving/.

Return To Top


This resource was developed by the Learning and Teaching Centre at Macquarie University.