The Standards for RTOs 2015 set the requirements and benchmarks for RTO's Assessment Practices. All RTOs must develop and implement a system to ensure:
- Assessment judgements are sound and consistent
- Assessment judgements are validated.
An assessment system includes not only the actual materials used directly in conducting assessments (assessment tools), but also policies, procedures and other supporting documents used to implement the assessment plan.
In the Standards, clauses 1.9-1.11 refer to Assessment Validations and set the minimum requirements for the review of the RTO's assessment practices and judgements.
Validation involves checking that your assessment tools have produced valid, reliable, sufficient, current and authentic evidence – evidence that allows your RTO to make reasonable judgements about whether training product requirements have been met.
The validation process involves reviewing a statistically valid sample of the assessments and making recommendations for future improvements to the assessment tool, process and/or outcomes if applicable.
The validation process also includes acting upon any recommendations for future improvement.
As part of validation, your RTO must have a documented plan, which describes:
- Who will lead and participate in the validation activities
- Which training products will be the focus of the validation
- When assessment validation will occur, and
- How the outcomes of those activities will be documented and acted upon.
Who conducts validations?
Validation can be undertaken by one person or by a team of people. The RTO must ensure the people who complete the review process collectively hold:
- Vocational competencies and current industry skills relevant to the assessment being validated
- Current knowledge and skills in vocational teaching and learning, and
- The TAE40110 Certificate IV in Training and Assessment (or its successor) or the TAESS00001 Assessor Skills Set (or its successor).
Validators can be employees of your RTO, or you can seek external validators such as representatives from: local employers, industry bodies, trainers/assessors, or external consultants.
Understanding industry and VET teaching requirements
Validation helps ensure your RTO's training and assessment practices are relevant to the needs of industry. The individual(s) conducting the validation must have current skills, knowledge and experience in the vocational area and can provide input on:
- The industry relevance of the context and conditions of the assessment
- The industry relevance of the resources used during assessment
- The tasks the learner completed, and
- The judgements made by the assessor.
Managing the assessment process requires high technical skills in competency-based assessment, and requires applicable knowledge of the principles of assessment, rules of evidence, and quality management of assessment outcomes. The validator, or someone within the validation team, must provide that expertise.
A trainer or assessor who delivered/assessed the training product being validated:
- Can participate in the validation process as part of a team
- Cannot conduct the validation on his/her own
- Cannot determine the validation outcome for any assessment judgements they made, and
- Cannot be the lead validator in the assessment team.
Scheduling validations
Your RTO will need to develop a schedule to validate each training product on its scope of registration (AQF qualification, skill set, unit of competency, accredited short course and module).
Validating qualifications
If you are validating a qualification, validate the assessment practices and judgements from a sample of the units of competency within that qualification. According to ASQA's official interpretation of the Standards, at least two units of competency should be sampled when validating a qualification.
For quality purposes, the number of units to validate at any time during the validation process can be expanded depending on risk factors associated with particular unit(s), previous internal audit reports, and previous validation outcomes that indicate assessment judgments are not valid.
Validating units of competency
If the RTO has particular units of competency on its scope of registration, the RTO must validate all these units.
Validation Schedule
A validation schedule is a five-year plan; each training product must be reviewed at least once in that five-year period. At least 50% of the training products must be validated in the first three years of the schedule. Again, this is the minimum requirement according to the Standards but this frequency can be increased depending on risk factors associated with particular unit(s), previous internal audit reports, and previous validation outcomes that indicate assessment judgments are not valid.
Your validation schedule must reflect your current scope of registration. Adjust the validation schedule when adding a new training product. When making adjustments, ensure your schedule continues to meet the timeframe and completion requirements discussed above.
Statistically valid sampling
You must validate enough assessments to ensure the results of your validation are accurate, and are representative of the total completed assessments for the training product.
To determine appropriate sample sizes for assessment judgements you can use a sample size calculator tool. ASQA has recommended Raosoft tool and is currently developing its own tool.
Error level
The error level relates to the assessment outcome results. To determine an appropriate sample size, you need to consider how likely it is that the sample assessment outcomes will be a good representation of the total assessment outcomes.
The margin error, or risk that the validation sample size will not represent the total assessments, is not high. The recommended/default error level is 15%.
Confidence level
The confidence level relates to the assessment judgements. The confidence level is how sure you need to be that the sample assessment judgements will produce an accurate validation outcome.
When there are risks that the validation outcome of a sample of assessments will not be reflective of the total assessments, you need to increase the confidence level.
When there are similarities in the factors that influence the assessment judgements, it can be assumed that validation of a smaller sample of these assessments will still produce an outcome that is representative of all assessment judgements.
When sampling, the generally accepted confidence level is 95%. If you want to increase the confidence level, and obtain a larger sample size, any variation between 95% and 99% is appropriate.
Number of assessment judgements
This is the total number of assessment judgements made in the training product you are validating, within a set period of time.
Determine the period of time to be considered based on your training and assessment strategies and timetabling. Consider assessment judgements made over a period of at least six months; this aligns with the retention requirements described in ASQA's General Direction—Retention requirements for completed student assessment items.
For example, if your RTO has assessed 150 learners against the requirements of a unit of competency in the last six months, you would enter '150' as the number of assessment judgements.
Random selection
Samples of judged candidate evidence may be randomly selected using a number of different mechanisms, for example, using an alphabetical list.
Random selection—alphabetical list example
Produce an alphabetical listing, sorted by surname, of all of the learners who submitted work within the training product being validated. From this list, highlight the fifth surname and then every third name thereafter. Go back to the beginning of the list if necessary to identify a sufficient sample size.
Those highlighted then become the randomly selected learners whose work is submitted for validation.
Using Validation outcomes
Validation outcomes should identify recommendations for improvement to the assessment tool, assessment process or assessment outcome. Validation outcomes can identify critical issues in the collection of valid evidence.
Should this occur, you may:
- Increase the validation sample size to assist in identifying patterns of issue
- Validate completed assessments from other units of competency to see if the issue is spread across the whole of the qualification, and
- Look for patterns of error (for example, consider if it is one assessor making invalid judgements – this could indicate the assessor requires further training in competency-based assessment).
Your validation plan must clarify how you will document and act on validation outcomes.
Standard assessment validation record forms should be used to document the validation outcomes, and show to what extent the RTO's assessment system follows the principles of assessment and that the assessment evidence meets the rules of evidence.
It is critical for the RTO to link validation outcomes with the RTO's continuous improvement processes.
Recordkeeping requirements include evidence of:
- The person/people leading and participating in the validation activities (including their qualifications, skills and knowledge)
- The sample pool
- The validation tools used
- All assessment samples considered
- The validation outcomes, and
- How validation outcomes were use.
Recommendations for a validation must be used to improve the RTO's assessment system. For example, if the validation outcomes recommend improvements to a particular assessment tool (from the sample unit), you should implement these recommendations across all training products, not only those included in the sample.
Planning, participating and leading Validations
Insources delivers a webinar explaining the requirements unders the Standards for RTOs: Conducting Validations, and a full-day workshop: Lead Assessment Validations - for those who want to develop the skills and knowledge required to lead a Validation.
Do you need help? Insources In-house Validation Services
As part of the In-house Validation Service, an outsourced Vocational Education and Training expert will lead your validations and the complete system to be used for validation including: Policies, Procedures, Forms, templates, guidelines, and Training for your staff members.
Reference:
- Standards for RTOs 2015
- Users' Guide to the Standards for RTOs 2015, ASQA
