Quality Assurance Testing
Due to the diversity and complexity of vocational qualifications, any attempts to specify a detailed quality assurance framework for each possible permutation of features would be overly complex and prone to failure. Even within qualifications of apparently the same ‘type’, there is still some diversity. Therefore, Agored Cymru applies a principles-based approach to quality assurance which maximises the most-trusted evidence; ensures fairness to learners; maintains the integrity and credibility of the qualification and fits an overall ‘estimation blueprint’, which includes the submission of Centre Assessment Grades (CAGs), quality assurance testing and an overall check on outcomes.
Whilst our planned approach to quality assurance aligns with some aspects of standard external quality assurance activity, much of the approach is new, and bespoke templates for recording decisions have been developed to reflect this.
Where appropriate to do so, centres have been provided with a spreadsheet to capture learner grades based on their judgement for either individual units or full qualifications. It is the responsibility of the centre to ensure that no unconscious bias has been introduced to the process that may disadvantage learners who have protected characteristics.
In cases where there is little or no banked unit data, or where CAGs are not possible, and/or there is little opportunity or evidence to support the generation of CAGs or the quality assurance process, we may determine that this is below the Minimum Evidential Threshold. In these cases, we expect it will not be possible to calculate a result. Our expectation is that in these circumstances, learners affected will be offered an adapted assessment wherever possible, or will have to wait until assessments can be rescheduled.
It is important that centres retain information on their rationale for arriving at a calculated grade as this may be requested during our quality assurance checks to enable us to sufficiently verify decisions. Centres must be prepared to provide this information.
The completed spreadsheets are checked to compare centre activity to the same period in previous years. The centre will judge the grade each learner would have most likely achieved had they been able to take their assessments in summer 2020.
On submission of the completed application for estimation, Agored Cymru will conduct a process of external quality assurance testing which uses the information provided. This will help confirm that outcomes are as fair as possible to learners; help our decision making about certification, and ascertain whether a centre’s estimates are consistent with outcomes in previous years. The information required from centres will also allow adjustments to estimates where necessary, to ensure consistency across the qualification, and to compare against previous centre outcomes. Where necessary, and to ensure the timely certification of learner achievement, Agored Cymru may decide to approve claims on condition of retrospective external quality assurance (EQA) where appropriate.
Agored Cymru will quality assurance centre judgements by:
- checking the minimum evidential threshold has been met (at least one trusted source of evidence)
- considering other forms of evidence with a lower weighting/level of trust
- reviewing historical centre profiles and comparing them to the submitted centre assessed grades (CAG)
- reviewing the risk profile of the centre
- reviewing the risk banding of the qualification
- applying a proportionate sampling strategy, based on the available evidence and the outcomes from previous external quality assurance sampling strategies
- reviewing the overall qualification level outcomes as part of a cross-centre moderation and standardisation activity for each suite of qualifications.
Management of Risk
Malpractice remains a serious risk which is potentially increased while the Extraordinary Regulatory Framework (ERF) is in force. Particular risks include fraud and bribery; false certification claims and falsified evidence. Agored Cymru remains vigilant to the risks of malpractice and other improper behaviour.
Condition A6 of the Standard Conditions of Recognition continues to apply in respect of VQs. Agored Cymru will take account of the changed risk profile with respect to the VQs that it makes available.
As part of the consideration of the changed risk profile under Condition A6, Agored Cymru will treat the potential for abuse of Condition VQCov2.1(a)(ii) as a risk which could give rise to an Adverse Effect. As part of the reasonable steps taken under Condition A6.2, Agored Cymru will monitor registrations to identify any unusual patterns of entry to ensure that there is a legitimate reason for any increase in a centre's entries or that the process is not being abused in any other way.
Agored Cymru will investigate any alleged maladministration or malpractice during the summer 2020 results process. This includes a review of irregularities such as late learner registrations and increased volumes / achievement, as well as investigating where adaptions have been applied by a centre without approval to do so.
Under Condition I1 of the Standard Conditions of Recognition, an awarding body must establish, maintain and comply with an appeal process in relation to all of its qualifications. This obligation continues to apply in respect of results issued under the ERF Conditions. Condition I1 allows an awarding body to structure its appeal process as it considers best, providing that it meets the minimum requirements set out in the Condition.
Agored Cymru will allow appeals on the basis that it has used incorrect information or that procedures were not followed properly and fairly in arriving at a calculated result.
Qualifications Wales has confirmed that there should not be an opportunity for learners to challenge their Centre Assessment Grade through an appeal. However, there may be rare cases where a learner considers a Centre Assessment Grade was demonstrably affected by bias or discrimination. If there is evidence of actual bias, which the learner could provide, then the learner must be able to raise this with the centre. Agored Cymru may investigate such evidence as indicating malpractice.
Agored Cymru would consider correcting results where an investigation concludes that these are incorrect as a result of malpractice or maladministration.
Agored Cymru will allow appeals where a centre believes it has made an error when submitting its information; or similarly, if a centre believes that we have made a mistake when calculating, assigning or communicating a grade.
An appeal brought on the grounds that a centre made an error in the data it submitted to Agored Cymru must be supported by clear evidence that an error had been made by the centre.
Centres must have a procedure which allows learners to request that a review is conducted of the centre’s decision not to appeal to Agored Cymru.
Results (calculated grades)
We expect the results of our qualifications awarded by calculated results under the ERF to be issued no later than A Levels and GCSEs, that is the 13th and 20th August 2020, respectively. This is so that no learner who is hoping to progress onto further study or higher education is disadvantaged.
It is important to note that centre assessment grades are provisional until Agored Cymru issue results. Centres must not communicate provisional results to learners until certificates have been issued by Agored Cymru.