Darllenwch y dudalen hon yn Gymraeg

We have today published the final 2017 Key Stage 4 results for secondary school pupils in Wales. At this time of year there is much interest in putting the national performance into context, whether that is by comparing Wales to another country or looking at trends over time. There have been a number of changes in recent years to the way performance measures are calculated and in this blog I will describe the impact that these changes have had on our ability to compare.

In Issue 9 (July 2016) I described how the changes to performance measures in both Wales and England had impacted on comparisons. These changes remain valid for 2017. In addition to the changes described in that blog, the independent Review of Qualifications recommended a further set of changes that have taken effect in 2017. These changes not only reinforce the issues around comparing to other countries, but also mean that comparisons over time are more difficult.

Changes introduced in 2017

Introduction of the Capped 9 Score

The Capped 9 measure focuses on Year 11 pupils’ results from nine of the qualifications available in Wales, including subject specific requirements. The main changes compared to the previous capped points score are:

• an increase from eight to nine GCSEs or equivalent volume of qualifications
• the subjects of English/Welsh, Mathematics – Numeracy, Mathematics and Science are now mandatory parts of the score.

Cap on contribution of non-GCSEs to threshold measures

From 2017, a maximum of two vocational (non-GCSE) qualifications will count towards all threshold measures, depending on size of the qualification. This means that if a measure is based on achieving a certain volume of qualifications across a range of subjects, vocational qualifications can no longer account for more than 40% of the threshold meaning that a greater weighting is given to GCSEs.

Literature in threshold measures

Literature qualifications no longer count towards the literacy elements in the ‘Level 2 inclusive’ measure or Capped 9 score from 2017 onwards, but can still count in the non-subject specific elements.

New Mathematics GCSEs

This is the first summer that pupils are expected to sit two GCSEs in Mathematics – GCSE Mathematics and GCSE Mathematics Numeracy.

How does this affect our ability to measure over time?

For the capped 9 points score, the mix of qualifications, and its influence on entry patterns and curriculum choice, mean that it is not possible to calculate this indicator for previous years. For example, the indicator requires both of the new GCSEs in Mathematics, which were not available prior to summer 2017.

For threshold measures (e.g. the ‘Level 2 inclusive’) the cap on the value of non-GCSEs has contributed to a shift away from vocational qualifications towards GCSEs, reversing the trends seen in some subjects in recent years.

Where possible we have tried to quantify the impact of these changes in our statistical release (e.g. the impact of removing literature from the “Level 2 inclusive”). However, most of these changes have an impact on entry patterns and curriculum choice and are ultimately associated with complex behaviour changes in pupils and schools which are not possible to measure.

How can we measure changes over time?

As suggested in the previous blog on this matter, international studies such as PISA, or  UK surveys such as the Annual Population Survey,  now provide the best opportunity to compare educational outcomes either between countries or over time. In addition, trends over time can be analysed by looking at performance within individual subjects where the specification of that subject has not changed significantly over time.

For how long will these changes impact on our ability to compare over time?

In 2018 we will introduce a further change to Key Stage 4 performance measures. Within the capped 9 points score, only GCSE Science qualifications will be accepted from this point onwards. We expect this to further influence curriculum choices and behaviour.

The Cabinet Secretary for Education recently announced that from the summer 2019 results onwards only the first entry for each pupil will be counted in official performance measures. Again we expect this to influence behaviour in schools in ways that cannot be measured.

Taking these future changes into account, and assuming that at least three years of data are needed to establish a general trend (more years would be better), we will not be able to use Key Stage 4 results to make valid comparisons over time until the summer of 2021 at the earliest.

Glyn Jones
Chief Statistician

06 December 2017