Archive

Archive for September, 2013

Threshold measures in school accountability: asking the right question

September 25, 2013 Leave a comment

Author: Simon Burgess

Threshold measures in school accountability: asking the right question

We are in the midst of a significant upheaval in the setting and marking of exams, and the reporting of school exam results. One feature of the system has been the centre of a lot of criticism and highlighted for reform: the focus on the percentage of a school’s pupils that achieve at least 5 GCSEs at grades C to A*, including the scores on English and maths. This is typically the most-discussed metric for (secondary) school performance and is the headline figure in the school league tables.

The point is that this measure is based on a threshold, a ‘cliff-edge’. Get a grade C and you boost the school’s performance; missing a C by a lot or a little are the same, and just scraping  a C is the same as getting an A*.

This has been described as distorting schools’ behaviour, forcing schools to focus on pupils around this borderline. The argument is seen as obviously right and strong grounds for change. In this post I want to make two counter-arguments, and to suggest we are asking the wrong question.

First a basic point. One central goal of any performance measure is to induce greater or better-targeted effort. This might just mean “working harder” or it might mean a stronger focus on the goals embodied in the measure at the expense of other outcomes. The key for the principal is to design the best scheme to achieve this. A very common scheme is a threshold one – this can be found for example in the Quality and Outcomes Framework for GPs, service organisations with a target number of clients to see, and of course schools trying to help pupils to achieve at least 5 grades of C or better. An organisation working under a threshold scheme faces very different marginal incentives for effort. Considering pupils: the most intense incentives relate to pupils just below the line: this is where the greatest payoff is to schools to devote the most resources.

The first counter argument starts by noting that the asymmetry in the incentive is not a newly-discovered flaw, it is a design feature which can be very powerful. If there is a level of achievement that is extremely important for everyone to reach, then it makes sense to set up a scheme that offers very strong incentives to do that – that focusses the incentive around that minimum level. This is precisely what a threshold scheme does.

So rather than simply pointing out that threshold designs strongly focus attention (which is what they’re supposed to do), the questions to ask are: is there some level of attainment that has that characteristic of being a minimum level of competence? And if so, what is it? If society feels that 5 grade C’s is a fair approximation to a minimum level that we want everyone to achieve, then it is absolutely right to have a ‘cliff-edge’ there because inducing schools to work very hard to get pupils past that level is exactly what society wants.  It may be that we are equally happy to see grades increase for the very brightest children, those in the middle or those at the lower end of the ability distribution. Or not: all the main political parties express a desire to raise attainment at the lower end and narrow gaps.

The argument should be about where to put the threshold, not whether to have one or not. Perhaps we are starting to see a recognition of this in the recent policy announcement that all pupils will have to continue studying until they have passed English and Maths.

The second counter-argument is based on a scepticism of what is likely to happen without the 5A*-C(EM) threshold acting as a focal point.

The core strategic decision facing a headteacher is how best to deploy her main resource: the teachers. Specifically: how best to assign teachers of varying effectiveness to different classes. It has been said that schools will be free to focus equally on all pupils.

Well, maybe. Or perhaps we should think of the pressures on the headteacher, in this instance from teachers themselves. Effective teachers are very valuable to a school and any headteacher will be keen to keep her most effective teachers happy and loyal. It seems likely (I have no evidence on this, and would be keen to hear of any) that top teachers would typically prefer to teach top sets. If so, we might see a drift of the more effective teachers towards the more able classes in a school (and therefore on average, the more affluent pupils). The imperative of the C/D threshold gave headteachers an unanswerable argument to push against this.

So threshold metrics have an important role to play in communicating to schools where society wants them to focus their effort. The current threshold, at 5 C grades, may or may not be at the right level; but discussing what the right level is, is a more useful debate to have.

Is education policy a blunt instrument when it comes to ‘social mobility’?

September 20, 2013 Leave a comment

Author: Matt Dickson

Is education policy a blunt instrument when it comes to ‘social mobility’?

Earlier this week, Tony Blair’s former speech-writer Philip Collins told a fringe meeting at the Liberal Democrats conference that social mobility was a ‘terrible objective’ and that in any case, education policy could do little to affect it.

“I can’t think of a single education reform in the 20th Century that had a marked impact on relative social mobility at all. Not one,” he remarked.

This conclusion depends on who you think it is important to be “relative” to. On the one hand you might think it is important to be compared to your own parents i.e. where you started, on the other hand you could think it is important to be compared to your peers – where you sit in the distribution compared to your peers from different backgrounds. Let’s think about the former comparison.

The 1972 raising of the minimum school leaving age (RoSLA) has been shown in numerous pieces of research to have increased the education, employment and earnings of the young people affected – relative to their school-mates in the years before the reform. Given that we know that the people who were made to remain in school an additional year were disproportionately from lower socio-economic backgrounds, this policy improved the economic position of young people at the lower end of the economic scale.

“The dull child of the middle class parent has to come down the wrung in order for me to go up, otherwise you don’t have social mobility,” is another problem that Collins identified with the objective of social mobility.

However, nobody had to come down the earnings or education ladder in order for the young people affected by RoSLA to move up – so this policy improved the chances that young people with low taste for education and/or lower ability and from poorer backgrounds, would gaining qualifications, employment and greater earnings. Technically this would be considered “absolute social mobility” and Collins is right in making the assertion that for there to be upward “relative social mobility” there needs to be an offsetting downward move of some.

But Collins is taking a very strong line here – arguably, what we should care about as a society is the extent to which people from all backgrounds can maximize their potential and not have their opportunities curtailed purely because of their parents’ education, income or class. This encapsulates what ‘social mobility’ is all about – and why it remains an important objective.

Moreover, it is an objective that is amenable to policy, as demonstrated by the impact of RoSLA and other education policies of the last fifty years. Another major structural reform in the post-war era was the abolition of selective education in most of the country. Despite on-going controversies, we know that the grammar school system was detrimental to the majority of children from poor households and its ending reduced a major source of income-based differentiation in life chances.

Furthermore, the expansion of higher education in recent decades has seen increases in young people from poorer backgrounds accessing university and the opportunities for progression that this affords. A study by the Institute for Fiscal Studies for the Nuffield Foundation last year showed that while higher education participation has been rising in general over time, it has been rising quickest for young people from the poorest families. This represents genuine ‘social mobility’, driven by a reduction in the educational inequality that separates children from better off and poorer backgrounds.

Taking a longer perspective, one hundred years ago most pupils left school aged 12, People “knew their place” in society and the education system offered very little means of escape for children from poorer families. While the labour market has also changed dramatically since those days, it seems very unlikely that education policy and the revolution in secondary education in particular has had no effect on the chances for poorer pupils of getting on in life.