Archive
Changing the exam system
Author: Simon Burgess
It will take a while to fully understand the scope of the proposed changes to the exam system in England. The exams are obviously very important for students, but also for schools and teachers. Here are some initial thoughts.
A key pledge is to ‘restore rigour’: exams are going to be more rigorous (of course, England has always had at least one exam noted for its rigour. More rigorous might mean different things: it might simply mean more failures or more broadly, greater differentiation between students.
It seems inevitable that one implication of this (in the short run at least) is greater educational inequality. One outcome of making an exam more rigorous is that more students will fail it. It seems hard to see how that could not be true. This is not fatalism; it may well be that over time, teaching brings more students above the line, but there will be a number of cohorts first with more failures. A tougher exam will not be credible if more people pass it. This coupled with a hint of no retakes seems particularly harsh. ‘No second chances’ is an uncompromising view of how to run an education system, and perhaps at odds with much of the rest of public life.
An alternative sense of ‘more rigorous’ is that the new exam will allow for greater differentiation between students. This is as much to do with the marking and reporting as the nature of the exam itself. Rather than proliferating letters and multiple stars, one way to do this that has been mooted is to simply report percentages. More or less differentiation – separating or pooling in economics terms – has a number of implications. On the plus side, it may enable more efficient matching of workers and jobs, raising average productivity; it is likely to incentivise greater effort at school as students currently more-or-less assured of an A* grade push on to shoot for 90%+. On the down side, there will be greater inequality of attainment and consequently greater inequality of earnings. It is not obvious how sizeable these pros and cons are, but there is no guarantee that the outcome will be a happy one.
We have a school system described by the Department for Education as increasingly autonomous. Diversity of supply is the maxim, and there is reduced oversight of what schools actually do. But what brings the system together is the common exam system. Autonomous schools can do their own thing with less outside ‘interference’ … as long as its students achieve good grades. Schools’ freedom to operate is constrained by close public scrutiny of their performance in common high stakes exams. If there is less monitoring of the process of education, then regulation of the outcome (attainment measures) becomes even more important.
School performance tables, the ‘league tables’, and Ofsted are the central performance components of the regulatory framework. While Ofsted is important, it is largely driven by the school performance tables reporting exam scores. So getting the exams right is very important. Schools will react to changes in the regulatory environment as they will react to any incentives (implicit or explicit) embedded in the system. So in addition to its implications for students, the exam system will very strongly influence how schools operate. Changing the exam system is a big opportunity to get things right or to get things wrong.
One possibility would be to use the reform to move away from the current sharp threshold recording attainment, the fraction of students in a school getting at least 5 good passes. If the students are given percentage scores not letter grades (see below), the metric for the school could simply be the mean score. There is a lot to be said for not having a sharp threshold, or for locating it at the point that policy makers truly do want schools to focus. But this could be done now under the present system – GCSEs have continuous scores that are then converted to letter grades (controversially this year). Switching from a threshold to a continuous average is a separate issue to the rest of the proposed changes, and is not dependent on them.
The proposal to have one exam board per subject seems like a good idea. While monopolies may well raise costs, the nature of the competition between the exam boards was dysfunctional. And the idea of ‘competition to be a monopoly’ is well established (train companies, power transmission and other infrastructure companies) and reasonably well understood.
Focussing now on the students, what is the best form of tests for students? The proposals favour a return to a single exam for each subject at the end of the course, and the end of coursework, retakes and modules. Doing well in such exams will require a particular type of skill, so these are what students will try to master and what schools will try to teach.
What sort of skills should education foster? This is not a straightforward question. One plausible answer is that education should test and certify the levels of competence that students have achieved in the skills that employers want.
No one knows what skills business will need in ten years time, and so we can only speculate as to what we should grade students on. But given the ubiquity of the web, it seems very unlikely that ‘remembering large amounts of information and writing it down quickly’ will figure high up on the list. It is hard to see this as a highly prized capability. Some other much-lauded education systems (keen on rigour) test abilities more likely to be of value. A great deal of research now focusses on different cognitive and non-cognitive skills, how they are built up and how they relate to inequality. I don’t know if there is any evidence whether the ability to remember large amounts of information is more or less socially graded than broader ranges of skills.
One argument for one-shot, high-stakes, closed-book exams is that anything more open is susceptible to parental help, and thus more likely to favour the middle class. But there are other ways to make sure that the summative assessment is solely based on the student’s work alone. Furthermore, parents support their children’s education in so many ways (conversation; providing books and computers; a quiet place to study; role models; trips) and this is unlikely to be the most important. It is worth emphasising that parents helping their children to learn is a good thing, and should be encouraged where lacking, not banned where present.
This preliminary set of points suggests that the proposed reforms of assessment will not promote skills likely to be valued in the labour market. They are likely to lead to more students failing and leaving school with nothing, and/or delaying their one-shot take, or just dropping out. This spells greater inequality in educational attainment and so in life chances.
What is the value of education?
The UK education system has been firmly in the headlines. Whether it’s the Coalition Government’s Schools White Paper, The Importance of Teaching, or students protesting against the Government’s response to the Browne report (Securing a Sustainable Future for Higher Education in London) the critical issue of whether education is a public or private good is being discussed and debated rigorously, though not necessarily in those terms.
Of course, it’s both. Though one might question the (historic) figures bandied around to assert the wage premium a higher education degree is said to attract, and certainly discount the idea it will apply to everyone who passes through a University, it must be true that one of the motivations that attracts students to University study is the prospect of a well-paid job. Nothing wrong with that. Indeed, it is often the transferable skills and training that Universities offer that make them an attractive proposition for internal as well as overseas students. On the other hand, at a time when the talk is of ‘the Big Society’ and about measuring happiness not merely economic output as a measure of the country’s progress, it would be odd to stop valuing learning as a social and cultural good in its own right. There may not always be an immediate and tangible economic return but were Universities ever intended to be just the training partners of industry, business and commerce?
Meanwhile, the new academies and free school programmes outlined by Michael Gove MP, Secretary of State for Education, have reignited the debate about social polarization if the best performing schools opt-out of local authority governance and, it is assumed, begin to attract the better (meaning middle-class) pupils who then receive a better funded educational experience than those left “trapped” in the less desirable schools. Here again, we encounter the question, who is education for? The individual recipients? Society as a whole? Or, both (in which case, how are they balanced?).
In A Journey, Tony Blair staunchly defends the New Labour policy of promoting school choice. The basic argument is who is government to hold back those who want to innovate, to be successful, to want the best for their children? It’s compelling but critics of those and current policies might argue they are individualistic. They miss the broader social point that it is not a level playing field: those with the resources and influence to do so, it is alleged, are best able to capitalise on the system. They will be the “winners” and inequalities will grow. And inequalities, according to the authors of the much publicised and debated book The Spirit Level are socially damaging.
But is it true? Have policies of school choice, partial marketisation and competition actually raised levels of social polarization? For schools the evidence remains unclear: see past CMPO papers and a forthcoming CMPO working paper reviewing the literature to date. In Higher Education, University education is not the preserve of the elite and the wealthy. In fact, the traditional image of the ivory towers where fresh-faced young students travel from the Home Counties along the M4 to study for three years away from home increasingly is misleading. Many students are part-time, mature and live at home. And, though young people from more affluent areas are still more likely to attend University, young people from disadvantaged areas have been substantially more likely to enter higher education since the mid-2000s (see Trends in young participation in higher education: core results for England, HEFCE, 2010). It would also not be unreasonable to suggest that the professionalization of teaching with Universities, though still not as strongly valued as it should be, is driven in response to the raised expectation of fee-paying students.
Yet, that was before a near three-fold increase in fees that will see the Aimhigher programme (promoting widening participation) axed and which will make the English University system the most expensive in the world, saddling students with a debt for up to 30 years. Quite what the effects on the housing and mortgage markets will be when graduates must first pay a fixed proportion of their salary to repay their student fees can only be guessed at. Will it be the case that only those from the “best schools” will be able to afford to go on and attend the “best universities”? Only time will tell.
In the meantime, the debate continues. What is that value of education? And who or what gains from it?
Paying to fail?
Today, a very special education policy experiment was revealed.
In the past, policies have been introduced aiming to incentivise schools and teachers to raise educational attainment. These have been effective to a degree: our evidence shows that performance pay for teachers does raise educational attainment; competition among schools also has some impact, albeit much weaker.
This new policy incentivises students themselves. And in a break from past policies, this scheme directly incentivises students to fail their exams. It is reported that Blackburn College will pay £5000 to each student who fails her/his exams. This intriguing new policy will certainly add to the research evidence on how (not) to raise attainment.
This issue is taken seriously in the US with a number of landmark policy experiments raising attainment for some of the more deprived and low-attaining groups in the country. At Harvard, Roland Fryer reports on the results of a large scale experiment in which students were incentivised in different ways. In some schools students were paid on results, and in some schools they were paid for activities leading towards better results, such as attendance and completing homework. The results were mixed, but the latter class of experiments were effective and cost-effective. Similarly, C. Kirabo Jackson at Cornell has shown that the Advanced Placement Incentive Program in Texas produced some very exciting results from paying 12th grade students for test-passing scores. Such students are more likely to attend college, do better when they attend college and are less likely to drop out. Similar experiments have taken place in the Harlem Children’s Zone
These experiments give us an idea of the value of a policy to incentivise student achievement. As far as I know, Blackburn’s policy is the first chance we have had to study the value of a policy to incentivise student failure.
More seriously, the idea of a commitment device is standard: something that penalises the provider if something does not work out as planned. A long warranty on a car is one way of the manufacturer raising the cost to itself of the car failing. But in a case such as studying for exams, where student effort is so hard to observe, and where good or bad luck can play such a role, what economists call the “moral hazard” problem is very severe.
There are obvious alternatives – the College could pledge to give £5000 to a local charity for every student that fails the exam. That would still appropriately hurt the provider for failure on their part, without giving marginal students a very high temptation for failing at the last.
Visit the CMPO Website
