By: Simon Burgess
One of the things to come out of the response to day before yesterday’s blog was a clear desire to hang on to the ‘London Effect’. This is the belief that the much higher GCSE scores in London than the rest of England are the result of some policy or practice in the capital’s schools.
As I show in the report and as Chris Cook demonstrated here and before, this can be done by focussing on a subset of GCSE exams and taking that as the outcome measure. This is currently the only way that that belief can be supported.
I will come back to this below.
But here is the main point: I think we are in danger of missing the big issue by focussing on a debate over vocational qualifications. In this rush to hang on to the effects of a slightly mysterious policy, we are just marching past a demonstrable achievement of London. Sustaining a large, successful and reasonably integrated multi-ethnic school system containing pupils from every country in the world and speaking over 300 languages is a great thing. The role of ethnic minorities in generating London’s premium shows that London is achieving this. How many of those are there? I don’t know enough about school systems around the world to say, but I’d guess it’s probably unique.
To my mind, this is a fact worth celebrating about the London education system.
Having dwelt on that thought, here are two points on vocational qualifications.
First, the comparison of London and the rest of England, with and without vocational qualifications (VQs), and across ethnic groups is quite complex. Here are some things that are not true and some that are true.
1. It is not the case that White British pupils make better progress than pupils from ethnic minorities if we exclude VQs. The performances on this measure line up very strongly with performances on the regular measure, as shown in Figure 1:
2. It is not the case that pupils from all ethnic minorities do more VQs than White British pupils. Some do, some don’t: see the final column of table 1. Bangladeshi, Black African, Chinese and Indian ethnicity do fewer VQs than White British pupils; Pakistani and Black Caribbean pupils do more.
3. But it is the case that pupils of each ethnicity did more VQs in schools outside London than inside. See the first two columns of Table 1 above. All these differences are statistically significant. You can also see that in Figure 2 below, which also shows that outside London pupils did slightly more entries overall, meaning less study time per subject.
4. And it is the case that if we do exclude VQs, pupils from each ethnic group score higher on this progress measure in schools in London than outside. See table 2 below.
So the key question is: why did schools in London enter their pupils for far fewer VQs? Was this a city-wide policy decision? Or more informal but still common across London?
This is the question to you.
There is more on equivalents etc in a useful blog by Dave Thomson of FFT. He finishes with the same question.
Second, is it legitimate to decide now, after the fact, that some qualifications count less or not at all in a measure of what schools do? Obviously some vocational qualifications were severely Wolf’ed but not all.
Chris Cook and I have swapped analogies on this:
Chris: It’s a 110m hurdles race and we let schools choose their own hurdles, and then only look at their times at the finish. But we know some chose lower hurdles.
Me: athletes running a 400m race to find a winner. After the race, someone says that actually the proper test of run ability is just the first 200m so we will declare the true winner as the person ahead after 200m.
I am sure there is some truth in both (Any one coming up with a shot put-based analogy wins a prize.)
Finally, a thought about why there is this desire to hang on to the London Effect. If the higher GCSE progress had been the result of a policy (about 8 GCSE grades or 9.8% of an SD more technically) then it would be one of the best large scale policies ever. One reason I guess why people want to believe in it is that we lack a portfolio of proven, large scale public policies to raise educational attainment. Reiterating my comment from the paper, there are no innate differences in ability between pupils from different ethnic groups, but higher aspirations and expectations and perhaps a strong social network encouraging success matter a great deal. How to encourage that in groups where it can be absent is currently unknown; that is where the research frontier is.
by: Simon Burgess
Urban areas are often associated with poor educational attainment. But London is different. Recent analysis suggests that the attainment and progress of pupils in London is the highest in the country. A leading education policy commentator argues that: “Perhaps the biggest question in education policy over the past few years is why the outcomes for London schools have been improving so much faster than in the rest of the country”. Some have argued that this is the result of policies and practices adopted by London schools. If so, identifying the key policies is a great prize, with the hope that they can be implemented more broadly. Another recent report emphasises more the importance of primary schools.
In this research I have set out the evidence that a big part, almost all in fact, of the answer lies in the ethnic composition of London’s pupils. More broadly, my interpretation of this leads to a focus on pupil aspiration, ambition and engagement. There is nothing inherently different in the educational performance of pupils from different ethnic backgrounds, but the children of relatively recent immigrants typically have greater hopes and expectations of education, and are, on average, more likely to be engaged with their school work. This is not by chance of course; a key part of the London effect is its attraction to migrants and those aspiring to a better life.
There are two key and indisputable facts that lie behind this: ethnic minority pupils make better progress through school than white British pupils do, and ethnic minority pupils make up a much higher fraction of pupils in London. London also has a lot more recent migrants into the country. We showed some time ago that ethnic minority pupils make better progress to GCSEs than white British pupils. Given that these pupils typically live in more disadvantaged neighbourhoods and come from poorer families, their advantages must be less material than books, museum visits and computers. It is argued that ethnic minority pupils have greater ambition, aspiration, and work harder in school.
Put simply, this is the story: London has more of these pupils and so has a higher average GCSE score than the rest of the country. In this research, I put these two facts together and show that this accounts for almost all of the London effect.
The best way to measure what a school, or a city-wide school system, adds to its pupils is to look at pupil progress. This is what I do here. I focus on progress through secondary school, partly because the attainment measure at the start of secondary school is better than that for primary, and partly because others are focussing on primary schools. Progress is simply and standardly defined as capped GCSE points score relative to Keystage 2 scores.
What do we see? First, there is a London premium in pupil progress of just over 8 GCSE grade points, where a grade point is the difference between an A and a B, or a D and an E, etc. (for international comparability this is 9.8% of a standard deviation (SD). This is a huge number: the difference between say getting 8 C’s rather than 8 D’s. Given this, it is unsurprising there has been speculation as to what lies behind it. There is also a very substantial gap in the fraction of pupils progressing to at least 5 A*-C grades, 2.5 percentage points more in London.
Second, ethnic composition matters a great deal. In fact, differences in composition account for all of the gap in the progress measure. If I assume that London had the same ethnic composition as the rest of England, then given the progress of each ethnic group in each place (London, not London), there would be no ‘London Effect’. This decomposition is discussed in the paper, but an easy way to see this is a simple regression. Estimate the ‘London effect’ over all the pupils in all state schools in England, first not taking account of ethnicity and then adding those controls in, and Figure 1 (in SD units) shows that the effect is wiped out.
If I analyse conditional progress – taking account of personal circumstances such as poverty, gender, month of birth etc. – a London premium of 11% of an SD is also entirely eliminated by controls for ethnicity; this is also robust to conditioning on neighbourhood disadvantage as well. The Figure also shows that there is no significant difference between the progress of white British pupils in London and in the rest of the country, nor between pupils of Asian ethnicity in London and outside.
Third, it may well be that this effect is not a pupil’s ethnicity per se, but a characteristic that is correlated with ethnicity for some ethnic groups – being the child of immigrants. There is no educational data with immigrant status, so we have to use imperfect approximations. One way is to look at language – whether the pupil speaks English at home, or whether the pupil has English as an additional language (“EAL”). I also try another approximation in the paper.
Figure 2 shows again the impact on the estimated London Effect of controlling for language:
Of course, there has been change over the last decade. In fact, repeating this analysis it is clear that the London progress premium has existed for the last decade and is statistically accounted for by ethnic composition in each year. Again, it is easy to see why: London has seen a relative decline in its fraction of low-scoring pupils and a corresponding increase in high-scoring pupils:
There are two important cases in which accounting for ethnicity reduces (halves) the London premium but does not entirely eliminate it. If we consider GCSE points excluding vocational qualifications, then there remains a small but significant London effect. This in turn arises because pupils in London were entered for significantly fewer of these ‘equivalent’ qualifications. The implications of this are unclear. It is certainly not appropriate to simply remove some subject scores from the total and claim that the remainder represents ‘true’ progress. Nevertheless, the fact that London schools systematically entered pupils for less of these qualifications may represent the outcome of a particular policy.
Second, a measure of very high exam performance also yields a small but significant London effect once ethnicity is included. While this may also be the result of policy, it is also plausible that it derives from the very high concentration of professional families in London and their high input into their children’s education (which is not captured by a socio-economic control variable that just measures eligibility for free school meals).
Many policy makers, school leaders and commentators enthuse about the major policy of the time, London Challenge, and view it as unambiguously improving schools in London. This unanimity carries weight, and no doubt London schools were improved in a number of ways. But so far at least, catching a reflection of this improvement in the attainment data is proving to be difficult.
It sounds somehow uninspiring and disappointing that the London attainment premium is largely “accounted for by demographic composition” rather than wholly caused by some innovative policy. I disagree. It can be seen as a story of aspiration and ambition. There is nothing inherently different about the ability of pupils from different ethnic backgrounds, but the children of immigrants typically have high aspirations and ambitions, and might place greater hopes in the education system.
London has a right to be pleased with itself in terms of the excellent GCSE performance of its pupils. These results help to explain the ‘London Effect’; they do not explain it away. My argument is that the London effect is a very positive thing, and much of the praise for this should be given to the pupils and parents of London for creating a successful multi-ethnic school system.