The next few weeks might be a horrible time of year if you are 15 or 16. There are some big decisions coming up. One the one hand: the final exams for the GCSE courses, completing two years of work leading up to this moment. There is a lot of studying still to do, notes to be read, exercises to be worked through, understanding to be really nailed down. Final revision.
But on the other hand: the World Cup. In Brazil. England qualified, and while no-one thinks of England as favourites … who knows? Who would want to miss watching Gerrard and the team confounding the pundits and cruising into the semis?
What to do? This is a classic question of time preference: jam today (watching the game) versus jam tomorrow (getting the grades and higher lifetime income). What is the trade-off between grades and goals?
Our research < https://cmpo.wordpress.com/2011/12/06/a-report-of-two-halves/ > can help. We have studied < http://www.bristol.ac.uk/cmpo/publications/papers/2011/wp276.pdf > the decisions of about 3.5m students facing this dilemma in previous summers. We compared the GCSE performance of as-good-as-identical students in years with World Cups (or the European Championship) and years with no exam-time distractions.
On average, grades were slightly lower in World Cup years. We interpreted this as some students taking some time out from studying to keep an eye on the tournament. While there are other possible explanations, our statistical techniques rule out more or less everything else.
That’s on average. Some groups of students saw sizeable declines in their grades. Again, the interpretation of this has to be that they prioritised the tournament and seriously cut down on study time.
How much does this matter? It depends on how close to the key borderline the student’s performance is likely to be. Achieving at least 5 good passes (C grade and above) including English and maths is widely regarded as a necessary minimum for further education or getting a good job. For students who are near this borderline, a grade or so either way matters a lot.
Missing out on 5 good GCSE grades can be very costly. Estimates suggest an average total lifetime cost of around £30,000. This seems a very hefty price to pay for watching some football.
The moral of all this research and numbers: if you are likely to be close to the 5 Cs borderline, stick with the studies, let others suffer the pain of watching England, and get the grades. In the future, you will have earned the money – and the right – to sit back and fully enjoy World Cups.
Should the government ban junk food near schools?
Last Friday, Jamie Oliver called for a crackdown on the selling of junk food near schools, arguing that it is completely at odds with the government’s investments to tackle childhood obesity. But is there really a causal link between fast food outlets near schools and childhood obesity?
Currie et al. (2010) directly investigate this. As they argue, the fact that fast food restaurants and obesity have both increased over time does not proof such a link. To explore this relationship, they use the exact geographic location of fast food restaurants in California, linked to data on 3 million school children, to explore whether schools’ proximity to a fast food restaurant affects pupils’ obesity rates. They show that fast food restaurants near schools significantly increase childhood obesity. More specifically, having a fast food restaurant within 0.1 mile of a school increases the probability of obesity by 1.7 percentage points (or 5.2%).
To explore the sensitivity of these analyses, they also study the effect of other (non-fast food) restaurants, but find no effects of these outlets on obesity. Furthermore, they investigate whether future fast-food restaurants are associated with today’s obesity. If it is, it would suggest that fast food restaurants may simply locate in areas where obesity is increasing independent of the restaurants. Their results show that only current locations matter.
These findings therefore support Jamie Oliver’s concern, suggesting the government should consider introducing policies to restrict fast food restaurants from opening near schools. In fact, some local authorities have already taken this approach. For example, the London borough of Waltham Forest will not give planning permission to new hot food takeaways if they are within 400 metres from a school, youth facility or park.
Jamie Oliver has had much success in convincing the government to improve the nutritional contents of school lunches. Recent research has shown that the new nutritional guidelines have not only improved the quality of school food (School Food Review; SFR), it also improved children’s exam results and reduced their absences (Belot and James, 2010). In fact, due to the benefits of school meals, Professor Terence Stephenson, chair of the Academy of Medical Royal Colleges, argues in Sunday’s Observer that academies and free schools may actually be damaging children’s health, as they are allowed to opt out of these nutritional guidelines.
Despite the benefits of school meals, their take-up is currently very low at around 43%, reduced from about 70% in the 1970s. In a recent study, I show that the main reason for this rapid decline was the introduction of two Acts of Parliament in 1980 and 1988, which increased the price of school meals, leading to a large proportion of pupils shifting to packed lunches. With school meals currently substantially healthier than the average packed lunch (SFR), the government should consider ways of increasing its take-up. One approach they have taken is to introduce free meals for all primary school pupils in England in their first three years. As the increased price of meals in the 1980s was the main driver for the drop in take-up, offering free meals to all children may again lead to an increase in their consumption. In addition, if alternative outlets, such as fast food restaurants, are simply not available near schools, pupils will not get tempted to swop their healthy school meal for some unhealthy fish and chips.
Author: Simon Burgess
Education spending, pupil attainment and causality
In these hard times, spending government money effectively is more important than ever. Last week Fraser Nelson challenged the effectiveness of spending in schools, one of the areas relatively protected from Coalition cuts. He said: “The biggest surprise, though, was the money: no matter how you split the figures, the amount spent didn’t seem to make the blindest bit of difference”, his reading of a report by Deloitte commissioned by the Department for Education.
What is the evidence? In fact, it is surprisingly difficult to establish the impact of spending more money on student achievement. This is partly shortage of data (researchers always want more data), but there is a more fundamental reason too.
Perhaps inadvertently, Fraser Nelson illustrated the difficulty in his first paragraph. He noted the variation in per-pupil expenditure “ranging from £4,500 in Lyme Regis to £10,000 in Salford.” This is absolutely right – there are very significant variations in revenue per pupil. But the key point is that these are not random: extra resources are explicitly and systematically directed towards schools in poorer neighbourhoods. The mechanism, accreting the new schemes of each successive government, may be incomprehensibly complex, but the intent is surely right.
Getting back to our question, on the one hand we have this systematic distribution of resources towards poorer neighbourhoods. On the other hand we know that pupil attainment is typically lower in schools in such neighbourhoods; not for every pupil, not in every school, but on average. So if money has no impact on attainment, and we line up pupil attainment and school expenditure, we will tend to see a negative relationship. This derives solely from the way that money is distributed to schools. The fundamental problem is that there are two things going on with opposite effects: low attainment is associated with more money (via the schools funding system) and more money may be associated with high attainment (via the education process). With no other information, there is simply no way of disentangling these two opposing effects, and by itself these numbers can tell us nothing about the causal impact of school expenditure on pupil attainment.
So the view that “the amount spent didn’t seem to make the blindest bit of difference” cannot be supported by this evidence.
What of the wider research evidence, based on studies with a plausibly causal research design? One of the most prominent economists in the field of education, Rick Hanushek from Stanford, is famously sceptical of the value of greater resources for schools. There certainly are studies that show money can matter, but it is probably fair to say that the majority view among economists is that simply providing more resources for schools is not the best option.
The really interesting question is this: why doesn’t more money raise attainment? More money usually helps most things. Either there simply is nothing that schools can buy that raises attainment. This seems unlikely, and would certainly be a surprise to parents paying many thousands of pounds to send their children to private schools. Or there are features of the system which lead schools to spending extra resources on the ‘wrong’ things – things that have little impact on attainment. This might be the manner in which the money is distributed by government (typically short-term, making long-term expenditure decisions risky); or the regulations and agreements governing its spending by schools; or other factors. We have speculated a little about this here.
Coincidentally, the Department for Education has just opened a consultation on school efficiency – they await your views.
Author: Simon Burgess
Should we have profit-making schools?
Profit-making schools have returned to the education debate in England. This is an emotive issue for many, but an economic analysis is useful in defining the real issues.
There are some simple claims that can be quickly dealt with.
- “Education is far too important to be left to the mercy of profit-making companies.” Education is undoubtedly very important, for long-run growth, for social mobility, and for personal well-being. But think about possibly the most elemental of human needs, the production and distribution of food. While this is regulated by government, we are happy to leave all the decisions to profit-making companies. No-one seriously advocates the nationalisation of food.
- “It just won’t work.” It clearly does at a general level. Countries around the world, including those with well-regarded education systems such as Sweden, allow profit-making schools.
- “No-one should make money out of education.” Obviously they do at the moment: schools buy things from profit-making companies. This obviously has to be the case unless schools are going to start making their own books, desks and computers. So the real issues are (1) what kind of deal can schools get to minimise profiteering, and (2) what services are best bought in from outside as opposed to provided by the school itself
The appeal of allowing profit is the view that it makes decisions matter more. It provides strong rewards to organisations to innovate, to raise quality, and to do things more efficiently. Crudely, on a per-unit basis, organisations are pushed to improve quality and therefore revenue, or to reduce cost.
What would be the effects of this in the current education system in England? To answer this, we need to think about the parameters of the market.
Start with revenue. Schools get revenue for having students on the books. It is more or less a per-capita fee, albeit with some extras and some adjustment by the LA (for community schools). But to adopt the language of business, this money is for processing the students. The revenue that the school receives for each student depends not at all on the progress that the student makes.
This is central to the issue. Given the current system, there is nothing that profit maximising schools could do to raise their revenue per student by raising quality. Immediately, a great deal of the appeal of profit-making is removed.
The only way that schools could make profits is by driving down costs. This may be fine; it may be that this doesn’t really affect the quality of education if done in a smart way. If not done in a smart way, the quality of education would suffer and attainment would fall. It is clear that even the optimistic scenario does not improve education systemically in any way, either statically or dynamically through encouraging entry. The quality of education is the same, and the overall cost to the taxpayer is necessarily the same.
The counter-argument is that the pressure for profit might reduce slack enough so that the fall in costs allowed for profits and an increase in money spent wisely so that attainment increased. For this to work, it has to be that school budgets are spent very unwisely, and that an outside organisation could identify and cut ‘bad’ spending, take some profit and raise ‘good’ spending. It is certainly true that there is a huge amount of idiosyncratic variation in school financial decisions, variation that is unlikely to all be the result of optimal decision-making. Schools either know how to better deploy their budgets but are not sufficiently incentivised to do so, or they do not know. If they do not know, it is unlikely that outsiders will do (other schools may know; but that is another issue, only very clumsily mimicked by profit-making). Profit-making may answer the first point, but so do two other approaches, discussed below.
So profit-making is pointless at best: under the current market set-up, improvements in attainment would not make money (so would not happen) with profit-making schools, and cutting costs would make money but would either reduce attainment or leave it unchanged.
There are alternative strategies that might get some of the benefits of the innovative drive that profits might unleash, but in a more productive way: paying for attainment and incentivising cost reductions through resources for the school.
Paying for attainment. A positive step that keeps the current non-profit system intact but provides some of the same incentive is tying schools’ revenue to their pupils’ attainment. This would be straightforward to administer in principle, but there are some critical issues to resolve before it could be implemented. Chief among these is: should we pay for the simple ‘output’ of the school (GCSE points) or for pupil progress? There are good arguments both ways, to be visited in another post. Of course, schools do much more than produce attainment, but this is the focus of policy.
Incentivising greater efficiency in other ways. What if any surplus generated by this process had to be re-invested in the schools? Perhaps schools need some strong incentive to reduce costs. This might well be true, but this is not profit-making: profit-making by definition means the taking of monetary reward out of the school. An alternative scheme would be essentially equivalent to a team (school)-based incentive scheme in which the incentive is not money for the teachers, but resources for the school – resources saved are kept in the school. This is again potentially a good idea, worth looking at and some way short of profit-making.
Profit making in schools would either solve all schools’ problems nor signal the end of civilisation; the issue provokes strong feelings, but largely misses what should be the central policy concerns. Big gains in levels of attainment depend on raising average teacher effectiveness and big gains in equity depend on weakening the importance of proximity as an admissions rule and on changing the allocation of effective teachers across schools. None of these would be strongly or directly affected by for-profit schools. However, there are certainly merits in piloting policies that link school’s revenue per student to the progress of that student, and incentivising cost reductions through keeping the surplus in the school.
This week the House of Commons Education Select Committee published its report on the teaching profession. This post gives the main points of our evidence to the Committee.
We think of Initial Teaching Training (ITT) as encompassing both the initial training and the probationary year. How should this be set up to produce the most effective teachers who will have the greatest impact on pupil progress? ITT plays two roles for the profession – training and selection with the emphasis typically placed on the former. Both are important and neither should be neglected, but we argue that the evidence suggests that if anything, selection is the more important, and this is our focus here. An important role for selection is completely standard for any professional accreditation system in either public or private sectors.
The key argument is this: the sharpest selection should be made at the point when the evidence on ability is strongest. The final decision on who can become a teacher should be made when we have accumulated enough evidence on the candidate’s teaching effectiveness. Where is this point in teaching? The two central relevant facts are that variations in teacher effects on pupil progress are very substantial, and that the future effectiveness of a potential teacher is hard to judge from their own academic record.
We believe that the current operation of selection in ITT (tight at the beginning, negligible thereafter) is the wrong way round. Instead, we should let a broader group try out to be teachers, but enforce a much stricter probation policy based around measures of teacher effectiveness in facilitating pupil progress. Full certification and an open-ended first job would only be granted once performance data showed a teacher to be effective. The expectation would be that only the most effective teachers would make it through to full certification.
Selection into ITT is about gaining a place on a course. The difficulty faced in identifying people likely to be good teachers is very relevant here. It is very hard to tell who will be a good teacher and therefore a high degree of agnosticism would be appropriate when faced with applicants. This is certainly true for selection based on objective criteria from the applicants’ own academic records. We know that these are unrelated to teaching ability, and so should be irrelevant in selection into ITT. Beyond that, even if selectors are highly skilled at spotting potential, and it is not clear that they are, it is impractical to ask each applicant to teach a practice lesson. Therefore, selection into ITT should be very broad, with a relatively low academic entry requirement. This of course is not the situation now, nor the direction of travel of current policy. The tightening of academic entry requirements into teaching is not helpful: it will restrict the quantity of recruits and have no impact at all on average teaching effectiveness.
Graduation from ITT should also be tough. Given that much of an ITT course is now school-based, time spent in the classroom will form an important part of the assessment. Arguably the classroom experience is the key part of the course. However, in such a short space of time it will not generate sufficient data for a robust and objective view of the trainee’s effectiveness. It will nevertheless allow the trainee to discover whether teaching is for them.
Once in a job in a school, the progression to being a qualified teacher should be very different to the typical experience now. The key decision on final certification should be made after a probation period of say three years and ideally, the probation should involve classes of varying ability and year group. The period probably cannot be less, though the appropriate length of the probation would need to be analysed properly, depending on the statistical reliability of any pre-hire indicators, school-based performance data, and the cost of being wrong. This is the point when enough data is available to make a reliable judgement on the effectiveness of the teacher. There should be an expectation that not all will make it through to final certification, and indeed only the most effective should be retained. The key judgement should be a minimum threshold of progress that the probationer’s pupils make. Obviously, the measurement of that progress and the parameters of the threshold require a great deal of careful work. Like any statistical data, estimates of teacher effectiveness will never be perfect, and a good deal of evidence over a number of years will be necessary to reach a decision, but this is clearly necessary to raise the average effectiveness of the teaching profession in England.
Another innovative route into teaching is through Teach First. In some ways this is a positive development, as it allows a lot of people to try out teaching and also gives the schools which employ them an ‘out’ at the end of the two years. On the other hand, it restricts entrants based on their academic background.
It is important to see the teacher labour market as a whole, and to see how the different stages of a teacher career fit together. It seems to be very hard to fire ineffective teachers. While the regulations on this have recently changed, generating a culture that encourages headteachers to take a more proactive stance seems harder. While this may change, it may be that the best way to reduce the problem of low-performing teachers is to make it very difficult for ineffective teachers to get into the profession in the first place.
These changes would make starting out on a teaching career much more risky financially. In order to maintain the same average lifetime expected income from the profession, the pay rate of those making it through to final full certification will need to be higher. And the lower is the chance of making it through, the higher is the full professional pay.
In summary, we think that the evidence shows that the selection aspect of ITT is completely the wrong way round. Selection is tight to get into ITT in the first place, but once in, progression to full certification is normal and expected. The process needs to be more appropriately agnostic about likely teaching ability in the first place. It should also allow a broader group of people to try out teaching, but have a much tougher probation regime before trainees be given final certification. It makes much more sense to make final decisions later once more evidence on effectiveness has accrued.
What is the best way to deal with under-performing schools? This is a key policy concern for an education system. There clearly has to be a mechanism for identifying such schools. But what should then be done with schools which are highlighted as failing their pupils? There are important trade-offs to be considered: rapid intervention may be an over-reaction to a freak year of poor performance, but a more measured approach may condemn many cohorts of students to under-achieve.
This is the issue that Ofsted tackles. Its inspection system identifies failing schools and supervises their recovery. How effective is this? Is it even positive, or does labelling a school as failing push it to ever lower outcomes for its students?
It’s not clear what to expect. Ofsted inspections are often dreaded, and a fail judgement seen as being disastrous. It has been argued it triggers a ‘spiral of decline’, with teachers and pupils deserting the school, leading to further falls in performance. But it might also be a fresh start, with renewed focus on teaching and learning, leading to an improvement in exam scores. Equally, we might expect nothing much to happen: after all, the policy ‘treatment’ for those schools given a Notice to Improve is very light touch. It is neither strongly supportive (typically no or few extra resources) nor strongly punitive or directive (schools face no sanctions nor restrictions on their actions). Schools are instructed to focus intensively on pupil performance, and are told to expect a further inspection within a year. In addition – and possibly the most important factor – the judgement that the school is failing is public one, usually widely reported in the local press.
Our research shows that the Ofsted inspection system works. Schools that just failed their Ofsted significantly improved their performance over the next few years, relative to schools that just passed. The impact is statistically significant and sizeable. In terms of the internationally comparable metric of effect sizes, our main results suggest an improvement of around 10% of a standard deviation of pupil scores. This is a big effect, with a magnitude similar to a number of large-scale education interventions. Translated into an individual pupil’s GCSE grades, this amounts to a one grade improvement (for example, B to A) in one or two GCSEs. From the school’s perspective, the gain is an extra five percentage points in the proportion of pupils gaining five or more GCSEs at grades A*-C.
Our findings suggest that the turn-around arises from proper improvements in teaching and learning, not gaming to boost exam performance through switching to easier courses. First, the impact is significantly higher in the second year post visit than the first, and remains level into the third and fourth year after the inspection. So it is not simply a quick fix to satisfy the inspectors when they return twelve months later. Second, we find a stronger effect on the school’s average GCSE score than on the headline measure of the percentage of students gaining at least 5 good passes; if the schools’ responses were aimed at cosmetic improvement, we would expect the reverse. We also find similar positive effects on maths results and on English results.
It could be argued that these results are implausibly large given that the ‘treatment’ is so light touch and schools are given no new resources to improve their performance. The instruction to the school to improve its performance may empower headteachers and governors to take a tougher and more proactive line about school and teacher performance. This may not be a minor channel for improvement. Behavioural economics has provided a good deal of evidence on the importance of norms: the school management learning that what they might have considered satisfactory performance is unacceptable may have a major effect. The second part of the treatment derives from the fact that the judgement is a public statement and so provides a degree of public shame for the school leadership. Ofsted fail judgements are widely reported in local press and this is usually not treated as a trivial or ignorable announcement about the school. It seems plausible that this too will be a major spur to action for the school.
Where do we go from here? Our results suggest Ofsted’s identification of just-failing schools and the use of Notice to Improve measures is an effective policy, triggering the turn-around of these schools. We need to be clear that our research does not address the question of what to do about schools that comprehensively fail their Ofsted inspection. Possibly this light-touch approach can be extended. Since leaving the Headship of Mossbourne school to become the new Director of Ofsted, Sir Michael Wilshaw has argued that schools just above the fail grade should also be tackled: that ‘satisfactory’ performance is in fact unsatisfactory. Such interventions in ‘coasting’ or ‘just-ok’ schools are very likely to be of the same form as Notice to Improve. Our results suggest that this is potentially a fruitful development with some hope of significant returns.
The high level of residential segregation for people from different backgrounds was highlighted as a factor contributing to the tension between people from different ethnic groups in the Cantle Report (2001) in response to the race riots in the same city in 2001. The Cantle Report identified that many people living in the same city led “parallel lives”; with little interaction with, or understanding of, people from different backgrounds.
Last night a Channel 4 programme explored the relationships between 8 residents of Bradford, brought together to interact with people from different backgrounds for the first time. The programme addressed the difficulties and benefits of over-coming entrenched prejudices, underlying (perhaps unacknowledged) racism in the city, and the similarities and differences between people from different cultures. The majority of participants in “Make Bradford British” found it a positive experience. Future improvements in relationships between different communities are likely to come through increased daily interaction, however, as well as the moving personal experiences witnessed on screen, and integration in schools can be a starting point.
Segregation in schools commonly results from residential segregation (as admission to schools is usually determined by proximity), but parents’ preferences can also play a role. Research by CMPO shows that pupils from most ethnic groups are more segregated in school than where they live, suggesting that there are factors, either parents’ preferences or other constraints, that restrict integration in England’s schools.
Pupils’ current level of integration in each area in England can be viewed on a website created by CMPO, funded by the ESRC and using national administrative data provided by the Department for Education. The figures for Bradford are startling: in 2009 around 80% of Pakistani pupils of primary school age attended a primary school that was primarily non-white, while just under 70% of white pupils attended a school that was primarily white. This suggests that these two groups of pupils are highly segregated in their primary schools, and the figures suggest that they remain so in secondary schools. Bradford is not alone in this high level of segregation (Oldham has similarly high levels), but the figures are higher than in other areas of the country with similar demographics; in Manchester under 60% of Pakistani pupils attended a school that was primarily non-white in 2009 and around only around 30% of white pupils attended a school that was primarily white.
Other indicators of segregation can also be compared: the dissimilarity index is a summary measure designed to capture the degree of “unevenness” between two groups of pupils (that is, the extent to which the school population reflects the wider population in the local area). Its value (between 0 and 1) represents the proportion of the group population that would have to change schools to achieve the same distribution of that of the local area, where, in general, a dissimilarity index of less than 0.3 is considered low, between 0.3 and 0.6 as moderate and above 0.6 as high (Massey and Denton, 1988).
Bradford has a decisively high level of segregation according to the dissimilarity index; in 2009 the figures show that around 70% of white and Pakistani primary pupils would have had to change schools in order for the distribution to reflect the wider population of the area, and the figure is similarly high for secondary schools.
There are encouraging signs, however; segregation according to the dissimilarity index has decreased marginally in primary schools since 2002, suggesting that the younger generation at least is becoming slightly more integrated, although levels remain higher than almost anywhere else in England.
The high level of segregation in Bradford’s schools suggest that integration and understanding between people from different backgrounds are unlikely to be fostered from direct experience in the education system, although there are some signs that the situation is marginally improving. What can make “Bradford British” and united remains unclear, but further integration of the city’s young people is not likely to harm the city’s prospects.
Notes on the author:
Ellen Greaves is a Research Economist in the education, employment and evaluation research sector at the IFS.