When an A isn’t ‘just’ an A: reforming examinations
August 12th, 2010Looking back on more than a decade of New Labour educational reform from so promising a vantage point as we are presently afforded, Ed Balls remarks’ about the sterility of the standards debate are all the more galling now than they were to me at the time. That the then Education Secretary and his colleagues at the DCSF could have been so unashamedly dismissive of this most important of all debates was nothing short of outrageous. Why, I wonder, did we put up with this insolence for so long?
Did we really buy the official party line that the steady improvement in the performance of students in examinations was ‘a reflection of improving standards of teaching and learning’? There were certainly grumblings, but very few parents actually spoke up in protest. Last month The Times Dr Richard Pike, Chief Executive of the Royal Society of Chemistry (RSC), one of too few high profile critics that have consistently sought to expose this politicisation of the debate, spelled out his answer. The status quo was, he said, just too comfortable. Politicians were simply unwilling to upset middle class parents’ expectations that their children should ‘make the grade’ – the only grade, that is, that matters. When it’s your child getting the As they deserve, when it’s your school that’s doing so well in the tables, they turn don’t they? Who wants to raise the voice of dissent when all are busy unwrapping their prizes? The result is that at the end of more than a decade of New Labour educational reform the belief that all should Achieve is more entrenched than ever. A is for Achievement: nothing else will do.
Of course it did not help matters that those who did speak up were pilloried as cynics, liars, or worse. Talk of grade inflation, as Balls and Adonis would splutter indignantly, was both ‘demoralising for teachers and insulting to students’. ‘Quashing question marks against ever-increasing performance’ however ‘by equating them with question marks against whether pupils deserve their grades’, as Anastasia de Waal, of the think-tank Civitas, has argued, was, to put it mildly ‘a great disservice to us all’, and most especially to the school leavers of that period themselves. In terms of the unemployability of so many apparently qualified school leavers, in terms of the cost to universities of having to administer tests to separate the best from the rest, in terms of our declining international competitiveness, we are now paying the price.
Unfortunately, up until fairly recently, there was very little objective published research into grade inflation that could be drawn upon to move what has been for many a wearyingly circular debate forward. For too long, as Francis O’Gorman pointed out at think-tank Reform’s annual conference this July, there has been too heavy a reliance (on all sides) on anecdote and intuition, coloured inevitably by ill-informed presuppositions of one kind or another. But in 2008 Robert Coe and Peter Tymms, of the Centre for Evaluation and Monitoring (CEM), published (to not enough publicity) the results of a 20 year monitoring exercise involving comparison of A level results with YELLIS, a standardised aptitude test taken in year 11. Across most subjects tested, a student with a particular score on the aptitude test would have received a C grade in 1988; in 2008 a student with the same score would have gained an A. The discrepancy was widest in Maths, involving a leap from a U grade to a low B by 2008. CEM’s research findings have been further substantiated this year by a joint study undertaken by the Institute of Electrical and Electronics Engineers (IEEE) and the University of Coventry, and lent further weight by a useful survey undertaken last year by Civitas and A new level, from the think tank Reform.
Of a number of factors that have contributed to grade inflation, chief among these, and least in dispute, is that we now have a much more efficient and well-oiled examinations process than was the case a decade ago. Civitas’s survey of 150 sixth-form college senior teaching staff, investigating reasons for the trend found that half of all respondents felt that it was due in the main to ‘students being better informed about what will be examined’.
Behind this statement lie a number of related developments in teaching and learning – all of which have contributed to making the examination process more efficient. What were once more general ‘syllabuses’ have been replaced by much more tightly defined ‘specifications’. Both questions and marking are more directed, reducing the margin of error, or divergence, from the marking standard. Much more is required in the way of planning than was once the case. The process of developing schemes of work from outline learning plans, and then fleshing these out further into detailed individual lesson plans, has been standardised. Examinations boards now publish reams of ‘teacher support materials’ to accompany their own textbooks. Learning generally takes place within much more clearly articulate frameworks, with the end in view in perspicuous focus at all times. Students get more exams practice. Teachers are accountable for their ‘delivery’ of the specified curriculum, and expectations on all sides (not just from parents), when it comes to results, are much higher than they were thirty years ago.
While no one wants to question the effort that has gone in to securing good grades, and proper credit should be given to exam boards for their efforts to improve process, if the overall achievement of the system boils down to the fact that we have got better at exams then there is clearly something wrong. The really important question is surely whether and how well young people’s educational experience has served to prepare them for further/higher education and the workplace. In A new level, the aforementioned report published by Reform last year, academic analysis of A level exams from 1951 to 2008 found that the combined effects of these developments, depressingly, have ‘allowed candidates [increasingly] less scope for using their own mind, thinking through problems and expressing originality’. It hardly needs to be said that on any gauge of present or future workforce requirements this is the exact reverse of the profile of the average school or college leaver that we want. We have a system in which exams, and good grades for that matter, are increasingly ‘accessible’ but correspondingly less meaningful. To the degree to which the purpose of exams is to assess students’ mastery of subject content, their ability to think, and ultimately their preparedness for further/higher education and the workplace, they simply aren’t serving the purpose.
An A grade needs to mean something. In 1994, when examining boards were forced to introduce A*s at GCSE to distinguish the best from the rest, it was with the clear implication that an A was not what it once was. How much more so 16 years on? This year A levels will go the same way. When universities complain of the difficulties of distinguishing between applicants and are forced to introduce their own tests to evaluate the relative achievement of top A level students, the argument is not ‘academic’ anymore: we have a real and relevant problem. A 2009 Supporting Professionalism in Admissions (SPA) survey revealed that across Britain’s universities 84 different subject-specific tests had been introduced by admissions departments to supplement the A level, with one in five universities setting their own tests. This year Imperial College introduce their own institution-wide aptitude test as a requirement for entry. At what cost, I wonder, to a sector already in financial crisis?
Of course exams are not just about determining students’ preparedness for academic study at university; they are also about assessing the competency of those preparing for entry into the workplace. To Labour’s acute embarrassment, Sir Stuart Rose, who runs Marks & Spencer, complained last year that British education is producing people who are ‘not fit for work’ and said of some school leavers that: ‘They cannot do reading. They cannot do arithmetic. They cannot do writing.’ His comments were echoed by Tesco chief executive Terry Leahy, and they were not alone in their criticism. In spite of all the banner waving, successive Institute of Directors (IoD) surveys show that the experience of leading employers has been that young people’s proficiency in a range of skills has weakened over the last decade, highlighting the emergence of a ‘credibility gap’ between the rhetoric of examinations success and perceptions in the workplace. A recent CBI report found that a quarter of all employers recruiting school leavers had to invest in remedial literacy and numeracy training. While employers need school leavers to be able to think and reason beyond basic literacy and numeracy, in 2005 basic ‘functional skills’ requirements for English and Maths were set which in fact lowered the standard below those required of the GCSEs.
Neither have accelerated attempts to enhance employability through the addition of vocational qualifications helped matters. It is a popular fallacy that demand for vocational qualifications has come from employers. A CBI study in 2007 confirmed that businesses overwhelmingly value solid academic grounding and the ability to think over specific vocational training, which can be developed on the job. In response to a recent survey of Federation of Small Business (FSB) members, 78 per cent indicated a preference for apprenticeships over other vocational routes.
Rather, as is persuasively argued in the Reform policy paper A new level, the development of vocational qualifications has been the result of a deliberate strategy for widening participation in education that has focused on ‘accessibility’ at the expense of rigor. This began with the replacement of O levels by GCSEs and the rebranding of polytechnics as universities in the 1980s. Successive governments sought to promote new qualifications like the NVQ, the BTEC and the new Diploma, steadily blurring the boundary between academic and vocational education. The National Qualifications Framework (NQF), introduced in 1998 to formalise the comparison of different qualifications, completed the process, allowing the value of vocational qualifications to be quantified relative to ‘equivalent’ academic qualifications.
From September 2010 the NQF will be replaced by the Qualifications and Credit Framework (QCF), according to which the new ‘Higher’ Diploma will be worth the equivalent of 7 GCSEs at A*–C, and the Level 2 BTEC Diploma will be worth 4. Given that these vocational qualifications are assessed entirely through coursework and are expressly designed to make it nigh on impossible to fail, there is a strong incentive for schools to prefer these over GCSEs – not least because they ‘count disproportionately towards league table success’. A full 70 per cent of Teach First graduates believe that such considerations are influencing teachers’ course recommendations to their students. The result is that take-up of non-academic courses is increasing, while the percentage of students taking traditional academic courses is in decline. This serves no one: not teaching professionals, certainly not their would-be employers; above all else, not young people themselves.
How then is Britain measuring up internationally? Not well, in short. Both in terms of the narrowness of the required curriculum and the expected level of attainment at 16 England ranks among the lowest of OECD countries, and compares equally poorly with Japan, the US and Canada too. Reform’s study Core business found that our Science and Maths exams in particular are markedly less rigorous than their international counterparts.
On all the indicators that really matter then, our current examinations system is not fit for purpose. What can be done? This year, at a conference organised by Cambridge Assessment, leading examiners debated the issue of declining standards in public for the first time. The only way to maintain standards, Chief Executive Simon Lebus offered, is ‘to let higher education, employers and subject specialists talk directly to exam boards once again’. Acting on this advice, and on policy recommendations from Reform, the re-focused Department for Education (DfE), taking over from the abolished QCDA, has begun consultations with universities and professional bodies like the Advisory Committee on Mathematics Education (ACME), whom ministers intend to charge with restoring and maintaining the academic rigor of the A level qualification. In a promising early indicator of the direction of things to come, Michael Gove announced in early July that consensus has been reached that an alternative A level to be released into the market next year would do without the interim step of the AS level, reversing a decade-long pursuit of modularity. Under plans for the further reform of the A level qualification, exam boards will be expected to work much more closely with university academics too, competing with other boards to raise academic attainment. AQA’s admission in 2008 that it had ‘under pressure’ from Ofqual been forced to lower grade boundaries across its GCSE science examinations to bring them into line with competing boards, should be the regulator’s last such embarrassment.
Correspondingly, in a move which led to the resignation of its first chairman, Kathleen Tattersall, the exam regulator Ofqual’s brief has been redefined to give bite to what Chris Woodhead always said under the last government could not be anything other than ‘yet another toothless quango’. Going forward Ofqual will be making past papers from examinations boards, including those now defunct, available to the public for online comparison. The regulator will also report on how British exams compare with those in other countries, so we will be able to measure the questions young people are expected to answer against those set by contemporaries in India, China, Singapore, South Korea, Australia, New Zealand and Canada. The market will be opened up to alternative, reputedly more rigorous, academic examinations, including the pre-U and International Baccalaureate, interest in which has surged in recent years. State schools will be allowed to offer the IGCSE as a more academically challenging alternative to able key stage 4 students. The government is reportedly looking at how take-up of academic courses may be incentivised and the QCF adjusted to redress the imbalance in favour of these qualifications. The practice of inflating league table results by precipitously encouraging students along vocational pathways looks set to end with the recent announcement that academies (apparently the worst offenders) will be required to make details of which examinations students were entered for available to the public under freedom of information laws.
Welcome developments all, but for all the new government’s laudable plain-speaking commitment to tackling the problem, will these reforms be enough to restore rigor to our examinations system? I don’t think they will and certainly not overnight. Why? Because after 12 years of ‘things can only get better’, most recent opinion polls, like that undertaken by Ipsos MORI to investigate perceptions of A levels and GCSEs, indicate that the majority continue to have ‘a high level of confidence’ in the qualifications. Teachers, parents and students remain largely confident in the quality and accuracy of marking, believing that students get the grades their performance deserves. We simply don’t recognise, or aren’t prepared to accept, the reality of the problems I have been describing. The thinking and the pattern is so well established that a dip in the pass rate – surely the logical outcome of the reforms proposed, and the true test of their bite – let alone a reversal of the trend, is barely conceivable. The electorate’s unwillingness to face these issues will present a powerful driver towards compromise: for all the reforms planned, the system may yet bottle out of the necessary strictures of grade deflation.
It is not reform that is required of our A level and GCSE examinations, but ground up reconstruction, and if the rebuilding is to be successful then a complete change of mindset will be required. This is because the roots of the problem in exams and assessment theory lie in what has been a steady undervaluing of the importance of ‘competency’ relative to the priority of building young people’s confidence. So long as we all remain committed to building confidence before all else, and insist that maximising A*–C grade outcomes is necessary to achieving that end, we will not see genuine rigor return to the system. There is a balance to be struck, of course but confidence that is built on imagined, rather than real, achievement is of no help to anyone.
At present, in delicate service to young people’s ‘fragile’ sense of self-esteem, the approach taken to the marking of GCSE exams, ‘positive awarding’, overlooks all manner of confusion and factual error, believing structure, spelling, punctuation and grammar (coherence, that is) to be an irrelevance in its enthusiasm to credit achievement. If candidates let slip enough facts in what is otherwise an incoherent stream of consciousness, they nevertheless deserve to succeed. Candidates may answer question ‘a’ in their answer to ‘b’, question ‘b’ in their answer to ‘c’, and question ‘c’ in their answer to ‘a’ and still receive full marks. At A level, ‘responses’ are assessed using inclusive ‘level descriptors’ detailing the characteristics of an A, B, C (etc.) grade candidate, a fuzzy approach that consistently gives the benefit of the doubt to answers that straddle the criteria of more than one band. If university academics at the DfE do not penetrate to address the issues at this level then they will have failed. No degree of help from those inputting later in the process, consulting for the examinations boards, or even from Ofqual, will be able to change that.
We may get there in the end, but not I don’t believe without sustained and unrelenting independent pressure from universities and employers. What is needed to drive both standards and expectations up above all is for a third party organisation, entirely independent of the process, to subject the combined efforts of the DfE, exam boards, Ofqual and their academic advisors to annual scrutiny and to present their findings in a way that is accessible to all, but most especially to parents. Real change comes from the ground up; only demand from parents will ultimately provide the drive that the examinations market needs to justify (in business terms, and to their own occupational interests) raising the bar. The brief for such a review, sponsored and undertaken by universities, professional bodies and leading industry practitioners and employers, would be shaped by the same real-world requirements as those informing the work of academics working with the DfE and the exam boards, to be sure. But reviewers would be in a better position to judge, fairly and even-handedly, the relative merits and demerits of different exam offerings. Such judgments would be made according to explicit and clear criteria, taking into account the content of different syllabuses, grading methodologies and criteria, as well as the requirements of the test questions themselves. Summary reports, written to Plain English standards, covering all available offerings in a range of subjects could be made widely available online. This would stimulate demand from parents, and teachers, universities and employers for the most rigorous qualifications on offer and, in the case of the latter stakeholders, provide them with the information they need to interpret the grades listed on the CVs of course and job applicants.
There’s certainly an opening in the market; it may even be that without such a review, progress towards higher standards will falter as the political fallout becomes clear. As with schools, so with examinations, it is parent power that provides the real keys to lasting reform.
