Education, Children and Young People Committee

Meeting date: Wednesday, September 29, 2021

Official Report 608KB pdf

---

Alternative Certification Model

Good morning, and welcome to the fourth meeting in 2021 of the Education, Children and Young People Committee.

We are meeting in hybrid format for our final evidence session on the alternative certification model. Joining us on behalf of the Scottish Qualifications Authority are Fiona Robertson, the chief executive; Dr Gill Stewart, the director of qualifications development; and Beth Black, director of policy, analysis and standards. Welcome to our committee.

Fiona Robertson will make a brief opening statement on behalf of the SQA, and then we will get into the questions.

Good morning. Thank you for the opportunity to appear before the committee to reflect on the experience of delivering the alternative certification model—ACM—in 2021 and to look ahead.

It goes without saying, but it bears repeating, that the ACM was developed and delivered in the context of a global pandemic for which there was no script. The whole education system came together through the national qualifications 2021—NQ21—group to design and implement the ACM. Members of the group had clear roles and responsibilities, working incredibly hard to deliver for Scotland’s learners and to ensure that credibility and fairness remained at the heart of our qualifications system.

That was not straightforward—and nor should it have been, given what was at stake. I am proud of the way that learners, parents, schools, colleges, professional associations and local authorities collaborated in the most exceptional of circumstances. That pride extends to all my colleagues, who are passionate about their work and are doing their very best for learners. They, too, worked hard to deliver in the past year. At times, tough choices had to be made, but the NQ group was clear that the ACM was the best possible approach when we were faced with a situation that was rapidly evolving in real time.

On 9 August, almost 137,000 learners received SQA qualifications. They can be proud of their achievements and can have full confidence in the results.

The experience of the ACM in 2021 has helped us to develop our approach to the coming year. Following the ministerial announcement in August of a return to exams in 2022—should it be safe to do so—we have announced contingency plans, which have been developed in collaboration with partners from across the education system. The plans have been clearly communicated, with messages and materials shaped by advice from those partners.

An NQ22 group will continue to be a feature this year. We will continue to keep learners, parents, carers, schools and colleges updated as further information becomes available. We have made it clear that learners will not be expected to undertake dual assessment, thus avoiding any increase in workload and stress.

Ministers remain very supportive of the work that we do, but they have considered that the time is right to look at the national organisations that support and serve the education system, in line with the Organisation for Economic Co-operation and Development’s findings on the alignment between curriculum and assessment. We will play a full and positive role in the review process that is under way, reflecting the skills, knowledge and experience of colleagues across the SQA, while continuing to deliver for thousands of learners during the transition period.

With regard to the future, I welcome the most recent OECD report, by Gordon Stobart, on assessment and qualifications. The pandemic and the alternative assessment models that have been used in the absence of exams have shone a light on the issues, and it is imperative that we learn from that experience and from the lessons of recent reforms. I sincerely hope that everyone, including members of the Parliament and this committee, will engage positively in the debate and that they will get behind Scotland’s future approach to assessment and qualifications and the organisation that will be tasked with delivery.

My colleagues and I are happy to answer questions this morning.

Thank you. My first question relates to something to which you have already alluded. In June, the Cabinet Secretary for Education and Skills effectively announced the abolition of the SQA. What is your understanding of why the SQA is to be abolished?

On 21 June, the OECD report “Scotland’s Curriculum for Excellence: Into the Future” was published. At the same time, the cabinet secretary announced the intention to accept the report’s recommendations, including the recommendation that a curriculum and assessment agency be established. It was on that basis that the cabinet secretary intimated that the SQA should be replaced. Of course, there is a review process under way to consider those issues in more detail. Ken Muir, the former chief executive of the General Teaching Council for Scotland, has been appointed as professional advisor to the review, which is being undertaken by the Government. We are discussing those issues with Ken and Government colleagues.

On what did the SQA fail to live up to its purpose to the extent that the Government’s decision was to abolish it?

I do not think that the cabinet secretary said that the SQA failed to live up to its purpose; I think that she said that she wishes to consider a new organisation with a new purpose. As part of that, I sincerely hope that there is some continuity of function around assessment and qualifications, and around the role that the SQA plays, particularly the role that colleagues with skills, expertise and experience play. The cabinet secretary has intimated the potential establishment of a new organisation with a new function and a new purpose.

The cabinet secretary did announce that the SQA would be replaced.

She did.

What are the reasons for that decision? Has there been a failure of confidence in the SQA on the part of ministers? Have they expressed a lack of confidence in the SQA?

Ministers have not expressed a lack of confidence in the SQA. On 21 June, the cabinet secretary set out her reasoning for the announcement, which was aligned to the publication of the OECD review. Obviously, you might wish to ask the cabinet secretary further questions on the matter when she appears before the committee next week, but the conversations that I have had with the cabinet secretary and the announcements that she has made account for the reasons that I have outlined to you in my answer.

I will shortly move on to Kaukab Stewart, the deputy convener. First, how would you describe the role of the SQA in 2021? In effect, teachers’ assessments were taken as read. Some quality assurance work was done, which we will come back to, but what, in functional terms, did the SQA do after the announcements were made last October and December?

Our responsibilities as an awarding organisation, as set out in the Education (Scotland) Act 1996, were, in principle, unchanged in that we were responsible for devising and awarding qualifications. Obviously, we were in a set of extraordinary circumstances for a second year. We made a commitment—I spoke to this committee’s predecessor committee in March about this—to work with the whole system this year to develop an alternative certification model that would fulfil our statutory functions but, more important, deliver for learners to ensure that they could be certificated and continue with their learning or progress to the next steps in learning—or, indeed, their careers. Our focus was on ensuring that we did that and worked with the system to do so.

The committee is, understandably, interested in national qualifications, which is an important part of what we do, but we are also responsible for delivering many thousands of other qualifications in relation to the college sector and training providers for foundation apprenticeships. Because of the reach of our work in delivering qualifications to learners across Scotland, including to young people at school but not exclusively to them, it was important that we progressed that work. We are also responsible for regulating qualifications that are provided by other awarding bodies.

I hope that the committee does not mind my spending a moment just highlighting the breadth of our responsibilities—I know that you are keen to establish how we focused on the alternative certification model. At the start of the year, we consulted on modifications to assessment because we knew that there was likely to be some disruption to learning in the coming year. We undertook a public consultation on the modifications across the 150 courses at national 5, higher and advanced higher levels. My colleague Gill Stewart led that work and would be happy to talk about those modifications.

The modifications were announced early in the academic year and we then formed the national qualifications group to help advise us and develop the approach to our work. We must bear in mind that, this time last year, plan A was exams, so we were looking at a set of contingencies—plan A was not an alternative certification model.

The Deputy First Minister cancelled national 5 on 8 October, and he cancelled higher and advanced higher in December. During that time, there was a lot of work to develop an alternative certification model for national 5 and, subsequently, for higher and advanced higher. In tandem with that, we provided advice on the assessment approach for each of the courses that were available to provide understanding of standards and, as the convener highlighted, undertake quality assurance. We were setting the framework for assessment for the year ahead.

I know that colleagues will come in on this issue, so I will move on now. Thank you for your answers so far.

I welcome the witnesses to the meeting. As an educator previously, I know that those were extremely challenging times for all in education, who had to turn on a sixpence—I put on record that I do get that.

I am interested in hearing all the witnesses’ reflections on the past couple of years, particularly on what lessons we could learn from them or what lessons they have learned in their professional capacities.

That is an important question. We have all been in the middle of a global pandemic, which has affected every part of our lives. That has been felt acutely in education, particularly by young people in education. All public organisations have faced a challenging set of circumstances. The tradition of a spring diet of exams, which was in place uninterrupted for, I think, 130 years, was interrupted in 2020 with the cancellation of exams a matter of weeks before they were due to take place, as schools closed at short notice.

09:45  

As I highlighted in my opening statement, there was no script. There was no model sitting on the shelf waiting to be activated, if you like. The SQA has been very conscious of the impact of the past 18 months on young people, particularly in relation to their learning and teaching and how that has impacted on their assessment. From our perspective, that is about what the impact has been on their ability to undertake assessments and achieve qualifications. Working with partners, we have been focused on ensuring that young people can continue to progress in their learning and achieve their qualifications.

On lessons—particularly in 2021, which is the focus of this meeting—the move to remote learning after Christmas and the fact that that became an extended period of remote learning put a lot of pressure on young people, schools and colleges. As the national qualifications group and I absolutely acknowledge, the assessment window was, therefore, quite constrained. That was a challenging and busy period that perhaps put more pressure on young people and the system than we would have liked. There is a lot of learning to take on board this year, particularly in relation to the ACM in that compressed period, post Easter.

With regard to this year’s arrangements, we have sought to learn from that period by putting in place very clear contingency measures for disruption to learning and communicating those to the system as quickly as possible.

Would Dr Gill Stewart or Beth Black like to add anything?

Yes, please. Some key communication issues emerged from the 2021 experience, particularly from speaking to young people. They wanted to know up front what plan A and the contingency arrangements were, so, as a system, we have done that, and we will continue to do that. It was very important for them to know what was ahead of them.

There are other, more far-reaching lessons for the education system as a whole around how we develop our remote learning and teaching not as a substitute for face-to-face learning but as an additional strengthening of learning and teaching. Similarly, with regard to assessment, we need to invest—centrally in SQA and its successor body but also locally—in technology to e-enable assessment, so that we future proof ourselves against a pandemic or another scenario.

I would also reflect on co-creation, which has not been an easy journey because everybody comes to the table with different perspectives—quite rightly, because that is what co-creation is about. However, inevitably, that requires compromise on everybody’s part. You learn to listen and to really try to understand, and then you try to find a way through that. That was very challenging. We had weekly working groups and a weekly steering group meeting to help us to develop the ACM model, and we have adopted the same approach for 2022.

We also had a similar group for higher national vocational qualifications, with all the key stakeholders from colleges, training providers, sector skills councils, the Government and Skills Development Scotland, to help us make modifications to vocational qualifications. We would like some of those modifications to be retained as we move forward, because the situation has perhaps highlighted some areas of overassessment in our vocational qualifications.

I thank the committee for inviting us. I will add a couple of points to those that colleagues have made. I take Kaukab Stewart’s point about people in education having to turn on a sixpence, and we have learned that we have to build that into contingency planning for 2022. The 2021 ACM was a big model that involved a large part of the system having to change course. That has featured in our thinking for 2022. We have set out a series of contingencies, which mean that learners and teachers can focus on the business of teaching and learning, with a clear course set ahead. They know that, if exams are cancelled, there is a clear plan in mind and that, if there is further significant disruption, there is also a clear plan in mind. There should not be a dual assessment model or additional workload for teachers; qualifications should not get in the way of teaching and learning but, instead, should support teaching and learning.

We learn lessons, but the most important thing is how we apply them. I ask Fiona Robertson to give me a couple of examples of how the SQA will apply those lessons in the immediate future, for the year coming—2021-22—and in the medium term?

I ask Ms Robertson to give a quick answer, because we have other questions to ask.

I will endeavour to be brief. I will pick up a couple of things, including a couple of things that my colleagues have highlighted. We have already reflected a little on how the experience of 2021 has informed our approach to the coming year, and that is an important part of the development of the approach. In particular, as Beth Black highlighted, we need as far as possible to have clarity on contingencies at the start of the year and to ensure that the assessment burden is appropriate. I say “appropriate” because it is important that qualifications remain valid and credible. It is a serious business, but we also need to be cognisant of the fact that there has been disruption to learning, and we are still in extraordinary circumstances, with some disruption to learning evident in the system.

Gill Stewart’s point about communication and engagement with young people is important as well. Through our learner panel and other discussions with young people, we heard that they wanted to hear more directly from the SQA. A lot of the communication is, appropriately and importantly, through teachers in schools and colleges, but the young people also wanted to hear directly from us, so we endeavoured to do more of that in 2021 and we will continue to do so in 2022. Ensuring that we strengthen our engagement with young people through that process will be important to us and any successor body.

You have said that your pupils should be proud of their achievement and have full confidence in the results, but I want to know how you know that. In advance of the committee meeting, the SQA submitted a document that says that the results this year cannot be compared with previous years, but it goes on to say that people should have confidence in the system that produced those exact results. I wonder whether there is an inconsistency there. Why have you published those results if people should not draw definitive conclusions from them?

I will repeat what I said in my chief examining officer’s report, which is published on results day, because it is important. I understand the point that you have made. We have had a couple of years in which the assessment approach that we have needed to take in the circumstances has been quite different, but we have all pulled together to ensure that young people got qualifications that reflected their hard work. That is an important collective message for Scotland: that the 137,000 young people who got their results on 9 August worked hard and deserved those results.

To an extent, there is variation every year in attainment and in the composition of attainment between courses. Every year, there will be some differences in the attainment pattern, for different reasons. However, over the past couple of years, we have seen more movements in attainment than we would see in a normal year when exams are held. There are a number of reasons for that, such as disruption to learning; periods of remote learning; modifications to assessment, last year; and the absence of external assessment. The assessment approach was different. The flexibility in the way in which courses were considered and assessed by teachers and lecturers, which was required due to the high levels of disruption, may also have impacted on attainment.

It is important that we highlight the credibility of the qualifications, because we put in place what we felt was the very best approach possible under the circumstances. However, we acknowledge that there have been differences in the way in which young people have been assessed over the past couple of years and that that is reflected in the results. In the sense of a run of data, there is discontinuity, but those results are credible.

It is important that I highlight the quality assurance process that we undertook. Quality assurance is a really important part of any system of qualifications that are internally assessed. In the college sector, quality assurance is absolutely central to the work that is done year in and year out to assure, for example, that students who get a higher national diploma in one institution in Scotland have parity with those who get an HND in another. The quality assurance gives additional assurance—as the name suggests—that standards have been maintained across Scotland.

On the basis of samples of evidence, we provided schools with advice on whether the standard was being adhered to. A range of approaches were taken, including within schools and colleges.

We have had reports from pupils and teachers about inconsistency—not only between schools but between subjects—in the evidence that was provided. How do you know whether there was consistency of evidence across the piece? We have considerable evidence that contradicts that.

There were some flexibilities in the way in which young people could be assessed. Exams were cancelled because it was not safe to assess young people in the same way and at the same time in every school and college across Scotland, as we had done for the previous 130 years. That decision was taken by ministers and was based on public health advice.

The alternative certification model that had to be put in place needed to have sufficient flexibility to recognise that disruption was being experienced differently in different schools and colleges across Scotland. It was agreed that there had to be flexibility in the approach. However, the NQ group and the SQA also agreed on the centrality of demonstrated attainment—that evidence was very important in determining what young people achieved. We provided guidance, and my colleague Gill Stewart can say a little more about what that constituted.

Was the impact of the pandemic the only reason for the inconsistency? Were there no other reasons for a different application by different teachers in different schools? Are you sure that it was all to do with the pandemic?

10:00  

I would like you to explain what you mean by “inconsistency”. A moment ago, you were talking about the difference between the 2019 results and the 2020 results. Are you now talking about issues between institutions?

You asked teachers and schools to provide evidence of their pupils’ performance. Different teachers approached that in different ways. The approach differed between schools and between subjects for the assessments and qualifications in the past year. You said that that was to allow flexibility to cope with the impact of the pandemic. Particular outbreaks in schools were one impact. Are you sure that that is the only reason why the evidence guidance was applied in different ways in different schools?

We provided guidance that we expected schools to follow. There was some flexibility in that guidance. I would not say that there was inconsistency; I would say that there was flexibility for the right reasons.

Schools, colleges and local authorities—you heard from local authority representatives last week—provided advice and guidance on local quality assurance to ensure that the evidence that was produced within that flexibility had credibility. We also undertook quality assurance across every school and college in Scotland. In addition, we undertook lots of understanding standards events so that we could provide as much assurance as possible that schools and colleges were following our guidance to ensure that the qualifications were credible.

My final question is on the moderation process. You were still looking at historical results to question individual performance or class performance.

No, we were not.

There was pressure put back on and feedback provided to schools using historical information.

No, there was not.

The directors who were before us last week and the Educational Institute of Scotland acknowledged that the historical performance in schools was used in providing feedback. Are you denying that completely?

I think that you are asking me about what the SQA did. For its quality assurance process, the SQA looked at evidence that was requested from schools. No information on historical attainment was sought. We looked at the evidence in front of us and considered whether it was to standard. If it was to standard or was not to standard, we provided that feedback. In addition, we provided course reports for every course in Scotland to help in ensuring that schools understood the guidance and were taking account of the feedback.

On the actions that local authorities and schools took, some work was done by Education Scotland and inspectors. Local authorities were clear with schools that historical attainment could inform a conversation but that it would not define results.

The system is yours. You devised it in partnership with others. Historical information was used to provide feedback, but surely that provided a cosh for schools that were previously disadvantaged and provided poorer performance. That pressure was not—

The SQA did not—

Hold on a second. Let me conclude. That pressure was not on previously better-performing schools. Why was that appropriate, even in an advisory or feedback loop? Why was that ever allowed to happen? Surely we learned the lessons from the previous year.

The SQA did not look at historical attainment as part of its quality assurance, and it did not encourage—

But you did not stop that.

—or request schools to look at historical attainment as part of its quality assurance process. Ministers asked Her Majesty’s inspectors to look at the quality assurance process that was in place at the local level, and inspectors did that.

Last week, local authority representatives highlighted the approach that they took to quality assurance. I am seeking to explain the guidance that we provided to schools and the work that we did. I can assure you that historical attainment was not part of that.

I listened carefully to your first answer to Willie Rennie, in which you said that the past few years cannot be compared with any others but that you expect some change in grade outcomes. Therefore, if we go back to the old system for this year, do you expect grades and passes to fall?

We are very mindful of the need to be fair to learners who take exams in 2022. Very few learners will have ever taken a spring diet of exams, given the disruption that young people have faced. We have modified course assessment to take account of disruption to learning. That is really important. We will work through the detail of awarding in 2022. Discussions are under way on the nature of that—

What is your expectation? Do you expect grades to return to normal?

It would be wrong for me to speculate about grades.

Do you not think that, in fairness, learners deserve to know whether their grades are likely to reflect those from previous years or those from the two exceptional years with which you have said that comparisons cannot be made?

We do not grade on that basis. It would be very wrong for me to speculate on the precise outcomes in 2022. I am clear that we need to be mindful of the disruption to learning that young people have faced. We are giving further consideration to issues relating to awarding in 2022.

The modifications to assessment are not trivial. They are helping, and will help, learners during their learning and teaching experience this year.

I will follow on from what we have heard from my two colleagues who spoke previously. Last year was unique, but how did the SQA ensure that that year’s results were consistent across the country? Fiona Robertson has not spoken in any detail about that. We have heard from Willie Rennie and others that local authorities and schools were doing things in slightly different ways, so how did the SQA ensure that results were consistent?

I appreciate that I might be repeating myself a little, but I will do so in endeavouring to answer the question in the best way that I can. In developing the alternative certification model, we set out clear roles and responsibilities across the system—for the SQA as well as for schools, colleges, local authorities and others. That was important, because we were acknowledging that, to make the system work, everyone needed to play their part in ensuring that young people got their awards in August.

In broad terms, we set the framework for assessment in 2021. That involved some modifications that we hoped would help young people and ensure that teachers could focus on learning and teaching. Those modifications were made across all subjects. There were obviously particular challenges for some practical subjects—we can talk about that in more detail, if you wish.

There was also a big focus on understanding standards. In effect, we moved from what I acknowledge was quite a centralised system in which every young person took the same exam on the same day and they were all marked by the SQA, with a lot of processes and procedures around that, to a system of teacher-assessed grades in which individual teachers, and schools and colleges, made those determinations, with some guidance from us.

All the support that we could put in place was therefore really important. We held a lot of understanding standards events and provided materials, building on the significant catalogue of guidance and support that was already in place. We provided schools with guidance on how estimates should be done, and we received good feedback on that. We also discussed with schools and local authorities how they were undertaking local quality assurance—I mentioned the Education Scotland report that ministers commissioned on that.

Importantly, we also undertook national quality assurance. I will get Gill Stewart to say a bit more about that, because it is an important part of the assurance process. It involved every school in Scotland and samples of evidence from a number of courses to see whether assessment was appropriate and to standard. We provided feedback to individual institutions and courses, and we provided national reports to schools on the quality assurance process.

I acknowledge that there were challenges, given the extraordinary circumstances that we were facing, but I want to provide the committee with an assurance that everyone involved—local authorities, which you heard from last week, and all the teachers we spoke to in all our work during the year—was focused on ensuring that young people got the qualifications that they deserved. All parts of the system were working together in the best way that they could, albeit in very challenging circumstances—and, as Larry Flanagan said, sometimes in the context of quite challenging conversations—to ensure that we got the process absolutely over the line.

I ask my colleague Gill Stewart to say a bit more about quality assurance, because that is a big part of the assurance process.

We have a very strong programme of understanding standards. We provide a lot of materials for each individual course, which contain examples of performances at C grade and A grade, and we have done a lot of continuing professional development online with teachers to explore that material with them. We also have a lot of online materials, which teachers accessed in their own time to clarify the standards. That is a big plank of how we maintain standards.

At a local level, we have had very positive feedback from local authorities. They highlighted that one of the positive spin-offs from the ACM was that it helped to strengthen their subject networks and enabled those networks to play a strong role. The process is particularly challenging for someone in a one-person department; they may need to discuss the standard with a colleague and get some reassurance on what it is.

There was local assurance at a local level, and then—over a short window, working with teachers—the SQA carried out national quality assurance. In that process, we were not assessing individual pupils, but looking at each centre’s application of national standards from a sample of evidence. We sampled every centre—the number of courses that we sampled varied according to the number that a centre offered. If they offered a lot of courses, there might have been up to six selections, whereas, with a lower number of courses, there might have been just two or three. We considered whether the centre had applied the correct standard—for, for example, higher biology—in making its judgments. We then provided feedback to each centre on the evidence that it submitted. It was supportive feedback; we did not say that centres had to amend their grades.

10:15  

In the main, the headline figures show that centres were applying national standards; not a lot of centres were not applying national standards. Some were perhaps a little bit off in some particular aspects, and we provided supportive feedback that they could use locally to consider that and to have conversations in their centres.

In an ACM, there is a distributed set of roles and responsibilities for quality assurance. The SQA’s role is to provide clear guidance about the nature of assessment for each subject. In modern languages, for example, we assess speaking, listening, reading and writing. In maths, it is about how candidates integrate all their skills to apply operational mathematical processes and carry out reasoning across the course. In English, it is about assessing candidates’ writing and lots of different things through a folio that is generated in the centre. In physical education, it is about assessing candidates’ performance as well as understanding—[Interruption.]

A wide range of things is looked at. That is very helpful.

I am not sure whether this question is for Gill Stewart or for Fiona Robertson. What role did the SQA play in the overall quality assurance process?

We supported centres by providing guidance for each subject through our understanding standards materials to exemplify the standard and CPD. We provided assessment resources that centres could use if they wished, and we answered centre and local authority queries relating to individual subjects and so on. We therefore supported local processes of assessment and local and regional quality assurance.

At the end of that process, however, the SQA’s key role was to carry out national quality assurance, to consider samples of evidence across a small number of courses and to provide feedback to centres on their application of the national standard. Teachers from schools and colleges carry out that quality assurance and provide that feedback on behalf of the SQA, and we were mindful of the need to provide crisp, clear and supportive feedback to centres if there were any issues. However, not a lot of issues were picked up during that process.

The SQA’s primary role was therefore to provide up-front guidance and on-going support through the process and then to carry out the national quality assurance exercise at the end and provide feedback to centres on their local application of national standards.

I have a question on the same topic for Fiona Robertson. What role did the SQA have in feeding into what that quality assurance was going to look like?

If I have understood the question correctly, it was for the SQA to determine what the national quality assurance programme looked like. This year, we were again mindful of the circumstances. We had choices around the extent of quality assurance and how we provide it. We felt that it was important to cover every school and college—every provider—and to have good coverage across all courses. However—this relates to a previous answer—we were also mindful of the fact that schools had been doing remote learning until Easter and that we had a small window. The process was therefore proportionate and reflected the circumstances that we were facing.

There were choices to be made, and we made choices that fitted the circumstances of last year. That is quite important to note, and it was important that we did that. We sometimes get feedback that quality assurance can create workload issues for teachers. We did not want to create undue stress in schools, but we had to do our job, which was to provide the very assurance that members of the committee have been seeking. Quality assurance was actually pretty important.

I have been a member of the education committee in previous sessions and I have not always been kind to the SQA, but I am not sure that there was an awful lot more that it could have done in the circumstances that were thrust on it on this occasion.

I would like to follow up on something. I was a bit confused by the conversation that Fiona Robertson and Willie Rennie had earlier. Has the use of historical data ever been part of the SQA’s guidance?

It is important not to have confusion about the role that historical attainment data might provide.

Was it part of the guidance in the past? Was the use of historical data part of the process or the system in the past?

There is nothing to prevent schools or colleges from considering data in informing the evidence that they look at. It is important that I get this right, as I do not want there to be confusion on the issue. I think that Mr Rennie was asking whether, in 2021, historical attainment informed any moderation or any changes to grades, and the answer is no. The answer is no from the perspective of the SQA.

So, in previous years, yes, but in 2021, no.

We did not look at historical attainment when we were deciding what quality assurance to undertake. That was not part of the approach.

You said that historical attainment did not have a role

“from the perspective of the SQA.”

However, it is clear that historical data did have a role in the system. If a question was asked whether a pupil or a class was out of line with previous performance, a question was asked. Even if it did not lead to the SQA changing a result, that put pressure on the teacher, the class or the school to question whether the result was right. That never applied to better-performing schools, where a poorly performing pupil was never questioned, because the school was sticking in with historical performance. Even though you never asked and you never moderated, the system did, and you allowed that to happen. It was your system, so surely you should take responsibility for that.

I do not have evidence that that happened.

The directors said it last week. The EIS said it. The trade unions were very clear: questions were asked, and people did not like them being asked.

The NQ group, which includes the EIS and local authorites, discussed that issue on a number of occasions, and we concluded—this is very important—that learners should receive grades on the basis of the evidence in front of teachers. That evidence might be subject to quality assurance, it might be subject to questions from the local authority or, indeed, it might be part of cross-marking. A range of approaches were put in place to provide assurance about consistency—the very issue that you have been keen to stress. However, results should absolutely be based on the evidence of demonstrated attainment. That was at the centre of the approach this year.

Perhaps I can put the issue of historical data to bed a wee bit. I know that there were some comments that a small number of teachers felt that they were under pressure to lower grades. That was certainly said, although it did not involve big numbers.

My understanding from what the witnesses said last week is that they looked at historical data. If there was a massive difference, that would be a bit of a red flag, meaning that the grades should be reconsidered to ensure that they were right. Is that correct?

The Education Scotland report highlighted that, in some cases, historical data was used in that way to ask a question or to have another look at evidence.

The committee asked local authority representatives some of those questions last week. I can only say again that what was very important this year—and what we have said time and again—is that results were based on evidence of demonstrated attainment. It was not a model in which historical attainment played a part—that is key.

There is nothing wrong with using data to ask questions, if I may say so. Data can be very helpful for that, but we and all partners were clear throughout the year that decisions on grading were based on evidence of demonstrated attainment. The evidence was important. We did a lot of work to ensure broad consistency of evidence, as far as possible, in what was an extraordinary set of circumstances.

You said that you have read the Education Scotland report, which I would have expected you to do. What did you do in practice when you read the line:

“Local authority officers expect staff to use these tools to review concordance data, including young people’s prior attainment, and identify and address any unexpected provisional grades.”?

That does not fit with what you are saying today. Did you read that report and think that something was going wrong?

That extract needs to be looked at in the context of the wider report. My reading of the report, which I acknowledge has been out for a number of months, is that it highlights the range of strategies that local authorities—

The report says “Most local authorities” and it specifies that that means that, I think, 70 to 90 per cent of local authorities used three to five years of historical data and that local authority officers expected staff to use that to identify and address unexpected grades. That does not fit with the picture that you have given. As the person responsible for the qualifications that are handed out, did you not have a problem with that at the time?

We had some discussions about that in the NQ group, including discussions about the report. ADES representatives were clear that data was being used to ask questions and to look at the emerging picture, but evidence of demonstrated attainment by individual young people was the basis of the awards that were made.

You said nothing publicly and you did not raise any concerns, despite the concerns that were being voiced in the Parliament. Were you happy to let the report from the Government’s other main education agency sit there in the public domain and give the impression to young people that data from their school might be used to identify and address unexpected grades? Were you happy to say nothing about it?

In the discussions that we had in the NQ group and the communications that we issued, we were very clear about the awarding approach in 2021 and consistent in our position. I have been very clear about that position this morning.

The consequences of this seem pretty clear. The attainment gap between the richest and the poorest increased. The performance gap between state schools and private schools increased. The gap between disabled students and non-disabled students increased. Do you not feel that those are the consequences of the issue that we have just been discussing?

The attainment gap in Scottish education is longstanding.

But it increased this year, under this model.

Are you saying that it increased compared with 2020 or with 2019? There is a different picture depending on which year you choose.

It increased compared with the previous year. We have discussed putting the evidence of previous attainment into the model, as was done in local authorities and as Mr Mundell has just pointed out. The suppression of those grades is surely the consequence of the changes that you made, as the leader of the organisation over that year, to put in place that model.

10:30  

No, I do not accept that characterisation. I have made clear the position with respect to historical attainment and I have explained clearly what the ACM was and was not.

We have a responsibility to report on gaps in attainment, and we did that this year alongside other equalities information that we collected during the process. An equalities impact assessment was undertaken for the alternative certification model and the modifications to assessment that took place.

We have a responsibility to ensure that the assessment approach does not exacerbate gaps that exist in Scottish education, but attainment gaps have existed in Scottish education for a long time, for lots of different reasons. The attainment gap this year widened slightly across different groups, but the picture in 2020 and 2021 is, as I highlighted, quite different from that in 2019, when the gaps were much wider.

However, we are interested in the model and how it was applied. That is what members are getting at. It is clear from the data that the gaps increased. How do you account for the increase between the two models—those in 2020 and 2021?

I have highlighted the fact that there were differences in the assessment approach in 2020 and 2021. There were also differences in the experience of learning and teaching in both of those years. That is crucial. The Scottish Government’s equality audit highlighted issues with respect to deprived young people and their experience of learning and teaching in 2020 and 2021. That is an avenue that the committee should explore, because the learning and teaching experience is the most important element in determining what young people can achieve in school.

That is fair, Ms Robertson. Are you saying that the data applied had no role in creating the gap?

I am saying that every part of the system this year worked together to ensure that the alternative certification model was based on evidence of demonstrated attainment for every learner. That was the basis on which learners received their awards, and every part of the system sought to ensure—

To be fair, you have said that several times now. We will move on.

I will return to the convener’s line of questioning but ask the question in a slightly different way. Are the OECD’s recommendations on assessment right? From your professional experience over the years and your experience at the SQA, are the changes that it identifies the ones that the Parliament should follow?

There are two OECD reports. The first did not focus specifically on assessment, although it highlighted curriculum alignment and assessment issues. We published our submission to the OECD, which documented the journey of reforms to qualifications, particularly the development of curriculum for excellence.

The Scottish Government commissioned Professor Gordon Stobart to undertake an additional report, which was published a few weeks ago, with respect to assessment and qualifications. That was a comparative study that looked at different countries and at what he called the “British tradition” of exams—the report highlighted the fact that Scotland and other parts of the UK have a tradition and culture of exams. An important part of what Gordon said is that assessment and qualification approaches are, in effect, a cultural phenomenon that reflects the culture, practice and capacity of the system. I agree. Therefore, in thinking about further changes to assessment and qualifications, it is important that we also highlight those wider issues around change.

However, do you think that the initial recommendation—to move the exam assessment part of our system in with the curriculum part—is right?

Are you talking about organisational change or changes to assessment and qualifications, or both?

First, I am talking about the first OECD report, which suggests that the SQA, or an equivalent body, should be merged with the curriculum body. In your experience, do you think that that would be a good move?

Ministers have set out their position with respect—

I am not asking about ministers. Given your professional experience as someone who has headed up Scotland’s exam body, if anyone was going to stand up for exams and make the case that what we are doing at the moment is right, it would probably be you. Is there another side to the story that the Parliament should think about? Do you think that those recommendations are right?

There are issues to consider around both function and form. The organisational structures that exist across Scottish education, including the national bodies, are important, but, in considering the constitution and structure of those bodies, it is also important that we are clear about what we want those organisations to do. That is an obvious point, but it is really important.

My observation is that we have significantly reformed Scotland’s qualifications, alongside the development of curriculum for excellence. Between 2014 and 2016, there was a balance of continuous assessment, through a unitised structure that was continuously assessed in schools, alongside exams and course work. Following the removal of units in 2016 by the Scottish ministers, there has been a slight move back to greater emphasis on exams.

A lot of good education systems have a balanced approach across continuous assessment, external assessment—including exams—and other forms of assessment such as course work. Sometimes, the debate around assessment and qualifications feels quite polarised. Even at the moment, many of our courses do not rely fully on the final exam in the spring; only a relatively small number of courses rely on that.

As I highlighted in my opening statement, it is important to reflect on the experience of the past couple of years. In considering those issues, some committee members’ questions today have been on their concerns about the alternative certification approach and, in effect, that kind of federated system of assessment.

My final question reflects on one of those issues, although I am not enthusiastic about asking it. I absolutely believe that all young people deserve the grades that they have got in the past two years, and people feel positive about seeing young people from more challenging backgrounds do better than they have in the past. I would like to see that continue, but are there unintended consequences of grade inflation? Do you think that we should be mindful of that?

It is not a popular subject to talk about, but does that bring other challenges with regard to what a qualifications body should be doing? That goes back to my previous question about whether—

Please get to the question, Oliver.

—there was the number of A grades that you would want to see in any qualifications system in order to get that differentiation.

I think that I have highlighted, in answer to previous questions, that the pattern of attainment looks different and has looked different over the past couple of years. There are a number of moving parts that might have contributed to that.

The awarding of qualifications is a serious business. We have a responsibility—and, indeed, as things stand, a statutory duty—to determine both the level of competence that is required to gain a qualification and the means of assessing learners to determine whether they have gained that qualification. That means that we need to be clear about our expectations of the system, and, sometimes, we have to say that the level of competence has not been met, which is hard, is it not?

I welcome a debate, here and elsewhere, about how important qualifications are, what they do—the function that they fulfil—and how we go about the process. That debate is to be had. Some of those issues are—rightly—policy issues for the Scottish Government to consider, and, as chief examiner, I will play a full part in those discussions. I am sure that the cabinet secretary will want to talk more about that next week.

In 2020, you pushed for the algorithmic element to keep grades where they would have been expected to be. Do you think that that is important in the system—

Please give a short answer to the question, so that we can move on.

Standards are important, and they are part of our responsibilities. The qualifications that young people get are important at the time and remain important over time. All of us will have been through that. At different periods in our lives, our qualifications remain important with regard to what happens next. That is why we did what we did when we were asked to maintain standards and to develop an alternative certification model in 2020. We were commissioned to do that, and we did it to the best of our ability.

We have a responsibility to maintain standards over time but, in addition, we have been through a pandemic and we have had to put in place very different arrangements. We have worked very hard to do that in the best way that we have been able to, and I am glad that you have acknowledged that young people got the qualifications that they deserved, because it is important that we all get behind young people to say that.

There is no division on that at all—we are united in our support for our young people.

I am glad to hear that.

Fergus Ewing, do you want to come in on the OECD report?

Yes, thank you, convener. No country in the world suffering from the global pandemic had a syllabus ready and waiting to instruct us on how to proceed when schools were disrupted and shut down. Hindsight is a marvellous thing, is it not?

I would like to look forward, not back. I have two questions, the first of which arises from the OECD report. I am sure that Fiona Robertson and her colleagues will have read the evidence. I was struck by the very positive comments made by, for example, Beatriz Pont, who said that

“Scotland was among the leading countries in global competency proficiency”

and that

“in terms of equity, Scotland is above average across OECD countries.”

She went on to say that the OECD saw curriculum for excellence, which I want to question Fiona Robertson about, as being

“a pioneer among education systems internationally.”—[Official Report, Education, Children and Young People Committee, 8 September 2021; c 11-12.]

There were lots of positives in the report.

However, for me, the takeaway from the OECD—the central conceptual thrust of its criticism—was that the four capacities of successful learners, confident individuals, responsible citizens and effective contributors needed to be better worked on and assessed in the overall system. In other words, it is not clear at the moment how we assess whether individual children, pupils or learners have or have not attained those capacities.

That seemed to me to be the central tenet of the criticism. If that is correct, how do we address that in the future? What needs to be done to take Scotland forward, if you like, and ensure that we can drive the CFE forward in a way that allows us to say with confidence, “Yes, our children are successful learners, confident individuals, responsible citizens and effective contributors”?

10:45  

I am happy to answer those questions.

With regard to CFE, I agree with your sentiments and those of the OECD. Indeed, the previous OECD report in 2015 highlighted the pioneering and innovative nature of curriculum for excellence and its endurance over time, and other countries have followed its broad approach. It provides a very strong basis on which to build.

As for the four capacities of CFE, one of the emerging issues in the OECD report was that somehow the qualifications and assessment system focused on one of the four capacities—successful learners—and, as a result, attainment was seen as part of quite a traditional model. In the “Building the Curriculum” documents for curriculum for excellence—I should perhaps say that, at the time, I was the chair of the curriculum for excellence management board; I was not in the SQA but had a wider role in respect of CFE—there was a lot of ambition with regard to how curriculum for excellence would develop, including in the senior phase and in curriculum models that might emerge over time.

I am actually pretty confident that the four capacities of curriculum of excellence are covered in the suite of qualifications that the SQA offers. We provide leadership awards, mental health awards and a huge range of other awards. I absolutely accept that curriculum for excellence should not be valued simply in terms of the qualifications that a person leaves school with. That is important. After all, we place value on lots of things that we do not measure, as well as on some of the things that we do measure. We offer a large suite of awards.

We also need to consider issues with regard to the curriculum models that are followed right across the system and the choices that young people have—or, in some cases, do not have. Some really good work has been done in that respect; in fact, I commend to the committee a report that has been published in the past couple of weeks on school-college partnerships and their integration of the vocational offer in schools across Scotland. It contains some really important lessons on what works well and highlights some perceived barriers. There is more to do to ensure not only that the offer is there but that choices are available to young people in schools and colleges and that the curriculum offer is such that they can take advantage of those opportunities.

You have given us some very good examples. I notice that the OECD witnesses told the committee:

“Scotland is viewed internationally as an example of high performance. When we compare the data with that from other countries, we see that Scotland is above average on a number of indicators, especially the OECD’s new indicator on global competencies.”—[Official Report, Education, Children and Young People Committee, 8 September 2021; c 30.]

It is easy to forget all the positives in the OECD report.

I would like to ask one more question, which is of a more practical nature. It arose from the comments that Fiona Robertson made earlier about the importance of the need to help children to prepare for examinations next spring against the background of the past 18 months or couple of years, when they have not been used to examinations.

It is a long time since I sat my last examination at school—it was five decades ago, I think, which is such a long time ago that dinosaurs were prowling around outside the classroom cave—so my experience might be a little bit dated. However, I guess that the essentials of exams, from a child’s point of view, do not really change: there is work to do to prepare for the exam and there is inducement to work and prepare, and there is also anxiety and a fear factor, which involves the unknown and the consequences of what the child does in the exam. All those elements are constants, irrespective of the passage of time.

How, in practice, can we best equip and prepare our learners—that is the word that we use these days—for examinations, given that they have not been used to doing examinations recently?

I want to make a particular suggestion that might or might not be of use. Is the use of practice specimen papers, which give children the chance to rehearse and try out the process of an examination before sitting the real thing, a structured part of the system these days? It could be helpful, because it would mean that children would not be going in cold to an experience with which they are completely unfamiliar. To me—based on my long-forgotten examination preparation—the fundamental part of preparing yourself for such an experience is actually trying it out in peacetime before the real thing.

Schools and colleges are experienced in ensuring that young people are able to prepare in the best way that they can. Many schools have maintained prelims and other sorts of assessment over the past couple of years. Young people are assessed on a range of things throughout their school lives, so assessment, as a thing, will not be new to many young people. I accept that, in the spring of next year, the exam diet experience might well be new to them, but schools will be very aware of that and will be developing approaches to ensure that the experience is as stress free as possible.

I would like to explore the functioning of the NQ21 group, and specifically the level of participation of those involved. You will be aware that, on the day that the appeals process was confirmed, Cameron Garrett, who was the member of the Scottish Youth Parliament on the group, said:

“As the only young person who sits on”

the SQA’s NQ21 group

“and the only member representing young people, I have not had an equal input into discussions around the appeals process this year at NQ group meetings.

Young people have been let down and ignored by this process.”

He went on to say that organisations such as the Children and Young People’s Commissioner and the people who are involved in the SQA: Where’s Our Say? campaign, as well as the Scottish Youth Parliament,

“have been calling for a no-detriment policy and exceptional circumstances to be taken into consideration as substantive points. Neither have been considered in this process.”

Subsequent to those comments being made, did you reach out to Mr Garrett to better understand why he felt that way about his experience? Could you talk a bit about what you believe you have learned from those discussions and how subsequent processes to this one can more effectively involve the voices of young people?

That is an important point. Cameron Garrett has been a helpful and good member of the NQ group, and it has been good to see him on a weekly basis, alongside colleagues from the Scottish Youth Parliament. Indeed, we commissioned the Scottish Youth Parliament to deliver our learner panel this year. There has therefore been a variety of involvement. I have, of course, had a number of discussions with Cameron Garrett and others in the SYP. I am keen to understand and to take all appropriate steps to ensure that our engagement with young people is as good as it can be.

There are two issues with respect to the appeals process—one around the operation of the NQ group more generally and one around the particular issue that Cameron and SYP colleagues felt so strongly about that Cameron said what he said.

Larry Flanagan highlighted some of the issues in relation to the NQ group. It is a large group with a lot of loud voices and strongly held views. Gill Stewart highlighted the issue of our having to work through some of those complexities and come to agreement, which was sometimes difficult during the course of the year. I think that Larry alluded to that. We have agreed with the Scottish Youth Parliament that we will increase the number of young people on the NQ group this year, which is in train. The number and variety of voices is important, so we will do that. That was good feedback, which we are taking on board.

In relation to the appeals issue specifically, with the enactment of the United Nations Convention on the Rights of the Child and, of course, the Priestley review, and through discussions with Government, there was an expectation that we would introduce a learner right of appeal this year, which we thought was important.

As the committee will know, although appeals previously absolutely involved young people, they were initiated by the school or college; indeed, in previous years, the final decision on whether to institute an appeal was taken through the school or college. This year, we were met with a different set of circumstances in relation to the alternative certification model and the expectation—which was perfectly reasonable—that we would have a learner right of appeal. We thought that it was important that we consulted on that, so we undertook a public consultation on the appeals process this year.

There was a lot of discussion, including learner panel discussion. I attended the learner panel discussion about appeals and, in particular, the symmetry or otherwise of the appeals process—that is, whether appeals should be able to result in not only an upgrade but also, potentially, a downgrade, as well as the grade remaining the same, or whether there should be asymmetry in that the grade should be able only to go up or stay the same. In addition to that, there were issues with respect to exceptional circumstances, which I am happy to go into if the committee wishes.

We had lots of discussions about that. The issue went through the SQA’s governance processes—it went to our board and through our qualifications committee. There were also lots of discussions about it with the Scottish Government. We decided that the appeals process should be symmetric—that is, that grades could, in principle, go up or down or remain the same. Obviously, that position was not supported by the SYP and Cameron Garrett.

However, I was keen to highlight to Cameron that we took the feedback from young people very seriously. There were lots of discussions about that both with them and more widely. We took the decision on the basis of fairness, recognising that it was an evidence-based process. We felt that, if we saw evidence that an award had been given incorrectly as a result of an appeal, we needed to take action. We took the decision on that basis after much consideration.

11:00  

I am sorry to jump in. I do not particularly want to pursue the specifics of the appeals process, but, given that you have raised the issue and that we have had exchanges about it in the past, I note that there was not a particularly large number of appeals this year. Did any of those appeals result in a downgrade?

I cannot share the outcomes of the appeals process because it has not yet concluded. We will produce official statistics on appeals outcomes at the end of the year. I am happy to share those with the committee when the information is available.

At the time, we said that there has always been a symmetric appeals process in Scotland and, indeed, in other parts of the United Kingdom, including for this year. Downgrades are very rare—they do not happen very often—for all the right reasons: the system gets it right first time most of the time. We took our approach to ensure that we got it right first time, but an appeals process had to be in place, because that was an important final part of the process.

I am happy to provide further information on the outcomes of the appeals process when I am able to do so.

Thank you. In a similar vein, the Children and Young People’s Commissioner Scotland made some pretty scathing comments about the concept of co-production. Dr Stewart said that that is an area of learning from the process over the past year. Bruce Adamson, the commissioner, made the same point that Cameron Garrett made. He said:

“Some very strange examples of ‘co-production’ being discussed at the moment. If you have very limited involvement of young people and then ignore their views, you can’t call it co-production just because they were in the room.”

My question is similar to my question about Cameron. Have you met the commissioner subsequent to those comments being made to discuss his concerns about the concept of co-production?

I have not.

I will explore that issue in a little more detail. Dr Stewart, you mentioned learning around co-production. What did you mean by that? Have you taken into account the concerns that have been raised about co-production and participation not just by the commissioner but by a range of other experts in the field of children’s rights?

We are all learning about what is best in that area. To support the work of the NQ22 group, we have supplemented the steering group and the working group with additional members, as Fiona Robertson said. We are also looking to expand the learners panel by including a wider range of young people on it. It is only by engaging throughout the process that we will improve our understanding of co-production and learn how best to do it. Do I have any magic answers? No, I do not. However, we are trying to increase the level of learner engagement, and we are in active listening mode to hear about how best we might do that.

I will reflect on a different piece of work. Prior to the Covid pandemic, we worked with young people and teachers on the future of assessment. That was very interesting, because the young people’s ideas about assessment were very different from those of their teachers. There was co-production in the sense that young people made suggestions. Teachers did not pooh-pooh those, but they pointed out some of the practical difficulties.

Different conclusions were reached about how best to assess in particular subjects. The one that stuck in my head was about history. Teachers often want to assess in the way that they have always assessed, because they know how to help young people through that, whereas young people come at it from the perspective of wanting more of the work that they produce throughout the year to be taken into account. To me, co-production is about getting the two sets of people together to co-construct what the future of assessment should be in different subjects. I use that as an example.

The issue is challenging because the young person and the teacher do not necessarily have the precise answer, but, by bringing people together and getting them to actively listen to one another, they can come up with ways forward. Those are the sorts of learning experiences that need to inform the future of assessment.

I appreciate that, and I entirely understand that there is no easy answer to the question. However, given the comments of the children’s commissioner, I would expect you to engage with his office and with others to understand those concerns. I know that Dr Tracy Kirk has engaged with you on those issues.

There has been engagement and correspondence with the children’s commissioner, and I am very happy to continue that. What Gill Stewart highlighted in relation to seeking to understand the different perspectives and responsibilities that people have is really important. At the start, I highlighted that a serious set of responsibilities is involved in the awarding of qualifications. I have statutory responsibilities and functions to fulfil. It is really important that we engage in conversations to seek to understand and to reflect, but it is also our job to ensure that we fulfil our responsibilities as an awarding body. Unfortunately, that sometimes means—I say this with some regret—that we are not able to do all the things that everyone wants.

I accept that.

I am conscious of the time, and I am probably intruding on the time of other members. If I could come back in at the end, that would be appreciated, but I understand if I cannot.

That is very generous. We will take you up on that.

I am interested in some of the questions about form and function, as you have put it. We all know about the very important job that the SQA has to do over the coming year, and the pandemic challenges remain vast for the education system.

Fiona Robertson, on 21 June, you issued a statement that welcomed the announcement that your organisation was going to be scrapped. Did you consult the SQA’s staff before you issued that statement?

I did not use the word “scrapped”. That is quite important. I highlighted the announcement that the cabinet secretary had made.

You said:

“I welcome the ... announcement of a new specialist agency with responsibility for both curriculum and assessment. This is an opportunity for significant change”.

Yes, I did say that. It is important to be accurate about what I said. I appreciate your repeating it.

I had a meeting with all the staff on the morning of the announcement, but the statement was mine. It was absolutely in my name. Obviously, I had discussions with the board and all the staff on the morning of the announcement. It was very important that I did that.

There was significant upset among staff. I have spoken with trade unions, which have said that staff were upset by the announcement and the way that it was welcomed by the organisation’s leadership. Is it fair to say that?

I have good on-going engagement with colleagues across the SQA and our two recognised trade unions. I highlighted in my opening statement and I have highlighted to staff that it is important that we engage positively with the review process. Members have highlighted in their questioning today more broadly the need for learning, reflection and some change. It is important that we as an organisation, along with other organisations in Scottish education, reflect on the need for change.

In answer to your question, I highlight very strongly the commitment, professionalism and integrity of SQA staff—every member of staff—in the face of quite a challenging period and quite a lot of political comment, which has, in my view, been unacceptable at times. We have had a set of responsibilities to fulfil, and a clear commission from Government to undertake tasks, and we have done so to the best of our ability. When people like you use adjectives such as “scrapped”, “abolished” and others, you will get a reaction from staff and from me because there is, in the SQA, a lot of skill and expertise that Scotland needs and will need in the future. It is really important that I highlight that to the committee this morning.

It is really important, and I welcome the fact that you have put that on record. I know, having spoken to the trade unions in the SQA, that there was real concern about the way that that happened. I am interested in the relationship between leadership and expertise. With regard to the model for 2020, did staff make representations to you that it would be the “disaster”—in their words—that it turned out to be?

During the 2020 process, we had a number of conversations in the SQA across a range of issues, in which we considered some of the challenges of awarding in 2020. There is no doubt that it was a very challenging period. My colleague Gill Stewart led the approach to the process in 2020, and I think that she would confirm that it was a challenging period, in which we were concerned to deliver on the commission from ministers and to get it right. Delivering, and getting it right, was our concern, and we had a range of conversations around that.

I appreciate that.

Given some of the previous questioning, I am a little worried about the distance from Government and the extent to which some of the advice and expertise that can be drawn on—which, as you rightly reflect, is so important to the education system—is independent.

I want to look forward a little to next year, if I can, to pull everything together with regard to the plans that you have set out. In my view, the guidance that was issued on 15 September was lacking detail and clarity. I recognise the context in which everyone is operating, but young people and teachers need clarity. That has come through very strongly in the evidence that we have heard from young people—there was a complete lack of clarity last year and even less the year before, so we are looking for more clarity this year.

In particular, the guidance sets out that, if mitigations to exams are needed, those decisions will be left until March next year. It does not set out any detail on what criteria those decisions will be made on, so any comment on that would be useful.

In addition, it does not say whether decisions will be made pupil by pupil, school by school, council by council or Scotland wide. At what level will the approach be decided? Will disruption be experienced differently by pupils? We need more clarity on that, so it would be helpful if you could give us some comment and provide clarity for those young people.

First, I totally understand the need for clarity. Looking back on 2021—my appearance before the committee this morning led me to look at all the communications that we issued in 2021—I was struck by the fact that we were dealing with a lot of change in real time. In general, as an NQ group, we were mindful of the fact that we did not want to bombard the system with changing guidance. Things were changing every five minutes, but, at the same time, we wanted to provide as much clarity as possible. There is a tricky balance to strike between how much we provide and when we provide it. None of us wanted to be seen to be chopping and changing in seeking to provide clarity. It is important to find the right balance in that.

11:15  

As things stand, Scotland is the only part of the UK that has been clear on arrangements for 2022. We have come out quite early—that context is important—and set out the contingency arrangements that will be in place for 2022. My colleague Beth Black will be happy to talk about them in more detail, but they are largely national contingencies.

If exams were to be held, we would expect that, as appropriate, exceptional circumstances would apply if there were significant issues that would impact on young people’s performance at the time of an exam. That happens every year. In some cases, those circumstances are highly individualised—a bereavement or an illness—but, in other cases, the situation is a little broader than that. For example, in recent years, we have had school closures for different reasons. Fires and other things have happened to schools and exceptional circumstances measures have kicked in. We have discussed with schools changes to deadlines and other arrangements that can take place. We have flexibility to ensure that, if circumstances apply locally or individually, measures can be put in place to mitigate their impact.

On the contingencies, in broad terms, we are saying that we could consider further modifications if there was significant further disruption to learning beyond that experienced in the 2020-21 academic year. We need to take advice from the education recovery group, Public Health Scotland and, indeed, the Scottish Government on that. That context is important. The further contingency is that exams would be cancelled only if public health advice in the spring was such that they could not be held because they were gatherings of young people.

I will ask my colleague to talk through that.

I would appreciate that clarity from Beth Black, but I have one question on that answer.

There are some other questions in this line of questioning, Michael.

In recent weeks, the level of absences in schools has been equivalent to the level when we cancelled exams last year. Do you have reflections about lost learning and where we might be at the moment? I do not mean to be alarmist, but do you agree that it is appropriate that we consider that in the decision-making process?

At this point in the year, the focus is not on assessment but on learning and teaching. That is important. Schools and colleges have responsibility for ensuring continuity of learning and teaching even if there are issues with absence levels. We are aware of the point that you make, but the focus at this point in the year is absolutely on learning and teaching rather than assessment.

Beth Black can highlight some of the measures that we could put in place should it be necessary. Keep in mind the fact that we have made modifications to assessment so that learning and teaching time can be maximised. Those modifications are in anticipation that there will be disruption to learning this year. We have taken that step up front and have reduced the assessment burden so that it should be more straightforward for young people and allow for a degree of disruption, should that happen.

Michael, before we hear from Beth Black, do you mind if I bring in Bob Doris? He has a question on this area. If we hear that, Beth can give us a fuller answer.

Thank you, convener, and I welcome what you have said.

My colleague Michael Marra’s line of questioning has been really helpful. I know that initial guidance has been issued, and we are all holding our breath for the more detailed guidance—perhaps that is what we are going to hear about from Beth Black. Looking at the guidance that is out there, though, I see that there are three scenarios at present: first, we run the exams as planned, with the modifications to assessment that Fiona Robertson has highlighted; secondly, we have an additional modified process with exams; and thirdly, we have some form of alternative certification model. It is that third scenario that I want to ask about.

The guidance refers to

“the type, quality and volume of evidence that would be needed to support quality assured estimates in a ‘normal’ year”,

which would be used to

“support ... provisional results”.

It goes on to say:

“Provisional results would be based on in-year assessments that normally take place during the school year such as prelims, practical activities, performances and class tests.”

There are, in theory, three different models. In two of them, exams take place, although modified, and in the third, exams do not take place but there is an alternative form of certification. However, according to the guidance, those are the types of assessments, observations and evidence gathering that teachers would be doing anyway. My question for Fiona Robertson—or, indeed, Beth Black—is, what is the actual difference here?

Beth Black will cover that and will answer Michael Marra’s question, too.

I hope that I can answer Mr Doris’s question, although I was slightly confused at the end of it about the difference that he was highlighting.

Modifications are already in place to acknowledge and anticipate disruption. As Fiona Robertson has said, they are significant—they are not trivial—and they take account of the assessment burden. As a result, teachers and educators can concentrate on teaching and learning.

However, we are keeping a watching brief on the disruption, because we are very aware of it and the trouble that it is causing in the system. If, beyond a certain point, there is disruption in the region of that which we saw last year, there will be further interventions to support learners in the final run-up to exams. They will help take the heat out of the revision period and support them through it. It is very difficult for us to put numbers on these things or to announce any numbers, because they become targets and everything then becomes incredibly complicated. It is therefore difficult to be entirely reductive about this, but we are definitely keeping a watching brief on it.

Assessment is a naturally occurring part of teaching and learning, so, instead of bringing in, possibly at very late notice, a large ACM, which would involve all sorts of semi-formal assessments, the instruction, if exams had to be cancelled for public health reasons in the spring, would be that the normal assessments that teachers rely on in their normal working lives would form part of the support for the provisional grade that they would submit. The ordinary practice in any year is that teachers submit estimates to the SQA just before exams. In an ordinary year, the estimate is prepared, and teachers are mindful of how to collect it. They know how to make judgments about the evidence in front of them or can further support that judgment with the understanding standards work, which we have heard was very popular and was welcomed by teachers because it increased their capacity and capability in that respect.

All of that feeds into the normal practice of preparing estimates. Should we face in March or April the worst-case scenario of large public gatherings not being allowed to go ahead—and we hope that we do not—that normal activity will get repurposed for the provisional grade. The advantage—and this is quite an important point—is that teachers should not worry about undertaking additional assessments to keep in their back pockets and learners should not worry about being dual assessed.

That is not everybody’s first choice—it is not the first choice to cancel exams. It is very much a plan C, but we need to have a contingency in place. We have listened to what the system has told us about the workload involved in the ACM. That is part of the thinking in the plan.

We will go back to Bob Doris for a follow-up question, and we will then go to Michael Marra.

That was very helpful. Beth Black said that she was unsure what I meant when I asked what the actual difference was—that is what I was trying to tease out. I think that the SQA is talking about embedding the normal, day-to-day practice of teaching professionals into any alternative certification model if that is what we have to end up with. That is very helpful.

I am wondering what role moderation would play within that process and in quality assurance. We still have the same situation with one-person departments and different approaches within local authorities or among different local authorities. It would be helpful to know about that as well.

I am not sure whether I will get back in a second time, so I will ask a second question—I would like to explore some of these matters further. Teachers will feel very empowered now, as they should do, such that, when they put in an estimate for a young person, that will be a true reflection of the competencies at which that young person will be operating. After all, that is what teachers have been asked to do with moderation and quality assurance.

We then go to the exams. As we know is the case with any exam, not every young person will perform as well as anticipated in those exams, and that is where the appeals process comes in. Has the SQA considered that, should the exams go ahead next year, as I hope they do, we can anticipate many times more appeals than before, given how teachers and young people feel empowered with high-quality estimates showing those young people operating at a very high level? If young people do not perform in that way in exams, a significant amount of appeals are likely to come forward. Has the SQA given consideration to that?

I am happy to start with an answer to that, and Beth Black can then come in.

I am mindful of the time remaining.

Yes—I will be brief if I can be.

On quality assurance, if we were in a position where exams could not go ahead, then, for all the reasons that we have discussed during this morning’s session, some quality assurance would be an important part of the approach that we would take, keeping it in mind that evidence is key. There will be some further guidance around estimation over the next couple of weeks to crystallise things. We would not wish to create unnecessary workload, however—we have been very conscious of that this year. All of the understanding standards work that Gill Stewart has highlighted will continue to ensure that the programme is developed and delivered.

The appeals process that we had this year obviously related to the approach that we took this year. It was absolutely learner centric and learner initiated. We will be considering the appeals process for 2022, and I anticipate that it will involve a learner right of appeal, for all the reasons that I have highlighted. We will be considering the detail of that. There is an existing appeals process that we can use, which has been in place for a number of years up to 2019, but we will be doing some further work to establish in more precise terms what that appeals process looks like.

Your point about high-quality estimates is important, Mr Doris. The communication between the school or college, the teacher and the young person is really important in setting expectations and aspirations about what young people might be able to achieve. That is an important aspect of the good, high-quality discussions that go on day in, day out across Scotland in relation to how young people are doing and progressing in their learning and what they are likely to achieve, either through an alternative certification model or through an exam process. Those are very important conversations to have.

I ask Michael Marra to come back in, but to be brief. I ask for brief responses, too. Stephanie Callaghan has been very patient, and we must get to her questions.

You talked about a watching brief. I will not ask for the number of days of disruption or anything like that. If schools in one part of the country or one local authority are significantly disrupted and schools in another area are not, could we see different approaches for those different areas? For example, could exams be cancelled in Glasgow but not in Edinburgh? We are talking about a national approach. I see lots of shaking heads. Fiona Robertson talked about exceptional circumstances and taking individual approaches into account. How do you square those two things?

11:30  

We would be looking at a national exams process. The exceptional circumstances that I have highlighted have also existed in previous years. Dr Stewart might want to say more about that. That is important because they can apply individually, and it is the individuals who are important here. Circumstances can apply individually for very particular reasons and can apply more broadly if a school or a cohort of young people has been affected in a particular way. In 2020-21, the whole year was about exceptional circumstances. The cancellation of exams by ministers reflected the exceptional circumstances that were experienced across the country, and the flexibility that schools were able to deploy was the way in which they could use the exceptional circumstances to ensure that young people got their qualifications.

Thank you for that clarification.

Does Bob Doris want to ask a follow-up question? It has to be brief.

Does the SQA anticipate a successful appeals process in the coming year? Teaching professionals always do a good job at estimating grades, but, over the past couple of years, they have had to follow a very detailed and specific approach to evidence gathering and submission under the alternative certification model. Can we anticipate a robust system of estimates, and, if those estimates are of high quality, should we expect more successful appeals in the year ahead? Should young people know the estimates before they walk into their exams?

I appreciate that Bob asked that question earlier.

It is difficult to speculate on the volume of appeals in any year. In relation to your point about the experience of the past year, a lot of discussion in departments, subject teams, schools, local authorities and regional improvement collaboratives has enhanced the understanding of standards in Scotland’s schools, which is a good thing. There has been positive feedback from schools and colleges on that.

I do not doubt that, in any year, teachers do their best to ensure that young people are closely involved in discussions about their progress. The feedback that we have had this year is that the focus and responsibility that teachers have had in determining grades has involved a lot of investment, hard work and commitment to understanding standards and applying them appropriately.

For the final round of questions, given the time constraints that we have, I hand over to Stephanie Callaghan.

Over the past couple of years, you have had to make lots of huge decisions and do a huge amount of work in a very short period of time, and that work is not necessarily what you have been used to doing in the past. I am interested in the work that you are doing now, and I go back to what Ross Greer talked about earlier. What work are you doing on plans and strategies for models of co-production? Given young people’s views on parity, and taking on board teachers’ views, are you able to collaborate and take decisions together?

That is a good question. I alluded to some of that work during earlier questioning. There are a number of strands of work around communication and engagement more broadly. The appointment of a new director of communications has enhanced our internal capacity in relation to young people and parental engagement. Communication and engagement are really important, and there will always be a variety of views about how effective or otherwise they are. I could go through all the work that we did on that this year in some detail. We did a lot of communication and engagement, including bespoke communication with young people and parents that was informed by feedback from those groups. However, it is fair to say that there is a never-ending appetite for that. We need to make sure that we do it in the best way that we can and that we mix up the approach through social media and other things.

In relation to engagement, I have highlighted that we are enhancing young people’s representation on some of our groups, including the NQ group. We are expanding the learner panel, which is really important, and ensuring that it has a forward work programme. As I said, a lot of the work was done in real time last year, and it was sometimes felt that there was not the lead-in time to things that we would have liked. The review that Ken Muir is undertaking on some of those issues will have a big focus on any successor arrangements, including the successor to the SQA. The reforms to Education Scotland will also look very closely at young people’s engagement, in particular.

Gill Stewart has highlighted the work on assessment futures that we did before the pandemic and before my time. There are good foundations for some of that. Some of it gets into the policy space, so the on-going discussion and engagement with the Scottish Government on those issues are important.

I hope that that is helpful as a fairly high-level summary of some of the work that we are doing. We are keen to take account of young people’s views and ensure that they are integrated into the work that we do as far as possible. We are here to serve learners. A lot of our work supports the profession to serve learners, but our job is to serve learners.

I trust that the answer to this question will be yes, but I take it that that will include care-experienced young people, children with additional support needs and so on? We often find that the things that work for those pupils work really well for other pupils as well.

Absolutely. We have an annual event with Who Cares? Scotland on results day, which we have had to do virtually the past couple of years. During my first year at the SQA—in 2019—that event was attended by the First Minister herself, who gave certificates to care-experienced young people and reflected on their very impressive achievements, given some of the challenges that they had faced. The short answer is therefore yes—absolutely. That is really important.

We have in place a substantial piece of work around assessment arrangements for young people who, for a large number of different reasons, cannot take exams in the standard fashion. We do a lot of work to support young people to gain qualifications each year and engage very closely with centres to make sure that arrangements are in place, whether that is scribes, font size or typeface. We can put in place an impressive array of assessment arrangements to ensure that young people are treated fairly and that we take account of their personal circumstances as they undertake our awards.

This question is for Gill Stewart. You mentioned earlier some of the bigger lessons, such as those around remote learning and assessment, which we have talked about quite a bit. You also talked about e-enabled assessment and co-creation. If you have more information on e-enabled assessment and the remote learning stuff that you spoke about, will you expand on it? I appreciate that that work might be at the very early stages.

Our toolbox of assessment approaches should contain e-assessment. Even with the existing practical courses, we have started to explore how we can turn what are effectively short question papers into e-assessments. That is just one example of the sort of things that we could do.

It would be good if we could build an infrastructure that would enable the SQA to receive evidence from schools and colleges digitally so that we could carry out marking and quality assurance processes electronically. It would also be good if we could find digital means of submitting a lot of the course work for some national courses. For example, as part of their music course, people have to perform on a number of different instruments. In the future, could we do that remotely? Similarly, in art and design, people currently have to physically send a large portfolio of work to the SQA. Could we use technology to do that digitally in a much more streamlined way?

Submitting course work digitally would open up opportunities with regard to the sorts of things that could be assessed in the future. It might widen out the type of things that could be assessed. One of the things that comes through strongly from young people is that they would like there to be more skills development. Technology could be helpful in that sort of area and the assessment arena around that.

Those are some of the things that we could explore as a system. Some of them might be things that we could do in the shorter term, while others might require a medium to long-term run-in. However, that is definitely the way that I would like things to develop.

There is a big system piece in all of this. This is a broader conversation for the wider system in terms both of the investment that is required and of the need for all parts of the system to have the appropriate digital tools in place. That has been picked up by the Scottish Funding Council’s tertiary review, particularly in relation to colleges and universities, and there is also a programme for government commitment around a national digital academy, which focuses particularly on learning and teaching, but there are also opportunities around assessment there.

There are potentially some good developments in thinking on some of those issues, which could lead to the possibility of remote assessment and to at least some of the challenges that we have discussed today falling away. There is a broader issue there. Again, that might be something that Ken Muir might wish to pick up on in the review that he is undertaking, particularly with regard to how curriculum and assessment integrate and are delivered. Obviously, e-Sgoil has been an important development and has been used during the pandemic for broader purposes. There are opportunities there, but it is a systems piece.

We did not quite get everything right and we did not quite get the balance right in some areas. It is good to know that we are in a better place this year and that, looking to the future, there is a lot of positive stuff going on. Thank you all for your commitment to working on the changes as they come through in the next couple of years.

Indeed, as I said earlier, we are all united in respect of caring passionately about the future of our young people, because they represent the future of our country and, indeed, our planet. These are big issues. They are highly politicised—I do not think that anybody would deny that; this is the big stuff of politics.

I thank our witnesses for appearing today. We are grateful for the time that you have given us. We have tested your stamina and you have not been found wanting. We have kept you going for two and a quarter hours—that is quite a lengthy session of cross-examination. At this point, I will bring our meeting to an end.

At next week’s meeting, on Wednesday 6 October, we will take evidence from the Cabinet Secretary for Education and Skills on her priorities for session 6 of the Scottish Parliament.

11:45 Meeting continued in private until 12:42.