Education, Children and Young People Committee

Meeting date: Wednesday, September 22, 2021

Official Report 703KB pdf

---

Alternative Certification Model

Before we hear from the first of our panels, I would like to thank all the young people who took the time to speak with us informally last week about their experiences of the alternative certification model. We really appreciate their taking the time to let us know their thoughts on what it was like for them, and how they think assessment could best be managed in future. They really were a credit to themselves. I hope that they found it as helpful as we did. It certainly provided us with a great deal of insight into the issues from their perspective. An anonymised summary of their views is provided in the committee’s public papers this week.

Our first panel of witnesses on the alternative certification model in 2021 are from the Association of Directors of Education in Scotland. We have with us Tony McDaid, executive director of education resources with South Lanarkshire Council, and Audrey May, head of service, children and families, with Dundee City Council. I welcome you both to the meeting. Does either of you have any initial comments that you would like to make before we get into the questions?

Good morning. We are happy to go straight to questions.

Audrey, do you feel the same way?

Yes. Thank you for the opportunity to be here. It is lovely to hear you talking about young people first and foremost, because that is why we are here, too.

Collectively as a committee, we are determined that we want to hear from young people and that we want their voices to be heard in an unfiltered way. Last week, we had a really good experience with the young people whom we spoke to.

We will go straight into the questions. As usual, I will take the convener’s prerogative and ask questions about your experience over the last period. How well supported by the Scottish Government and the Scottish Qualifications Authority did you feel during the past 18 months?

There was a genuine attempt by people to come together and collaborate, certainly on the 2021 model. I was part of the national qualifications 2021 steering group, and there was a working group as well. There was a genuine effort to understand what was a pretty complex situation and a willingness to engage with local authorities, teacher professional associations, School Leaders Scotland and a variety of others, including young people, who were on the group, too. It was a genuine effort to collaborate.

Ultimately, we have a qualifications body that delivers national qualifications, so some individual decisions had to be made by that body. There was never going to be a perfect match; there was always going to be a scenario in which some of the things that were suggested were not possible from the qualifications body’s perspective, because it delivers the national qualifications, compared to what we were looking at for schools or children and young people via the local authority. It was about finding the balance in achieving appropriate national qualifications that had credibility for the young people while, at the same time, making sure that the process was manageable for us, for teachers and for young people on the ground.

There was a genuine willingness to collaborate and a genuine opportunity to do that through different engagement exercises involving the SQA and teachers or central officers. However, at the same time, we had to reconcile that with some of the practical things that were going on. Something might work in theory but, when you try to put it into practice, it can become a bit more challenging.

We have a number of questions about the communications dimension, which we really want to understand more about.

Audrey, do you want to come in?

I support what Tony McDaid said. There was no script for working through a pandemic. We were all finding our way and having to be adaptable, flexible and agile as things changed—as we all know, things changed a lot over the period. People rallied round and worked in collaboration—perhaps even more so than in the past—because we were all so concerned about the big impact on our communities and on our children and young people and their families. In particular, we were concerned for the young people. We were hearing from our young people and trying to listen to their voices throughout. We were hearing about their anxieties, worries and concerns. It was really important that we worked together to deliver the best job that we could for our young people in the toughest of times.

We will come on to communication, but we certainly had regular updates from the Scottish Government. Although things changed and that perhaps kicked us back into the practicalities of having to change the operational work, at least we were getting regular communication. Of course, we also got updates from the SQA, and the situation changed throughout. That brought stress to the system, but it was part of the job of people like Tony and me in local authorities to manage that and help our schools and establishments to deliver on what was coming next during the pandemic, which was new territory for all of us.

It must have been difficult to maintain any kind of meaningful communication with the young people during that time. Teachers were struggling as well. How did you cope?

We have a city-wide youth voice group that we regularly meet to discuss and debate the big issues for children and young people growing up in Dundee and Scotland. We have a lot of those regular communications. Obviously, we went online with Teams and Zoom calls to keep the communication open. We did a lot of electronic surveys and had forums in which we brought together young people in smaller groups to hear from them. They were trying to do that in their schools as well.

There has been a lot of criticism of the SQA. The evidence that we have received—including even today—shows lots of criticism regarding communications, decision making and the timing of decisions. Given the circumstances that the country found itself in during the past 18 months, are those criticisms fair?

I think that they are people’s response to how they felt at the time, when things were constantly changing. From the perspective of someone in the system, I am sure that it felt fair to talk about the frustration, worry and concerns that we had for our young people going through the system. The SQA was in the same boat as everyone else—it was trying to manage the situation through a pandemic. People had to do the best that they could with the information that they had at the time, and that was an ever-changing picture.

To be fair, the SQA kept in touch with us. It has co-ordinators for all our local authorities who keep in touch with our officers, so there was regular communication to and fro. People were frustrated because there was often change to what we thought was going to happen, but that was part of the world that we were in at the time. To be fair, we all need to take that into account.

We can all be wise after the event—that is the reality. Tony, do you have anything to say on that?

What Audrey May says is exactly the case.

We should remember the sequence of it. We moved forward with national 5 exams, which were changed in October. We then had a period in December in which we changed the highers and were still anticipating going back to school. However, in December, we found out that we were not going back to school in January. We then had the period from January to March, which I describe—I speak as a former secondary headteacher, like Audrey May—as the workhorse term in the senior phase, particularly for young people doing highers and advanced highers. That term is about doing assessments and on-going activity for young people, but it was then taken away. Ultimately, we were trying to find a balance.

There were some confusing forms of communication. Let us take communications on assessment as an example. Those communications were genuinely trying to say that we did not need to have a big hit at the end. However, that changed when we got into some of the subject guidance. By the nature of some of our qualifications, we leave lots of the activity and assessment to the end. That is not the case in every subject area but, with a good number of our subjects, the balance of the qualification will be sitting a two-hour exam at the end, in May. Inevitably, in trying to get a quality activity that gave an accurate assessment, we needed to be able to do the learning and teaching that comes with that. The advice was that we could do general assessment, but some of the subject-specific advice said that we needed to almost replicate what had been done previously for the assessment to be valid. That sometimes appeared confusing to class teachers, faculty heads or, indeed, school leaders who were trying to navigate the situation on the ground.

We will come back to that issue. I think that Audrey May was saying that some of the criticisms are a little bit after the event. It was a very difficult situation that people were managing their way through.

It was imperfect. I do not think that we could ever look back and say that it all went perfectly. We were genuinely in the most challenging scenario for young people and schools. Obviously, schools and young people were looking for clarity at that point. I felt that lots of the decisions on young people did not have the clarity that was needed—even on issues such as whether to study—that would have allowed them to be in control. The decisions were always changing, so it was legitimate for young people to say, “Wait a minute!”

Thinking back to that period, were you looking for more guidance than you got at the time?

Sometimes, it was about the nature of the guidance. I spoke to some young people—indeed, members of the Scottish Youth Parliament were on the national qualifications 2021 group—and, to them, the guidance felt very technical, as if the language was not necessarily always designed for them, although there was a learners panel. I think that young people needed consistency of guidance, although I am not sure that that was possible. We can apply a retrofit, but I do not think that it was always possible to do that in the scenario. Young people and teachers were asking for consistent guidance, but it was not always possible.

As Audrey May said, for those on the receiving end, due to the complexity, some of the messages might very well have looked mixed, because they were.

Audrey, do you want to comment on whether there should have been more guidance, or less?

I agree with Tony McDaid. There was a lack of clarity sometimes—that is how it felt to the people receiving it. There was guidance, but it was changing and it did not reach young people in the way that would happen in—dare I say it?—normal times, when we prepare young people for what is coming next.

It is a very different pathway in normal times.

Absolutely. We do all the work in our schools to help young people to prepare for exams. We provide study support and put in additional tuition, if that is needed. We give them feedback on the work to date and help them to plan their study timetable. We do all those practical things to help young people to prepare, but we were not able to do them, because the situation changed frequently.

I want to drill down a little further into some of the themes that the convener has raised. I am interested in the communications. Can you tell us a little more about the quality and quantity of the communications that you received, and about the timescales? Did you get the right information from the right people in the right timescales?

There is no doubt that there was regular communication. It is about timing of communication, when we had the information and how we got it. Although the communications were from the SQA, there was a genuine attempt through the national qualifications group to consider the kinds of communication involved and who they were directed to. There were communications to central officers in local authorities, to our school leaders, to teachers and to young people. There were also messages to the general public, to reassure people about the ACM and to say what the model might look like.

People sometimes feel overwhelmed by communication. If a communication says some of the same things that people have heard before, we need to consider whether it provides clarity and what its importance is. Has it changed from the previous communication or does it build on it?

09:45  

There was regular communication. You could always look back at the timing of it. As a teacher, you want communication as early as possible—you want to have information on course construction, the learning and teaching and what the assessment might look like. Individual schools were very good at talking to young people. The individual teacher in front of their class was trying to communicate the message to young people. We then had layers on top of that, which added complexity for young people when it was not the same message. What they were hearing started to make them a bit anxious—and understandably so. We certainly picked that up in April, May and June, when we came back into schools. That part was confusing and challenging for young people simply because of the scale of what was being asked of them in trying to navigate what they were doing and what counted as their assessment.

Normally, there is a consistent assessment diet at the end, but, in the alternative model, the assessment could be based on their teacher judgment and did not need to be at the end—it could be based on some activity, which is very different. Further, that varied not just from school to school but from subject to subject. It was about getting a consistent message to the young person to say, “Don’t worry. It’s about understanding standards for your teacher and being able to moderate so that we come to a collective view. Your grade will be from your teacher, but you will know that it has been checked and there is a consistency about it.”

Michael Marra has an interesting declaration of interests.

I declare an interest as a serving councillor on Dundee City Council, which is Audrey May’s employer. Also, Audrey May is a former teacher of mine. That is the more interesting point. She was a very young teacher.

We are going to talk quite a bit about qualifications, grades and outcomes, first and foremost. The other side of the outcome that I am interested in is knowledge and what young people learned.

I think that Tony McDaid made a point about missing the workhorse term and an awful lot of people missing an awful lot of time in school. As we look forward, what concerns do you have about what those young people might not know because of what they have missed, and what they have learned?

That is an excellent question. As a teacher, it is about the notions of knowledge, skills and ability. We need to separate the grading and qualifications and the knowledge that young people learn. There was a tremendous effort between January and March. On the whole, young people have shown incredible resilience, and they should be commended for their engagement with their schools.

I have spoken to young people in schools. Audrey May has described what has happened, as well. I am not glossing over the fact that things were really difficult for them, but people were able to get involved. This time round, the engagement from teachers in learning and teaching has been much better than it was the first time, when they were just reacting. There has been engagement with class teachers via Google Classroom and the national offer through e-Sgoil, which has involved live lessons activity. As part of the west partnership regional improvement collaborative, there have also been the west online schools pre-recorded lessons.

Young people have shown us that we can learn in a different way. I am not as worried about the knowledge gap, because, from a qualifications perspective, adaptations were made. There were practical adaptations around some of the content.

The question will come as we roll forward into this year’s qualifications. We still need to ensure that those adaptations are there. We should think about the experience of a fifth-year student at the moment. Fifth-year students have had two periods of lockdown. They have come through a period in which there was an alternative certification model, and they are on their highers courses this year. We all have a responsibility. All the young people count, but I have an eye on those ones.

We need to ensure that the knowledge or sense of understanding is built on and also that the alignment of the qualifications meets where people are at just now. My worry is that the qualifications system has a full expectation that nothing has been missing, and the leap for them will be considerable. We need to ensure that the approach continues to be tailored to their needs, if that makes sense.

That makes sense to me. However, it worries me a little when I hear from young people who are going from highers to advanced highers, for instance, and when I speak to university principals and lecturers about people coming out of school with perhaps a lack of knowledge compared to what they might have had otherwise. The question whether we, as a country, are adapting to address that worries me a little.

I do not want to burrow too deeply into that, because I have another question.

We are aware of a gap, but adjustments have been made to address it. That does not take away your worry about the future, though, and it is important for the immediate future that we address that issue.

It is really important that, when our young people returned to school, we did not use the language of catching up and put extra pressure on them. We were very much concerned about their welfare—we wanted to care for them because of the impact of their experience through a pandemic and the worry about qualifications. We very much took a health and wellbeing and welfare approach to the return to school. We have therefore not focused on people having a huge gap in their knowledge. We need to say that out loud and admit that that is the case, and we need to think about what that means as we go forward, especially for the young people Tony McDaid and Michael Marra have mentioned. It is really important for us to think about that.

I reiterate that support existed. People really rallied round in the online and digital offer. In Tayside, we worked with the west partnership to grow that across the country, not just in our own regional improvement collaborative. We worked nationally, and we built on the great work of e-Sgoil in the first place. A lot was put in there.

It is important to do what Tony McDaid said—to recognise that some young people engaged very well online and that some people hit the ground running when they returned to school. Not all of them did, but some did. The medium was really successful for them. It suited their style of learning, and they totally engaged. We need to take forward that part of the learning, as well, for what we do in the future with any disrupted learning.

Data that were released yesterday show the scale of the attainment gap produced by the ACM—or the alternative certification model; I am trying to avoid acronyms as best as I can. They show that around 75 per cent of private school pupils got A grades compared with fewer than half in the state sector. Do you think that the system benefited the most affluent at the expense of the poorest?

No, not necessarily. The poverty-related attainment gap existed pre-pandemic. Trying to ensure that we close that gap and have as much aspiration among all our children and young people is a mission for all of us every single day. From our perspective—I am speaking for my local authority—we have seen more As than we have ever had before. Nonetheless, I think that we are seeing a scenario around the poverty-related attainment gap and disadvantage. I do not think that the ACM accentuated that. What we are looking at is our kinds of assessments for young people.

I support that. We are very aware that feedback from young people and our schools shows that young people who are often associated with disadvantage, deprivation or poverty—however we wish to describe it—sometimes benefited from not sitting exams, because they did not have the support in their wider community to be able to turn up to exams ready and prepared. For some of those children, who would probably be in that gap, the system suited them better. Quite a number of our colleagues have reported that to us.

Audrey May talked about some people going back to school and hitting the ground running, as if nothing had interrupted their stride. However, did the gap not widen for those who did not have the same connections? I am asking that because I find it quite hard to believe that it did not. If a person was not able to take advantage because of a home environment or a material basis, did they not fall further behind? I apologise; I am using a pejorative term. Did the gap not widen?

I think that it definitely did for some young people. If we look at the data, we can see that the gap widened, but not for all the children and young people. It was about drilling down to what suits some better than others and ensuring that they had access to digital services, for example. That is part of what we tried to do. We should remember that, during lockdown, our schools brought children and young people into our hubs. We were concerned about them not being seen and being vulnerable for a number of reasons. A lot of those young people had one-to-one teaching or small group teaching in the hubs. It was a mixed picture.

I have no doubt that, despite your best efforts, some young people were left behind.

Yes.

I am a bit confused about the difference between 2020 and 2021. My understanding is that, in 2020, the grades of state school pupils improved faster than those of pupils at private schools, and then, in 2021, the opposite seems to have happened and we seem to have seen a reverse of the progress in narrowing the gap that we saw the year before. Do you have an explanation for that? It is fine to talk about A grades, but, for a lot of young people who are looking to get qualifications and leave school with something meaningful, it just seems a bit odd. I am trying to understand what changed between 2020 and 2021.

That is a good question. We should also remind ourselves that the system was not the same—there was a different model, as schools were giving the grade on the base of an internal estimate. The grades in 2021 were broadly aligned, if I think of the number of pupils gaining one, three and five highers. Of course, I take your point about whether we are talking about As or simple passes. If you compare that to 2019, you can see that there was a considerable improvement. If you take this year’s grades—

They are worse than the grades in 2020.

Yes, I accept that.

So, over the year, we have had more time to get the ACM organised and, in that time, people from the most challenging communities have been disadvantaged. Is that the case, or has the system just adjusted back to what we would normally see?

I do not think it has. I think that the system has worked with individual young people as the model has been put in place, and there has been an attempt to quality assure the model, too. The internal estimate system in the ACM was slightly different from what went before, and there was different moderation of that activity. There is also an issue around the technical part of the transition from fourth year into fifth year. If young people were doing well in fourth year through an internal estimate process, the jump into fifth year might have been considerable for them, so that might have been a contributory factor. Those young people might have got an A or a B, but, as they went into fifth year with the alternative certification model, which allows for demonstrated evidence, they faced a jump that might have been difficult for some young people.

10:00  

I am interested in Mr Mundell’s line of questioning—

Sorry, Bob, but I think that there is something wrong with your sound. It sounds as though you are very far away.

I am told that it is all right on the broadcast and that everyone in the room can hear you well enough. Please carry on.

I am interested in Mr Mundell’s line of questioning, because it gets to the heart of some of what we are talking about. We need clarity about what happened rather than snapshots of attainment. My understanding is that, in historically challenging circumstances, the attainment gap technically closed in both 2020 and 2021, compared to 2019 and previous years. Is that the understanding of our witnesses?

There is another important point. Does some of the difference between 2020 and 2021 have to do with the role of internal moderation procedures, whether those are departmental, whole school, local authority or across local authorities? I would like to better understand the moderation procedures that existed in local authorities in 2021 compared to those used in 2020.

One of the issues, of course, is that, without an exam—as unsuitable as those are for many young people—you do not have that benchmark. In 2020, we did not have that benchmark to refer to.

I am interested to know about 2020 and 2021 compared to 2019 and previous years, and I am also interested in the role of moderation in schools at a local authority level in 2021 compared to 2020, because that might flush out some of the issues.

Of course, the lockdown from January to March clearly had a massive impact. It might not be ACM that led to that differential; it might have been that January to March lockdown. I am interested to know the witnesses’ views on that, too.

When the lockdown started in March 2020, there needed to be a gallop towards helping our young people to get the best qualifications they could, so we had to use a completely different system. As has been said, we submitted estimates at that point and then the SQA awarded the awards. That was the fundamental difference.

In the ACM, we had a bit more notice—it was staggered notice, from that October to December, then into a lockdown in January—so we had more time in that term, when we would normally be really going for it in order to give every young person the best opportunity to get their internal work done and prepare for the exams.

We had to set up a system of moderation and quality assurance at every level. Part of the work of the local authority officers who were working with the SQA was around understanding standards and the offers that were out there for supporting our teachers in the system. We also had to look at our quality assurance so that we were really confident that we were submitting provisional awards—that was the difference; in 2021, we were dealing with provisional awards—that were subject to robust and reliable moderation across every level. We were moderating at a departmental level, within subjects and curriculum areas; we were moderating across the school; and we were moderating across the curriculum improvement networks, where principal teachers of the same subject or curriculum leaders were coming together. We also used our SQA colleagues to support us with the work around understanding standards. We then took that to the regional improvement collaborators level.

For subjects in which there were just one or two members of staff in a department, we would moderate across three local authorities, and we then partnered with other RICs in order to take a wider approach. We even got to the point of getting that online as part of the digital offer for our colleagues and built up hubs of resources. In fact, in our RIC, we even started to look at nationals 1, 2 and 3, which is now a national development that we are leading on, to make sure that all our young people were given accreditation for the qualifications that they deserved.

I am confident that we did everything that we could in that time to make sure that our young people were getting the results that they deserved, because people were working really hard. The commitment of teachers at that point to do the best job that they could for the young people was second to none. There is learning there, and we want to take forward that learning in relation to how we better benchmark our standards and how we better moderate and quality assure.

Along with local authority colleagues, we have to think about that. There might be things that we need to stop doing in order to do more of that in the future, because we felt very confident about our presentation of the provisional awards. Out of thousands of children—around 3,000 in Dundee—we had 59 appeals. You might want to talk about appeals later.

That process also allowed us to deliver a no-surprises agenda. Young people were being tracked and monitored through continuous assessment, so they knew what their award was likely to be, right to the wire, and our schools all agreed to release that information on the same day, so that we were all on the same page.

Perhaps that was more than you needed to hear.

No, that was excellent.

It is important to say that the process has been about having confidence in teachers’ judgments. It is based around what young people have done in class, and, ultimately, a young person wants to be assured that they have been treated as fairly as possible. That is why demonstrated evidence based on teachers’ professional judgment is important. Teachers’ professional judgment is based on something, and we need to ensure that what it is based on is consistent and that the young person is able to see that, even if the assessment tool might be different and the evidence might be collected slightly differently.

Last year, in some of our urban schools in South Lanarkshire, there were young people who were self-isolating two or three times, but that was not the case in some of our more rural schools. We could not possibly have had the same assessment tool for both kinds of school—that would have been completely unfair to those young people. We needed to have a balance there, so the moderation became vital. That involved an understanding that the teacher who was awarding the grade had been through the understanding standards process and was confident about what they were doing.

The process also gives the teacher a bit of protection in relation to the grade that they are awarding. Although it is the teacher who is awarding the pupil the provisional grade on the basis of on their work, they have been involved in checking that work across the department, across the school and with other schools within the local authority and beyond it. The notion of demonstrated attainment that is based on the professional judgment of teachers is quite important.

There is lot of confidence in the robust processes that underpinned the alternative certification model for 2021. There will always be room to improve that model, but it is consistent across the country.

There was, however, a contradiction in some of the SQA guidance in relation to there being a reliance on exam-style evidence but there also being an encouragement for schools and departments not to have traditional exit-style exams. We heard from young people that, in some schools, they got a large exam and, if they did not meet the standard, they got a second exam and then a third exam—so they got multiple opportunities to prove that they had reached the standard. Unfortunately, the process was top heavy in the last few weeks. What guidance did your local authorities give out about what best practice looks like in relation to unavoidable assessments in those last few weeks, and is there need for greater consistency? There seems to be a bit of a patchwork experience across and within schools and local authorities.

I think that there was an element of that, but, putting that into context, our guidance was that the assessments should be based around the learning and teaching that had happened. If you think about it, teachers might need to take a diagnostic approach when young people come back into school. Although they will have been engaging with young people, having the class in front of them has an important function at the start.

I also think that schools were caught in a dilemma with individual teachers as well. By the nature of our qualifications system, some of that drawing together of concepts and knowledge comes at the end. We need young people to practise, to develop concepts and to gain an understanding. If you assess them too early, you might do the young person a disservice, because they grow in confidence and gain an understanding across the year.

I also think that teachers and schools were trying to give young people certainty around when they might get that assessment. The one thing that young people would have wanted was a notion of what they needed to do to get an A and when they were going to be assessed. We were trying to leave that as long as we could, but there was a nervousness around the possibility that a young person could disappear for 10 days, because that was happening all the time. The practical effect of that was that teachers were saying, “We need to put something in the diary. Don’t worry if you are out of school for 10 days, because we can catch you up.” That happened regularly—I was involved in a number of conversations about that happening. It comes down to the brass tacks of how you manage that situation and the practicalities of it—which, some young people would present to you, might mean 15 assessments at the end.

It is worth pointing out that schools managed young people’s anxiety, on the whole, and tried to take that into consideration in a practical way. If a young person said, “I am struggling a wee bit,” the school could say that they should not worry, because they could do that bit of work the next week. I have numerous examples of that happening.

I was heartened by both witnesses’ statements at the outset about the genuine co-operation between all players involved and their positive comments today. That gels with what we heard from young people from Inverness in the session last week who gave positive reports about their experiences in lockdown despite all the difficulties, some of which we have heard about this morning.

Looking to the use of alternative certification alongside examinations in the future, how do you believe we can provide assurance to colleges, universities and employers that, in a system of assessment that does not involve objective examination and in which the testing is done by independent third parties, the qualifications have been earned? How can we avoid any criticism that it is unfair to expect teachers to do anything other than have an optimistic and favourable response to the children who, after all, they have taught, and that, in a sense, they are marking their own jotters? That is not a criticism of the great work that teachers do, but it is a fundamental question. I feel that we have been skirting around it a wee bit because, quite rightly, we have been looking at the difficulties of Covid.

When I was at school—although, as it was such a long time ago, it is probably no longer relevant—examinations were the be-all and end-all. There was no other assessment of any sort. I think that that is the wrong approach, but, if we are to move away from examinations, I would like to hear from our experts how we can demonstrate the bona fides and the robustness of an assessment procedure. How can we get it right for Scotland?

That it is a big question. There is a request in there for trust in the system and trust in our colleagues and in our schools. I would absolutely support that ask, because the teachers and leaders I see in my schools every day just want to do the very best for their young people—that is why they came into the profession.

We can we have a big debate around this. I think that that is part of what we need to take forward from the Organisation for Economic Co-operation and Development review and the recommendations in it around shifting the balance. There might still be an exam—we cannot say, because we have not had that wider discussion yet—but perhaps we can shift the balance towards more continuous assessment so that young people get the opportunity to see their progress throughout their learning, not just at the end, which is the case in what might be described as a high-stakes exam system, which, as the OECD says, does not suit all our learners.

We have a really good fundamental starting place. Curriculum for excellence is great and is renowned throughout the world, but perhaps it does not articulate well with the qualifications part in the senior phase.

There is work to be done to look at all of that together, and, when we start that dialogue, the most important thing—it was mentioned right at the start of this session, but we have not come back to it—is communication. We really need to communicate with our stakeholders—that is, our young people. They need to be part of the story and the debate, as do their parents and carers. We need to give them, as well as further and higher education institutions and employers, confidence in the system. We also need to look at what we have learned, which I tried to touch on—I know that our international council of education advisors touched on how fabulously we did during the pandemic. We need to take that learning around moderation and quality assurance and make it open and transparent so that there is confidence in the system.

10:15  

The question strikes at the heart of what we want for our national qualifications system. I do not think anyone is saying that exams do not have a part to play. We need to clarify what important assessment is.

If you look internationally, you will see that other countries do this differently, as Professor Stobart shows in the second part of the review. I heard Professor Beatriz Pont talk about that last week—I chaired the practitioners forum, so I spent a lot of time with the OECD on this issue. Doing things differently does not mean that there are no exams or class tests. The baccalauréat in France involves an oral assessment, for example, and people can do a written report as part of their assessment. You need to strike a balance.

The fundamental question is, what do we want out of our qualifications system? I think that that is a bigger part of the debate. I do not think you can just tinker around with the assessment tools; you need to look at the course and the skills that we are trying to give young people. We need to think about the fundamental form of our ambition for children and young people.

There are also a couple of unintended consequences that can come out of the notion of continuous assessment. With internal assessment, it can sometimes seem like you spend more time doing the assessment than the learning and teaching. The process has to be proportionate; it has to assess the things that matter, and you have to make sure that it is not overbureaucratic and that the teacher is not spending time filling in bits of paper rather than doing the learning and teaching part. My background is in physical education, so I am used to the practical context of learning, but the reality is that we need to strike a balance that is meaningful. Then there is the debate that we need to have with our families and our communities. We need to tell them that they should not worry, because we will put in place a moderation system that will ensure that those skills are robust.

I feel strongly that the young people have earned their grades this year and that they stand up against the grades in any other year. We need to continue to give that message, but we need to tailor our assessment to the things that we are looking for and the ambitions that we have for children and young people.

It is interesting that the OECD called on us to restate and re-evaluate our vision of curriculum for excellence, which, in essence, is what you have just both said. Ross Greer has a supplementary question on this.

My supplementary was on Bob Doris’s line of questions, and the conversation has moved on a touch, so I am happy to bring it into my line of questioning.

In that case, I will pass back to Fergus Ewing.

I will pass to other colleagues, as I appreciate that time is short. I thank both witnesses for their full answers.

I was interested in the Scottish Secondary Teachers Association’s survey, and I want to ask a question that is related to what Fergus Ewing was asking about. We were talking about demonstrated attainment. According to the survey, 36 per cent of teachers believed that the evidence that they had collected truly demonstrated their pupil’s attainment, but that means that 64 per cent did not. What is your reaction to that data? Should we be questioning it?

I think that you have to listen to what data tells you. Again, that was at a fixed point in time. You need to get to the end of a process in order to fully understand it. At that point in time, we were right in the mix of the assessment processes. There was a lot of talk about what was high predictive value versus low predictive value—that is, if you mirror some of the assessment activity that the exams would do, that has a higher predictive value than a class test that was done in October.

You need to be confident that teachers have a range of assessment tools at their disposal. The danger is that you assume that teachers’ professional knowledge is not based on anything. It is important that teachers’ professional knowledge is based on the professional experience of the assessment tool and the balance of that assessment, looking holistically at the young person. Therefore they were not giving a grade exclusively on the basis of the sitting of two or three exams in May but were able to take into account some of the work that was done in class in August, November and December and use it as a contributory factor at the grade boundary. When I have a young person sitting on an A or a B, I can look at something else and say, “This person is on the boundary based on the couple of assessments that I have, but I now have some evidence that they are at an A.” That is where teachers’ professional judgment and understanding of standards is crucial at that stage.

Thanks for the evidence so far. I want to focus on the stress for pupils and teachers. It is often claimed by pupils that an assessment model is less stressful than a big-hit exam, but we also heard from pupils that the model this time meant repeated assessments, which for some were just as stressful, especially for those who did not get the results in the first assessments that they did and had to do them over and over again. I wonder whether the claim that assessments are generally less stressful than big-hit exams is true, or was what happened peculiar to this year?

I will start, but I am sure that Tony McDaid will want to come in. Thank you for the question—again, it is something that we are all grappling with, because we want to look after our young people and care for them as well as get them qualifications. It is important that we understand their stress levels and their anxiety. If I was to give a very short answer, it would be that it is because of this year.

There is wider research out there that we want to look at. We have already referred to some international research on the balance between internal assessment and a one-stop exam at the end of a course. We need to learn from that and dig deeper into that research. However, this year, there was so much uncertainty and so many changes along the way; I think that Tony McDaid has partly covered some of that in an earlier response about young people isolating for 10 days. In one of our schools in Dundee, we had young people isolating three or four times during their fifth or sixth year of school, so there were huge pressures. So, yes, there were repeated assessments, but that was probably because of the other part of your question—teachers were stressed and were worried that they were not doing enough to get the young people every bit of evidence.

There was much more emphasis on the professional award being based on evidence of assessment. The overarching message was that we needed to get good evidence to be confident that we were giving the right award. This year, we have been through a pandemic, so nothing was as it should or could be in terms of bringing in a new system. It was an emergency set-up that we put in place to do the best for our children and young people, and, of course, there was lots of anxiety around it because it kept changing, as we heard earlier.

Just as a follow-up to that, there was criticism from the EIS but also from some—[Inaudible.] The criticism from the EIS, in particular, was that the SQA was saying that these were assessments but it was reverting to a more traditional model. What is your view on that?

As a final follow-up, there have been some claims that this year and the previous year can teach us a lot about what we are going to do with assessments and exams in future years—is that true or not? You have indicated already that any model in the future would not be like this, so I wonder what we can learn for the future from the past two years.

Did you catch the whole question?

I missed a bit right at the start.

There was a broadband blip at the start of the question. Could you repeat the first 15 seconds or so?

The EIS and pupils have criticised the fact that the assessments were, in effect, exams by another name. The EIS, in particular, was critical of the SQA’s advice in that it was, in effect, reverting to a model of exams but was calling them assessments. What is your view on that?

There was some mismatched activity in that were was very technical advice for some subject areas versus the general advice to look at it as a broad assessment piece. Ultimately, you can understand a qualifications body talking about the high predictive value of an assessment and getting something that looks like a small exam at that point, rather than something that looks at the more broadly based activity.

In terms of what this means for our qualifications going forward—the point about balance was repeated in the OECD report—we need to be careful to avoid unintended consequences for young people if we go into a continuous assessment model. We need to be careful that we do not overburden young people so that every single assessment feels like they are doing an exam in May. We need to find a balance for young people over the course of the year.

Perspective becomes important, and managing young people’s wellbeing and staff’s workload becomes central. Too much assessment activity can take away from the quality of the dialogue and the learning and teaching. Instead of running a dual system, I think that we need to run an integrated system. Instead of trying to achieve an external exam for the bulk of it and, at the same time, having evidence that sits separately from that, we need to ensure that it aligns completely. If it aligns completely, we can take some of that assessment evidence out and focus only on the things that really matter and that allow us to give a grade.

I would say that there were some confusing aspects around some of that activity, but I understand why. You can understand why that advice was given out in the middle of something, but, when that translated into reality, that could have caused some confusion. However, we need to be cautious about what that tells us about how we move forward and what any new qualifications model will look like.

Audrey, do you want to comment?

No, I think that Tony McDaid has covered it. That is where we are.

We are going to have evidence later from the NASUWT—I have to get these acronyms right. It talks a lot about the “compression” that you have been describing in terms of assessment, how difficult it was—“impossible”, it says—for teachers to complete all the work that they were asked to do, including the quality assurance and certification, and about the effect that that had on the teachers. I am sure that you are conscious of that.

I am completely conscious of that. We need to make sure that teachers’ workload is manageable, because that is integral to the experience that young people have and, ultimately, the qualifications that they achieve—and it is about schools, not just teachers; we have school support assistants who were heavily involved in this type of activity, and it is also about school leaders. We have shown that our teachers can do pretty much anything, but it is asking them to do everything all of the time that is too much. We need to be clear about what we are trying to do. In these extraordinary circumstances—the changes to national 5 in October and higher in December and the lockdown between January and March—no matter how it was managed in April, May and June, when teachers were searching for as manageable a way as possible, inevitably there was a workload associated with that.

With regard to your comment about the teachers’ experience being reflected in the experience of the young people, one of the schools that we spoke to last week highlighted the need for fun to be put back into learning. I think that that is an expression of what you have just been talking about.

The move from teacher judgment to demonstrated attainment in the second year obviously created many of the pressures that you are talking about in terms of the assessment model and having to go through that. As part of the process, it removed taking into account the circumstances that many of the young people you describe were facing. They had to get the exams done, as there had to be demonstrated attainment rather than judgment. Does the absence of exceptional circumstances and appeals not go against what you are saying about the lived experience of young people?

I absolutely agree with that, and it has to be part of a discussion going forward. In the previous model, before Covid, exceptional circumstances were about what happened on the day of the exam, but, of course, we have had huge disruption for some people’s learning throughout, which has led to some of the other stresses in the system that we have been discussing. When we look to the future, that is an area that we really need to get right.

But there is a cohort of kids now and from the previous year who feel that the system has not served them well and that those exceptional circumstances have not been taken into account.

10:30  

Could I come back on a couple of things? We spoke about inferred attainment in 2020. Again, we need to look at the timing. Inferred attainment happened after 20 March, in lockdown. Teachers’ professional judgment was based around some demonstrated evidence up to then. The demonstrated evidence that they had in the first lockdown was prelims activity as normal, so there would have been a teacher judgment on prelims, and some schools had second prelims. There would have been internal assessment evidence that was not taken away, such as the practical elements in drama, music and physical education, which were already there. It was the last part that teachers were inferring—the drawing together and asking whether, if someone was on a grade boundary, there was enough evidence to go back and look at how they might have developed and to see what kind of test there was.

So, this year, we said that we needed to look at the demonstrated evidence. There was a second part to this year, and you can see that from the EIS’s presentation. There was an opportunity for candidates to present any evidence now, during this term, if they had missed any assessment evidence. We said to our schools, “You need to be relaxed. It is not about the quantity of evidence that you have. You need to be confident and speak to those young people about not feeling that they need to cover every single aspect of coursework.”

I accept that that might not have come as the reality for some young people, but I would say that, in terms of the evidence of our schools, the vast majority of young people were given opportunities to be assessed and to come back and do it again, and a further opportunity was put into the system to try to make sure. I do not think that many candidates took up that opportunity, but it was there, and I think that it was well argued both by School Leaders Scotland and the EIS, as part of the national qualifications group, that that opportunity was there for those candidates into this new academic session.

That was to do with additional evidence and not exceptional circumstances.

I take that point.

I want to return to the earlier line of questioning. Her Majesty’s Inspectorate of Education conducted a review of what local authorities were doing in terms of the ACM, and it found that most local authorities had developed bespoke data analysis tools to support school-level quality assurance, which were used to check against three and five-year data trends. That information was then used to identify and address any unexpected provisional grades. Is that your understanding of what happened across the country?

It was a sense-checking exercise. Just to be clear, it was based on candidates’ work—it was what the teachers thought and what those candidates demonstrated, which, for us, is a quality-assurance system. Professor Mark Priestley put that in his review of last year. Although that system was not ideally followed, Professor Priestley makes reference to sense-checking activity.

I will give you an example of the use of that data. In a class or a school environment in which candidates are doing particularly well on a trend basis and the level suddenly dips, I think it is legitimate that we speak to the school leaders and ask whether there is something specific about that group.

It was about trying to understand the picture and have confidence in the moderation process. It was not, under any circumstances, about trying to take an algorithmic approach and put a ceiling on candidates’ number of As; it was about making sure that candidates and teachers were reassured about the process.

For me, there was a sense-checking activity. What was good about it was the fact that our school leaders came to those activities and meetings and were completely clear that there was a robustness about understanding standards. When there was a slight difference, they were able to talk about it—it was the class teacher who had come up with that grade, there was a professional discussion and dialogue about that activity, and there was confidence in the grade that was given.

The reason I ask is that I am trying to identify what changed between 2020 and 2021—was it the heavy moderation process or the reintroduction of that? I note that ADES was in discussion with the SQA as early as October 2020 regarding statistical analysis, quality assurance and moderation. There is a feeling that the normal SQA processes, rather than taking place at the SQA end, were front loaded in that process. Is that a fair assessment?

That is fair. We had been—as, I am sure, lots of local authorities were, because we share a lot of our work—looking for the best way to track and monitor progress across the piece, and it was part of our quality assurance as a central team to open up a dialogue with our headteachers and provide them with the toolkit. Certainly, the toolkit that we used in Dundee was commended because we looked at five years of data on evidence and also outcomes versus estimates.

It was a sense-check of whether there were departments or areas in which the patterns did not correlate, and we were able to open up discussion when we saw discrepancies. We would then go back to the school so that the headteacher would have the opportunity to use that data to open up a dialogue and dig a bit deeper. That was the point of developing those systems, as well as to give a wee bit of confidence.

The tracking and monitoring that we want to happen in our schools is about individual learners and where their progress is in their learning. However, the process gives us an overview as a local authority, and it gives us an opportunity to have a conversation and ask the right questions if there are discrepancies. That is how it was used.

I agree with your earlier comment that trust and transparency are important. I am not asking you to comment on this, but I personally feel that there was a lack of transparency in the run-up to this year’s grades being awarded, both from the pupils’ point of view and from the public point of view. The cabinet secretary said something quite different in Parliament to what was said on the news on 8 June: the assessment process was being carried out by teachers and they would submit the grades—no one was coming in to overrule them, to second guess them or to look at any other material; the teachers would decide the grades. People then heard about what the normal moderation process is. I am not trying to suggest that that is not what would have happened in a normal year, but I think there was a suggestion that the ACM was somehow different from what happened at the SQA—although, in reality, it was very similar to what would normally happen.

I take the point around the moderation activity, but I reassure you that the grade was given by the teacher.

And no one was going to second guess it? It sounds to me as though the process was trying to arrive at the grades you would have expected.

No, not at all. It was about making sure that there was consistent professional judgment. From a moderation perspective, teachers will come at grading with a variety of experience. Some will be well versed in the application of the marking scheme for highers; some will not. I think that it is right and proper that the local authority, on behalf of the young people, is able to say, “I’m not sure that makes sense to us.” What then happens is a legitimate conversation that might involve a more experienced marker from another school.

Ultimately, it is not about us changing any grades at all; it is about asking, “Is that accurate? Does that sit with understanding standards? Does that show a degree of consistency?” Teachers award the grades, but we need to make sure that understanding standards ensures consistency between one school and another. It is not about applying a threshold or a benchmark; it is just about making sure that those standards are achieved and that we are confident about that. It is about fairness for young people and fairness for the teachers as well.

I understand the logic of that; I just do not think that that was the message that people were getting at a political level or the explanation that we got of how much work had actually gone in. To me, it seemed perfectly logical that ADES was meeting the SQA in October to discuss quality assurance, but the message that was being delivered here politically was a suggestion that the normal SQA processes were somehow not happening and that classroom teachers would be making the decisions by themselves, although I do not think that that is actually what was happening.

I will not comment on the political part of that statement, but the discussions that would have been part of the national qualifications 2021 activity, as opposed to a separate conversation between ADES and Education Scotland, were simply about the formation of an alternative certification model rather than about anything specific. All those conversations were about trying to develop the alternative certification model.

On 22 October, ADES met the SQA separately as part of a joint CAQ network meeting, and the minutes of that meeting show that there appears to have been quite a lengthy discussion about the need for quality assurance and statistical analysis, as well as about the appeals process. Those were separate conversations.

I take that point. What I am trying to say is that that then fed into the alternative certification model. I do not know who was at that particular meeting—it might have been Stephen Quinn, the director from Renfrewshire who chairs that group and is also in the NQ21 group. The purpose of that meeting was to feed into the NQ21 group.

In the spirit of sharing—and we do share across our ADES network—I can talk only about my own experience, but one of the things that brought confidence to our system was that we shared the timeline of all that quality assurance with our parent council chairs. Throughout the process, we regularly met not only our young people but their parents and carers. We met the chairs of our parent councils throughout and we had additional meetings with the chairs of our secondary school boards. We shared with them the timeline that we had for our processes of quality assurance and moderation so that there was local transparency, and that brought confidence in the system as well. Maybe all of us should learn to do more of that.

Would you share that information with us, as an example?

Absolutely, yes.

I will be very brief. I am concerned that we were confused about the purpose of moderation—Mr Mundell mentioned what the former cabinet secretary said about second guessing. It would help to hear from the witnesses that moderation is about professional support and assistance for teachers, that it is about checks and balances that teachers also want to see within the system, and that it has been that way in relation to continuous assessment and processes within departments. Some departments have only one teacher whereas some have five or six teachers, and that may change the balance in terms of the support that is needed.

Do you have a question?

Do you recognise that moderation is not about second guessing but rather about providing professional support and assistance to assure the professional in the classroom?

I agree with that.

Absolutely.

I think that our teachers, our pupils and their parents have all been absolutely incredible throughout this situation, so I extend a huge “Thank you” to all of you.

We have already spoken a little bit about supporting people’s mental health. I am quite interested in what worked, what did not work and what recommendations there are going forward. Pupils really did step up and demonstrate their resilience. At the evidence sessions, it was clear that they had a huge amount of empathy and respect for their teachers—it felt as though teachers and pupils were a team, which I thought was really nice.

My second question is about how your organisations will make sure that young people’s views are listened to and acted on going forward. In the evidence that we got, there were some suggestions that were really quite different, such as putting things up on Google Classroom so that young people could comment anonymously throughout. Some young people with additional needs who struggle with exams said that, if they had some music on in the background, that could make a really big difference to them. How we can incorporate those views?

Right at the beginning of this session, I touched on the fact that, across the local authorities, we all try very hard to value young people’s voices in all the big decisions that we make as local authorities and education authorities. I could talk about my own experience in Dundee, where we have a youth voice forum and the young people—maybe about four to six from every secondary school—come together to meet me regularly, and I hear from them and share with them. Through that forum, I learned that—as you can imagine—right at the top of the list of things they wanted to talk about was the SQA and their qualifications. So, there is lots of discussion there.

There is something about having that on-going, regular communication and having a timeline, as I have described, for trying to make sense of what is coming nationally, which helps young people to see a pattern and what they should expect, while also listening carefully to what they are saying about their concerns and asking them to be ambassadors and to go back to their own schools and ask questions.

10:45  

The young people were running surveys, holding pupil focus groups and having online discussions, but there has to be a feedback loop—they need a forum in which to feed that back. In their own local schools, pupils are getting to speak to their senior leadership teams about good practice in their schools—we have many young people who sit in on leadership team meetings and so on. The youth voice is being heard across the piece. It might look different in different authorities, but there is something like a forum where young people can share their views and feed in. We need to be able to feed back as well, and we need to feed into the system what has been raised and what their concerns and worries are.

The other thing that I touched on at the beginning of this session is that we needed to keep getting the message from our leadership team to our headteachers and their teams that health and wellbeing were right at the centre of what they needed to be doing, before they got to thinking about everything else that had to happen in learning. We needed to make sure that, when our young people returned, especially after lockdowns, and even when we were engaging online or bringing them into our hubs, we were addressing the issues and concerns that our most vulnerable young people had.

We have also put mentoring programmes and counselling into our schools for individuals who need a bit more targeted help. We have targeted mentoring programmes for our care-experienced young people, our young carers and other young people who have additional challenges in their lives that might be a barrier to their maximising their own potential. We started a counselling programme in schools in Dundee a few years ago, as part of our attainment challenge work, to address the mental health and wellbeing of our young people. As I said, we have listened across the piece to the feedback that the young people are giving us, and we are telling them what we can offer and what we can do.

No one would want to repeat any of the past couple of years, but there is learning that we need to not lose. We stuck to a no-surprises agenda right through the process so that, when young people got their professional award, they knew exactly what that was going to be. Yes, there was an appeals process, but there was a very small number of appeals across the country compared to the number of presentations. We need to think about what that means for how we engage with our learners about their progress all the time and be really clear about that learning conversation, so they know what they are on track for and what else they can do, or get help and support with, if they want to raise the bar.

Those are the kind of things that we want to do. In our local authority—I am sure that my colleagues would say the same—we spent a lot more time listening to our young people and discussing things with them. That is our message as well. How frequently we met the chairs of our parent councils also increased significantly. We have had nothing but positive feedback and support for the local authority because we have kept people informed throughout the process, which I think is key in terms of communication. We are very aware of the stress in the system, and we want to take forward that learning into whatever happens next.

I will add a couple of points to build on that. Teachers have a particularly special relationship with young people. We often think about the system, but schools are built on the positive relationship between the young person and the teacher who has been with them since they taught them in their first year and who has seen them develop and grow through the course of their schooling. Ultimately that attachment-informed activity is fundamental to the wellbeing of all our children and young people, as is having somebody to turn to.

As Audrey May said, there are structures in place around young people who are struggling. Also, from a practical point of view, communication with young people must be about clarity for them. They must feel that they are in control of some of the decisions—that they are able to make some of the decisions and be part of the decision-making process—and must be able to see the change that their influence has made. The information that is given to them also needs to be as consistent as possible. I was struck by the fact that, in the evidence that was given by young people, which is in the meeting papers, they spoke warmly about the communication that had been developed about the appeals process. They need to be involved in developing communications for young people, because that will result in their being appropriately tailored to meet their needs. How they deal with their peers is more sophisticated than our telling those young people.

This is quite a big question: what are the key lessons, and what priorities would you like the Scottish Government to look at for this year specifically?

As far as this year is concerned, the model is out in the sense that there is adaptation. It is an assessment activity; there is a series of external exams, with modification. Indeed, there is the potential for further modification, depending on what happens. If the past two years have taught us anything, it is that none of us can predict what will happen. The third part is around estimates. That becomes important in ensuring that there is no need for a dual system that overburdens our staff and our young people. The clear message for us is that we must make sure that we are confident in the kinds of assessment that are used.

When we talk about teachers’ professional judgment, we all need to have a collective understanding of what that means. Sometimes, that might mean looking at an exam or an assessment activity or at a piece of work such as a report, or a practical performance element. We need to have clarity on what that means when we give an estimate for young people. For me, it is really important that we make sure that we do not say that we are running exams while, in the background, we are running the ACM—that is not happening. This year, we need to make sure that the evidence is tailored and that the natural assessment points are used for generating assessments and estimates for young people.

I should have declared an interest as a councillor on South Lanarkshire Council, where Tony McDaid is the head of education.

I would like to return to the questions around moderation and the issue that Oliver Mundell raised about the use of historical data. I completely understand the need for a level of moderation to ensure that an A grade in one school is equal to an A grade in another, but moderation that includes the use of historical data—school-level performance data—seems to do the opposite. We have had an attainment gap in Scotland for a long time—a socioeconomic attainment gap, as well as one between those with and those without additional support needs. Surely, any moderation system that uses historical data automatically puts more of a question mark over higher levels of achievement by young people from a deprived background compared with such levels of achievement by pupils from a more affluent background. If a class of higher pupils in Drumchapel had got straight As, that would have been viewed with more suspicion than a class of higher pupils from Newton Mearns or Clarkston having done so. How did you deploy a moderation system that included the use of historical data without simply having far more conversations with teachers at your schools in areas with higher levels of deprivation?

It is very important that people receive reassurance on that question. You are quite right in saying that historical data is backward looking; it involves looking at the past rather than the future. The hope is that it gives us a sense of what we are doing. For us, it is a case of continuing to make sure that teachers are confident in the grades that they give.

The process involved having a conversation with our school leaders. It was not a case of us saying, “You need to go back and make a change.” It was a genuine professional dialogue, which involved saying, “Here’s what the provisional grades look like and here’s where we have been in the past.” I think that it is legitimate to say, in particular subject areas, “Here’s what it’s looking like.” It was a question of engaging in a moderation process as part of the qualifications process. That was about understanding standards; it was not about saying, “You’ve just made a grade boundary,” or “You’re looking at a percentage, and it’s just gone up or down.” It was just about making sure that candidates and young people were given the benefit of the doubt.

I can understand people being worried, on the back of last year, about an algorithm being applied to that. The whole way along, in all our conversations with our school leaders and our teachers, we said, “This is about the candidate.” This year, if there are 20 As in a class and the candidates have earned it, they should get it. That is really it—nothing more and nothing else. If that has never happened before, it is legitimate to ask, “Are you confident about the standards of that?” Once the process has been gone through, at that point it is about the teacher delivering the grade and there being a confidence level across different schools and different subject departments.

I completely understand that there is a legitimacy to that question if it is suddenly the case that there are 20 straight As in a class where that has never previously been the case. However, the questioning of high grades is disproportionately more likely to have happened in a school in a more deprived area.

Have you done any assessments since last year to check how many quality assurance conversations you had with school leaders in your most deprived communities compared with the number that you had with school leaders in the least deprived communities? Have you checked whether there was a disproportionate amount of such quality assurance going on?

It was the other way around—the schools would come to us. The agenda was set by the schools. They would come to us and talk through their assessment story: “Here’s what we’ve done, here’s the understanding standards activity and here are the grades that the young people will be getting, and we’re reassured that that’s the case.” It was not a case of us saying, “By the way, this looks out of kilter.” It was down to the schools. The schools have been doing work around understanding their grades for a long time. It is also a case of making sure that school leaders, faculty heads and individual class teachers have a sophisticated understanding of the process of applying a grade. For me, that was the important part of it; it was not about a statistical analysis.

Would you like to come in, Audrey? I keep firing more questions at Tony, but I realise that I have not given you a chance to respond.

I am sure that I would have given much the same answer. I emphasise that it is our responsibility, as a local authority, to have quality assurance, which means that we have to look at that overarching data. That is what opens up the professional dialogue. This is about us being part of the story; it is not about moderation of a standard at the level of an individual child’s qualification. That happens in a classroom, with a faculty lead or a principal teacher; it then happens with a senior leadership team leader and at headteacher level. We have an overview of that data, which means that we could just give that data to our schools. As I said, this is the third year that we have used that approach for quality assurance. It has not been brought in because of the situation here.

When it comes to analysing differences, we tend to want to ask questions where there have been significant improvements. We want schools to tell us the story of that, because we want to share the good practice. I could give you examples of a significant improvement in results, where we were able to go back to the headteacher and say, “We appointed a new principal teacher last year.” There is a whole story there about how the position shifted. It is that dialogue process that is important. It is part of our accountability as a local authority to have an overview.

Most local authorities that used historical data as part of the process used data from in the region of 2015 to 2019; they excluded data from 2020. What did your local authorities and RICs do? Did you include the 2020 data in the historical average?

We did.

We did.

It told us another bit of the story.

I am keen to hear the rationale for that. In the areas where local authorities excluded that data, such as those in my region, I heard much more from teachers and pupils, who came forward with concerns, because the one year in which the gap closed quite considerably was excluded for moderation purposes. Could you explain why you felt that it was appropriate to include 2020 data for moderation?

From our point of view, it was because it was there, so why would we not look at it? We did not overthink it. We were gathering data at that point, because we were doing a longitudinal study around our data. We had five years’ worth of data, and we added in the 2020 data. As part of the professional dialogue process, we were very aware of what that data was telling us and of the fact that it was an exceptional year, but, in my view, there was no point in our ignoring it. It was there and we had to ask questions about it, which informed our discussion.

Although the models were not the same, there were similarities between the 2021 model and the 2020 model, so it was legitimate to look at that. In addition, the young people in 2020 earned those grades, so that would have to be taken into consideration as well.

Is there time for me to ask another question, convener?

Just.

In that case, I will be very brief.

I want to go back to the issue of the volume of assessments—it might have been Willie Rennie who raised it—that young people had to sit in the three or four-week sprint in April and May, in particular. Yesterday, I spoke to a young person who had had 30 assessments in a fortnight, and they were taking two highers and two advanced highers, so that was on top of dissertation deadlines and so on. Did you receive any guidance from the SQA as to how those final assessments should be timetabled to avoid that kind of compression? A lot of that was due to the perfectly valid motivation of teachers to let pupils sit the same assessment over again a couple of times to maximise their chance of getting a good grade, but the cumulative impact was quite negative for the mental health of some young people.

11:00  

The cumulative impact was really challenging. Would we ever want to set up a system that would hold the process that you have just described within two or three weeks? Without repeating what I have said, it was an exceptional circumstance in that it was not really possible in the time to get to the stage of gathering some of the evidence that we would naturally have gathered in January, February and March in relation to concepts of learning.

In terms of the SQA, the message on the general data was about not repeating assessments, but, given that the subject guidance said that there needed to be close alignment, that is what will happen. Again, I would make a point about the kinds of professional judgment that are made and what evidence can be used for those; I am not just talking in absolute terms about high predicted value and low predicted value—actually, it is higher and lower. It can all be used; the issue is the extent to which it can be used. If—understandably—schools try to keep the predicted value as high as possible, that is what we end up getting. Looking forward to this year’s process, we need to make sure that something like that does not happen.

Did you—

We are now out of time.

That is fine, convener.

Kaukab wants to say a brief final word.

I want to thank our two witnesses. Implementation at local authority level is a huge challenge. I worked as a teacher throughout the whole period in question, and I pay tribute to the local authorities for responding so rapidly in unprecedented times. Today, you have explained to a wider audience the complexities involved.

I thank Audrey May and Tony McDaid for joining us. The evidence that they have given has been insightful and very useful.

I suspend the meeting briefly to allow us to change panels.

11:02 Meeting suspended.  

11:07 On resuming—  

Welcome back. We will hear further evidence on the alternative certification model from our second panel of witnesses. I welcome Larry Flanagan, general secretary of the Educational Institute of Scotland; Seamus Searson, general secretary of the Scottish Secondary Teachers Association; and Tara Lillis, national official for Scotland at the NASUWT.

Thank you for providing us with your written submissions, which are very interesting and useful. We have a lot of ground to cover and not as much time as we would like, so we will move straight to questions unless any of you has something short and specific that you would like to say first. I see that you are all happy to move straight to questions.

Larry, you state on page 3 of your written submission that

“the Scottish Government or the SQA ... were determined to push ahead with national sampling of all courses”.

Did you ask for a rationale or a justification at the time for the Government’s determination to push on with that? If so, what did it say to you?

I do not want to put words into the Scottish Government’s mouth, but the discussion was largely about whether S4 pupils could bypass qualifications in the previous school session on the basis that about 94 per cent of them would be moving into S5. If they could progress from S4 into S5 on the basis of teacher assessment, that would give them more teaching and learning time and allow them to consolidate their understanding ahead of the S5 diet. It would also lighten a significant workload. Because S4 is the biggest year group in terms of qualifications, it would have benefits for teacher workload in relation to S5 and S6.

We noted that S4 pupils who were exiting school would have to be catered for. However, the vast majority of them go on to college courses, and most of the articulation between school and college courses at 16 revolves around unit assessments, so arrangements could have been made.

We did not get into a discussion with the Scottish Government about its rationale or its reasoning on that. I assume that it thought that the proposal would be politically challenged and it was keen to be seen to be delivering qualifications for S4, S5 and S6 pupils. It would have been a big political decision and the Government clearly had a view that it wanted to certificate all pupils, as far as possible, in the normal manner.

In your submission, you mention a number of times that you believe that there was a hugely political dimension. On page 3, you mention the awareness of the forthcoming Scottish Parliament elections. How much of the decision making in the process was driven, in your view, by a political agenda from the Scottish Government?

Trying to be even handed, I would suggest that all the political parties viewed what was happening in schools through a political prism with a view to the elections. In fact, I would be fairly critical of some of the noises off, because the politics around what was happening in schools was unhelpful to them. We would have preferred to have much stronger consensus from the parliamentarians on supporting teachers to accredit young people. As we say in our submission, the noises off were unhelpful.

Once we got to the point of the ACM, teachers focused on delivering for their students. I mention in our submission the Herculean effort on the part of teachers to make sure that young people were not disadvantaged any more than was obviously the case due to the pandemic. Ultimately, although there are lots of issues around aspects of the ACM and it was not our model, it delivered the qualifications for young people. That was partly due to the resilience of the young people, but it was largely due to the work of schools.

There were a lot of noises off, as you call them, in political terms. Education is a huge issue in Scotland. It is a priority in people’s lives, and that is a jolly good thing. However, the Government was clearly in the driving seat and making the decisions.

There are many questions that I could ask, but I will put just one more question to you, because I want to bring my colleagues in. On the sampling issue, you make in your submission the rather incendiary comment that

“the EIS’s trust in teacher judgement was not matched by that of the Scottish Government”.

On what basis do you say that?

11:15  

The comment refers to the Scottish Government and the SQA. They were very keen on what they referred to as a robust quality assurance process. We were not opposed to a quality assurance process, but we were keen to limit the role that the SQA would have in that, beyond promoting the understanding of standards. We did not want the SQA to be able in any sense to overrule the professional judgment of teachers. We were insistent about that, particularly on the back of the previous year’s experience with algorithms. We believed that the quality assurance process must support the professional judgments of teachers and the process to arrive at those. We were not prepared to concede any kind of veto for the SQA.

In our submission, we include the Scottish Government in relation to that issue of trust. Our challenge was for the Scottish Government and the SQA to agree to a system that was based on trust, on the professionalism of teachers and on their ability to arrive at the estimates—the grades—on behalf of the pupils.

The point that I am trying to make is that you are saying something fairly damning about the Scottish Government’s trust and belief in teachers.

Before I bring in Kaukab Stewart, the deputy convener, I have a question for Tara Lillis on a subject that we touched on with the previous panel. Tara, you mention in your submission the concept of overassessment and the compression that went on in the assessments this year. Have you or any of your members sought to define that? If so, what conclusions have you reached?

It gets to the nub of the issue of manageability and workload within the system of the alternative certification model. One of our executives posed a question about overassessment directly to Education Scotland when it came in to review a local authority’s process and procedure. Did Education Scotland have a concept of what overassessment would be within the alternative certification model? An answer was not forthcoming: Education Scotland did not have an idea of what overassessment would be.

The experience of our membership on the ground was that the alternative certification model was unmanageable. The report by Professor Gordon Stobart says that any model of assessment has to encompass a balance between manageability, validity and reliability. In the system that operated in 2021, manageability somewhat fell off a cliff in that there was compression—a last-minute dash—and overassessment that was unmanageable for teachers, but also for the pupils who were undertaking the assessments.

We could talk more about the costs that go with that compression and overassessment, but I have no doubt that my colleagues will get on to them.

I will bring in Fergus Ewing, who wishes to ask a question at this point.

Mr Flanagan said that, although the Scottish Government did not accept the approach to the alternative certification model that the EIS recommended, the outcome was nonetheless satisfactory. I welcome that as a positive comment. With that in mind, I note that the national 5, higher and advanced higher examinations will be held in spring 2022 if the public health advice allows it. The decision will be informed by the public health advice, and course content will be reduced compared with a normal year, with further contingencies if there are further difficulties because of disruption due to Covid.

Mr Flanagan, are you broadly happy with the approach that has been set out, which I have described very briefly? Do you have any particular views and suggestions on how this should operate in 2022?

The EIS supports a system of senior phase assessment that is predicated on exit qualifications. We think that, across the senior phase, we should look at the qualifications that young people will leave school with as their passport to the next step. Currently, we have a year-on-year, stepladder approach that starts with S4 qualifications and then moves on to S5 and S6, which robs time from teaching and learning. In our view, we have a poorer assessment system now than when we had standard grades and the national qualifications intermediates, because of the challenge of time with the way that the senior phase is structured over three years.

This year, the Stobart review and the OECD report create a context for our looking afresh at the issue of exit qualifications so that we can achieve the ambitions of breadth and depth of learning and parity of esteem between vocational and “academic”. This year’s S4 were last year’s S3, and we hear young people saying that S3 was forgotten last year because it was outside the qualification focus. We would have targeted this year’s S4 on a two-year pathway to qualifications in S5. That would have allowed more time for consolidation of learning, particularly given the challenges that those young people faced throughout last year. However, that was a step too far in relation to political decisions, and we are now going for the qualifications in terms of the diet.

I can understand why the Government decided on that approach. A review is under way, which will have to avoid the mistake that was made with the previous set of changes, which was to push ahead without getting buy-in from the profession. The review will have to take a bit of time for any changes to bed in. However, reverting to the existing system was probably the most straightforward approach. We could not have moved to a continuous assessment system this year. That would have been a challenge too far, and people are already exhausted even though it is only September. The diet was an almost inevitable decision, but we would have liked to see a bit more courage around the S4 cohort, with a focus on their learning and also their wellbeing, because it has been a challenging period for them.

I understand the line of argument that you are pursuing and which you pursued last year. It did not have a successful outcome, but you had the good grace to acknowledge that there was a good outcome in terms of the overall results and the highers achieved. My understanding is that the Scottish Government placed immense trust in teachers and that it values their work enormously.

Are you still in discussions with the Scottish Government about your suggested tweak or amendment to its approach to S4? Is that an on-going matter of discussion between you and the Scottish Government?

We are involved in what is now called the NQ22 group. It is not meeting as frequently as the NQ21 group did, because there is no immediate pressure for a contingency. There is no discussion on S4—that decision has been made—but there are discussions about what the contingency would be in the event of the diet being cancelled. We do not believe that it should be the ACM model that was run last year, nor do we believe that it should be the expediency of the previous year. We are keen to ensure that schools are clear about what the contingency would be and that any contingency would minimise additional workload and assessment for students.

The SQA has put out guidance that says that any contingency will be based on naturally produced evidence. That ticks the box, but what does it actually mean in terms of quality assurance? The issue last year was not so much the evidence; it was the quality assurance process to validate that evidence. We would be comfortable with the notion of naturally produced evidence, but there are issues around it.

Thanks. It is nice to see you again, Larry. I declare interests, as a former member of the EIS and a current associate member of the NASUWT. I am also registered with the General Teaching Council for Scotland. That is that over and done with.

We have taken lots of evidence from young people over the past few weeks. I have been speaking to children, so I am trying to put that together with your presentation. I am interested to hear more. Your submission states:

“schools defaulted to running exam type assessments”,

which takes us back to Tara Lillis’s comment about compression. We heard a lot from young people about the additional stresses, but they were at great pains to say that the staff and teachers helped really well. Can you say a little bit more about how that happened, because the message was that there were meant to be no exams? How did that turn into there being multiple assessments that were regarded as exams?

There are a few aspects to that. There was quite a contested dialogue within the national qualifications 2021 group on the validity of evidence. In the alternative certification model guidance it is very clear that it is for teachers to decide what evidence is appropriate for them to arrive at their estimates. That evidence can be a class test, it can be a preliminary paper from another year or it can be class assignments. SQA subject guidance was unilaterally issued to the system. In some subjects, particularly maths and physics, the guidance leaned very much towards valid evidence being based on exam practice, because that tests the full range of skills. A number of schools immediately latched on to the SQA advice.

What was appropriate was hotly contested within the NQ21 group. Ultimately, it was left for schools to decide what they regarded as being the best assessment approach for their students. As Kaukab Stewart will know, as a petitioner, the arrangements will vary from subject to subject. I am an English teacher, so it is pretty straightforward for me: I can have pupils write a critical essay or write an essay to do a close reading passage, and I have the evidence that I need. In maths, in which the topics are much more disaggregated, schools will tend to go for an integrated paper. I do no disservice to maths teachers when I say that they probably quite like to use past papers because they cover a broad range of skills. It was left for schools to make those decisions and it was not required that the school take a single approach.

The other big factor that then came into play was the three-month lockdown that was not anticipated. Pre-Christmas, we were arguing to the Scottish Government there should, as a firebreak, be an extended break over Christmas. That was resisted; we then came back to three months’ lockdown. The advice to the system was that evidence gathering should not happen remotely during lockdown because of challenges around validity, equity in terms of access and so forth. After schools came back, there was a very truncated window in which evidence could be generated. Bear in mind that the highers had only switched to an ACM model in December, so some schools had not held prelims. What then happened in many subjects, I think, in the majority of schools—I have not factually assessed this—was that mini-exams in the classroom were used. For fourth-year pupils, in particular, who were doing seven subjects, that was a lot.

11:30  

We did away with unit assessments because the unit assessments were ending up as a treadmill of assessments, particularly around February and March. Some schools organised timetables and took people off-timetable for study leave. I understand why the majority of students thought, “This is just an exam diet.” Many schools tried to give young people who did not achieve their perceived potential the first time round a second chance. In one sense, that adds to assessment, but it also gives the student a second chance, so we can go with the best result.

During that period, when there was media coverage on issues of equity, not all of what was being done used the ACM model per se. That was an impact of three months’ lockdown and there being a very tight deadline. Members will see in the documentation that, within the NQ21 group, we pushed for extensions to deadlines in order to maximise the time that schools had in which to overtake the assessment challenge.

I am keen to hear from the other panellists who are online. Larry Flanagan was quite critical and said, in effect, that there were exams of sorts, although teachers were trying to do assessments. Some have said that the assessment process is much less stressful than the exam process, in which there are “big hit” exams. Does this year really tell us whether that is right or not? As you said, teachers and pupils were incredibly stressed by repeated assessments throughout the period.

Secondly, on your point about changes to the fourth year this year, does that throw up a much more significant question about the age of leaving education? If we want pupils to leave with something worth while, do we have to change the age at which they can leave school or education?

On the second question, as part of the curriculum for excellence senior phase arrangements, around Christmas of S3 schools are supposed to map out a three-year programme for young people to the age of 18. That might involve leaving school, going to college or going to work, but there is meant to be a three-year plan. Schools have a responsibility for planning the learning journey for the16-to-18 period onward.

There are real difficulties around that because of how our qualifications work at present and because of the challenges around S3 and the lack of connection between the broad general education phase and the senior phase. In a sense, Willie Rennie’s suggestion about raising the school leaving age has, de facto, happened because students are staying on at school. When standard grade was introduced, 90 per cent of boys left at the end of S4. We now have 94 per cent of pupils staying on to S5, and around 85 per cent stay on to S6. We do not need to raise the leaving age; young people are staying at school, so we just need to cater for that.

On whether we have learned anything from this, I say that we absolutely have. We need a mixed economy in assessment because what works in English does not necessarily work in science. We need to get away from high-stakes do-or-die assessments because they are unfair. Some people relish the challenge and do well; teachers are, by and large, winners at that because they have come through the education system. However, that is not the only way to assess, which is what Professor Stobart is telling us. That means not that we will do away with exams but that we put them in a context in which they are not the be-all and end-all.

We know that high-stakes exams disadvantage people from poorer backgrounds. A mixed economy in which progression that they can bank can be made by young people is one of the things that Scottish Education likes, which is why we have S4 qualifications. We need to build that into the system. The really big thing that we learned from this year is that young people and their parents trust teachers. Our qualifications system should be built on the premise that there is trust in the professionalism of teachers to validate young people’s learning experience through qualifications.

I want to focus on the background. What has been said this morning is what many of our members have reflected as having been their experience. Before the ACM was applied, after Easter this year, our members were very worried about the young people whom they were teaching—the impacts on them from lockdown and Covid, and the disruption that it had caused to their families.

A major thing for us was that, after Easter, when the ACM was implemented, teachers were confined to using only evidence that was demonstrated during that period, which did not take into account children and teachers being off during that period. That means that some youngsters missed vital periods of time for assessments. We tried to focus on all the knowledge and experience that teachers had of the young people over many years. As soon as a youngster walks into a secondary school, they are being tracked and monitored regularly in all their subjects, which helps in their development.

When the new model was introduced, in April 2021, that was all ignored. That hurt our members and it is why many of them were frustrated. They were under a very strict timeline in which they had to produce evidence, but they were not quite clear about what evidence was required. Many over-collected evidence in a short period in order to try to get something that would be approved by the SQA. Our view is that we should have been able to use previous knowledge and experience. We talk about trusting teachers; trusting their professional judgment means that sometimes—not in every instance—teachers would override evidence and could justify that because of other things that had gone on in the past, or because of difficulties that a youngster was facing at the time.

An issue for us was that communication appeared to be one way. I raise that because the SSTA represents 6,500 secondary school teachers and headteachers. When it came to dialogue with the NQ21, the SSTA was refused permission to join that group and has not been invited to join the NQ22 group. Communication from people who represent teachers is not getting to that group. In the membership of the NQ21, there is only one representative of those who actually deliver exams in schools. There is a deficiency there in that, in an argument at NQ21, it was one person against the rest. We need to balance communication about what teachers are trying to move forward from.

On where we will go in the future, the ACM has some good things that we need to look at. It was application of it this year that caused the greater problems.

I also want to focus on moderation. Back in 2020, the teachers gave their professional judgment and it was accepted, although maybe some people had concerns. However, the 2021 exercise involved a series of different moderations, as if the teacher was not being trusted. There was moderation at department level, at faculty level in the school and at school level. Then there was moderation with other local schools, with the local authority and with the regional improvement collaboratives and then, at the end, with the SQA. You have to ask yourselves how much of that was necessary. One lesson to be learned is about whether we need all those levels to deliver assessment.

The downside is the impact on the pupils, unfortunately. If the youngsters’ experience of April to June this year is what they think exams will be like in the future, many will opt out. In that period, we should have been building on teaching and learning, providing encouragement and—as was mentioned earlier—putting the fun back into education. We did not do that this year. We drove them into the ground to get the assessments. If there is one lesson to be learned, it is that we have to assess youngsters properly and that teachers’ professional judgment is the overriding concern in the process.

Given those comments and the comments from Larry Flanagan, is there a feeling that the SQA was given too much say in the ACM and that really it was trying to introduce its normal methods earlier in the process? That point came up in the earlier evidence session. In the development of the process, was the SQA’s voice stronger than that of classroom teachers, or was the balance right?

Seamus Searson said that there was one voice in the NQ21 group. To be fair, School Leaders Scotland and ADES were often on the same page as the EIS, so there was quite a collective voice on the reality in schools. There is a level of quality assurance even around prelims, through things such as cross-marking and sampling by principal teachers. That was expanded, which is why quality assurance was one of the biggest drivers of workload—people were marking other schools as a double check. Local authorities were supposed to support the quality assurance processes using their quality improvement officers, although not all of them have QIOs. That local authority aspect of quality assurance was a cohesive approach that was about supporting and understanding standards and making sure that schools were being as accurate as possible.

The SQA quality assurance, in which it sampled scripts, was a different agenda. The SQA probably wanted to sample more scripts than it did, but it was impacted by the three months of lockdown as well. The key issue for us—we were supported in this by ADES and SLS—was that the sampling was about a professional dialogue with schools and not about the SQA overturning any school results. The SQA had its fingers burned from the previous year with its algorithms and quota systems. I would be delighted to talk about that if you want me to, because that is where the real fault lies in high-stakes exams. When you have quotas for As and Bs, the kids who are Cs and Ds end up losing out. The SQA was clear that it was not looking to overturn school-based judgments, and that was one of our red lines in the NQ21 group. The SQA could have a quality assurance input but it could not veto what a school decided on qualifications.

Even during the lockdown period, there was a lot of staff engagement in the understanding standards webinars that were run by the SQA. However, as Seamus Searson alluded, the whole thing was under huge pressure because of the truncated period when pupils and staff were back in schools. I reiterate that it is a testament to the work that was done that, when the results were officially announced—although pupils knew their results before the summer anyway—there was not a huge outcry in the way that there was the previous year. By and large, the student cohort is content that it has been appropriately credited for other learning and with the qualification outcomes.

I ask the same question of Seamus Searson and Tara Lillis. Did the SQA have too strong a voice in developing the ACM, given the clear feelings in 2020? Was it trying to retain influence over the process?

After 2020, the SQA wanted to put control back on the system, so we got an SQA model that was implemented by different groups on the way, but it was the same thing. We have to realise that it is about the young people we are teaching—we want them to have a positive experience in schools. However, with the way in which the national qualifications were implemented this year, in effect, we just reverted back to type. I would go so far as to say that, for 2022, the intention is to go back to where we were before.

11:45  

Even though the SQA has sent out messages, the worry for us is that schools will already be collecting information for the version of our ACM for this year, should the exams not be able to go ahead. Therefore, we are already increasing people’s workload. Because we do not know what the future holds, there was an opportunity to work with teachers. That takes me back to the point that I tried to make earlier: you need to listen to the teachers who are delivering in schools about what their experiences are and try to make a system that works to everybody’s benefit and does not put horrendous pressure on everybody during a very difficult time.

My concern is that the SQA has decided to go back to exams. We were not consulted on that; the decision was made by the NQ21 or NQ22 group—whichever one it was. There is real concern that we should have planned for the situation and put something else in place early on as a makeweight that would have allowed teachers to plan and guide their youngsters through this academic year.

Since 2020, the NASUWT has been calling for the SQA to bring teachers into its confidence and to make them a core part of the discussion and decision making, because of the wealth of expertise that they can bring to the discussions. As has been outlined, there has been a lack of collaboration and engagement. The NASUWT is not on the national qualifications group. It was hoped that, given the controversies in 2020, there would be recognition on the ground that a more inclusive approach would be taken that put teachers’ views at the heart of the decisions, but that has not been the case. As it turned out, what was adopted in 2021 very much felt to teachers like a done-to approach and not a done-with one.

I am interested in how you will incorporate young people’s views. How do we move forward on that and how do those views influence you?

Clearly, the three of us are primarily concerned with representing our members’ views collectively. As Seamus Searson and Tara Lillis have outlined, they were not involved in the NQ21 group and are not involved in NQ22. We have a dialogue in the teachers panel—the Scottish Negotiating Committee for Teachers—but that is not a direct line. The decision on inclusion in the national qualifications group was, I presume, made by the SQA—it invited us to join the group.

It is important that young people’s experiences are captured. There was representation on the NQ21 group from the Scottish Youth Parliament, and the SQA has a learners panel. Because of the relationship between teachers and pupils, we often get feedback from our members on how young people are feeling about things, so we have some insight into some of the challenges. If you look back at the EIS’s communication during the pandemic, you will find that the health and wellbeing of young people has been one of the key issues. That is because it is difficult to teach young people if they are stressed and face personal challenges.

There is a balance to be struck in looking at qualifications. Every S4 cohort is new to qualifications, so they do not know what they do not know—they do not know what the alternatives are. Young people who do well in exams like them, because they do well, and young people who do not do well in exams wish that there was something else. It has been interesting to hear some of the comments last week on continuous assessment. We need to think about how young people learn and make sure that that has been factored in.

As the convener referred to, the approach that was taken at a school level meant that lots of young people felt that the teachers were on their side. That is a critical dialogue. Young people were given information on their progress and were kept up to date. Some in the NQ21 group wanted a summative approach, with schools telling kids their results at the end of the year. So, in the last week of term, a pupil might be told, “You’ve managed to get a C.” However, we were clear that it had to be an iterative process across the year, with teachers talking to their students and keeping them informed about progress, what the next steps were and what they had to do to improve.

That on-going dialogue is critical. I am slightly wary about simply setting up panels and thinking that that captures how young people feel. The key issue with that is whether the panel is representative of most young people. We need to listen to young people at every level, but particularly at the classroom level.

It is important that we learn from the experiences over the past few years and that we talk to youngsters who have been through the process. As Larry Flanagan said, when pupils start in the process, they do not know what the options are, but teachers know that. We need to think about different approaches for different pupils going through the system.

Teachers should be listened to, because they engage with young people all the time. I am not saying that they can speak on behalf of their pupils, but the teachers have a view of how the situation has impacted on young people, and that needs to be taken into account. Teachers cannot speak totally on young people’s behalf, but we can give our view of how we see them. In the period from April to June, our members were very worried about the stresses and pressures on young people and felt that that was totally unfair. We need to take that on board and ensure that we do not repeat it.

I reiterate the comments that Seamus Searson and Larry Flanagan made that, although listening to teachers does not preclude engagement with young people, the feedback from our members over the period has been that they have been advocating strongly on behalf of their pupils, in addition to making representations about the impact of the alternative certification model on teachers’ workload and wellbeing. Frustrations were voiced by teachers of practical subjects about the fact that the approach was like making pupils sit a driving test without their ever having sat behind the wheel of a car. The teachers made those representations with the best interests of their pupils in mind. Under the United Nations Convention on the Rights of the Child, there absolutely is an onus on the system to consider and incorporate the views of children and young people, but we should not think that those of us who are advocating on behalf of teachers are not also incorporating those considerations in our representations.

Where should this committee’s focus be going forward? Where can we offer best value?

The big issue is the OECD report and Ken Muir’s consultation. That will require systemic change. From looking at the age profile of the committee, some members might remember that, when you used to go to the pictures, you sometimes went in halfway through a movie and saw the second half first. You then watched the first half, and at some point, you got up and said, “This is where I came in.” That is a wee bit like how I feel now. I came in with the last big changes around the senior phase, and now, with the OECD report, it looks as though we are getting back to where we wanted to be 10 years ago, which is thinking about how we have an assessment system that captures the ambitions of the senior phase and CFE.

The committee might well want to focus on how we progress that, because it would be absolutely brilliant if we had all-party buy-in to changes that were beneficial to our senior secondary schools—I mean our secondary schools. Sorry, that was me going back in time again.

That is helpful—thank you.

That is what the OECD encourages us to do, is it not—to go back to the vision?

Yes.

Larry Flanagan has distinguished a couple of times between the problems that were inherent in the ACM and those that were compounded by the lockdown period and school closures from January to March. When our predecessor committee was scrutinising the SQA last autumn and in the spring of this year, it was very hard to get an understanding of what scenario planning had been done for a period of prolonged school closure during the year. What is your understanding of the scenario planning that was done by the SQA and by the Scottish Government last summer? The answer that we often got was, essentially, just the repeated affirmation that schools were not going to close. Are you aware of any scenario planning being done on the impact of prolonged closure on the certification model?

Before the NQ21 group, there was an SQA contingency group, so there had been an on-going dialogue about the 2020 diet and the algorithm agenda. There was discussion about what would happen in the subsequent year, and some of that was about what might happen with S4, for example. To the best of my knowledge, there was no serious discussion about the potential for a three-month lockdown—I do not think that that was anticipated. As I said, before Christmas we were trying to get a firebreak and that was being resisted. The Kent variant, which hit Scotland in December, was a game changer for that, and the contingency that had been developed for national 5 in October, when the exams were cancelled, which was mainly to do with disruption rather than lockdown, became the template for the alternative certification model for S5 and S6. That was then challenged by the full lockdown period. If your question is whether I think that there was sufficient contingency planning, the answer is that there probably was not, but hindsight is a wonderful thing.

Would Seamus Searson or Tara Lillis like to comment on scenario planning and whether that took place?

We raised concerns in summer 2020 about what was happening in 2021. Larry Flanagan talked about discussions that took place in other places that we were not party to—not only the NQ21 but the coronavirus education recovery group. As far as we are concerned, it was full steam ahead as planned. Therefore, to answer your question, we were unaware of any contingency planning, and I would imagine that most of the public would be of the same view.

12:00  

Tara Lillis, were you in a similar position of representing a union that was not on any of the relevant Government groups?

Yes, we were in a similar position but, obviously, we engage in informal dialogue with the SQA and the national qualifications group. I stress to the committee that it is a concern of the NASUWT that the lessons about a lack of contingency planning have not been learned as we look forward to potential disruption this year. For example, when we met the SQA informally on 28 May, we said that it was important that there were sufficient contingencies in place for the year coming so that we could avoid the mistakes that had happened with the ACM and that there was clarity in the system as to what the potential options would be moving forward. More than that, we said that it was important that we were clear what the triggers would be for moving from one option to another.

Further, in direct communication with the Government, we were stressing the need for a decision on exams to be made before the summer—again, to provide clarity and consistency of messaging to the system. As it turns out, we did not receive any clarity about exams going ahead until 18 August, over a week and a bit after some schools had returned. Indeed, the SQA scenarios were published only on 15 September, over a month after schools returned. That scenario planning is attached with a caveat that says, “Here is a broad outline, but more information will be provided in October.” There is a consistent “jam tomorrow” approach being adopted by the SQA in the information that it provides to the system.

That sounds wearily like the exact same conversations that we were having this time last year.

I will move to a question on the moderation system. Seamus Searson, you listed the various levels of moderation that provisional grades had to go through before they were approved, and you spoke about the workload issue that that created. I am interested in the feedback that you have all had from your members about how much moderation changed grades from what a teacher might have initially been minded to give. Did that moderation process result in much in the way of grades changing, and was there a particular level at which that was most common? Did grades typically change on the basis of the conversations that were taking place at the faculty level within a school, or was it on the basis of conversations at a local authority level? Did the RIC-level moderation influence grade changes? Was there much change as a result of that process?

The feedback from our members is that there were some alterations going on. There were pressures in some places to downgrade grades because they were out of keeping with those in previous years, so those examples do exist. There were not too many asking for grades to be pushed up; it was more questioning what the level of the grades was.

That happened at different points. Sometimes it happened in the department, and sometimes it happened across schools. Local authorities—whether they like it or not—did look back at the history of the schools and used that as a guide. The schools did not want to put their head above the parapet and be identified as being of concern, so many schools were conservative in their grades and tried to have them in keeping with previous years. That is what we have been told by a range of different members, so I imagine that that happened in many, many cases.

I accept that that is anecdotal feedback from your members, but were the areas where there was pressure to downgrade disproportionately schools and areas that historically had lower performance that is typically linked with lower socioeconomic status and deprivation?

If I said that we had very few appeals from the high-performing areas, that might be a way of answering the question.

Tara Lillis, what was your members’ experience of this? Did you get similar feedback, or was there a different experience?

The feedback was variable. It is one of the challenges in the Scottish education system that consistency is not necessarily seen across the picture, and different approaches were taken to moderation within individual schools, local authorities and RICs. The feedback came from members who felt confident in themselves to push back when they disagreed and felt that their judgments were sound, and the feedback that we received was from members who were informing us that those kinds of discussions and interactions had taken place. The concern is that we may not have heard from those members who had those discussions and who felt that their judgment had been overturned, perhaps wrongly. I do not think that we have the oversight across the system to make evidence-based decisions on what did or did not happen in practice. That is partly down to the flexibility and the variability that are inherent in the system.

A key part of the quality assurance process was understanding standards, initially. There were a lot of teachers engaged in professional learning around understanding standards. A lot of the cross-marking is literally that—it involves a second marker. In small departments, it is useful to get another school involved. We do not have any difficulty with quality assurance other than the time that it requires, which is always a challenge.

Similar to Seamus Searson and Tara Lillis, when we were contacted by the very few members who said that they had been told to raise or lower the grade boundary, we simply said that that is not what the ACM said and that they should refuse to do it. Such occasions were few and far between, though. There was a big discussion around grade boundaries because, in the absence of national data and the SQA setting a grade boundary, some schools varied it and some stuck with 50, 60 and 70 as their defaults. There was no easy solution to that other than setting a national standard, which takes us back to the point that, if you have national standards, you have national exams.

Our members are assured that their judgments held sway in relation to the final qualifications, and, on the few exceptional occasions when that was not the case, we certainly challenged that.

My final question is for Larry Flanagan, given that his union represents college lecturers.

The experience of college students and lecturers is a significantly underexplored aspect of the past two years, but I have had a college student get in touch with me recently to point out what they felt was the inequality of a system in which they were, in the end, graded on the same terms as any school pupil even though they had spent the entire year learning remotely. There were at least periods of time when school students were in school in a classroom, but that was not the experience of college students. How would you reflect on the communication that the SQA issued to both college lecturers and college students? What was their experience relative to that of teachers and pupils in schools?

Colleges Scotland is represented on the NQ21 group, but EIS had a very specific dialogue with the SQA about college national qualifications and other qualifications in college. I would say that it was a robust dialogue. There was clearly a different dynamic because it involved adults rather than pupils, so, as you say, lockdown measures and remote learning had a different role to play.

In the case of national qualifications—the same exams that pupils would be sitting—ultimately, we were comfortable with the arrangement, which was the same in that it would involve professional judgment based on evidence. Your point is that some students might feel that they did not have the same opportunity to produce the evidence, and I accept that that would be the case. As part of our dialogue, we provided the SQA with quite a detailed student survey that we carried out on the impact of Covid on students from disadvantaged backgrounds, which showed that the impact on their potential to learn was quite dramatic.

There were other qualifications that the SQA oversees in the college sector, some of which had to be deferred because they were based around a competence model, and you cannot accredit somebody with safety standards if they have not completed a course. However, I think that our message around colleges would be similar to the one around schools in that, ultimately, college lecturers pulled out all the stops to make sure that young students were accredited appropriately. I take your point that the learning experience of many students was disadvantaged because they are a different cohort that includes young parents with families and childcare responsibilities who had to miss lessons because their kids were off school. It comes back to the point about the pupil and student voice—we should be listening to students about what their experience was.

Thank you for asking that question. I, too, have had representation from students in colleges who are reporting similar experiences to the ones that you just described.

I want to look slightly beyond the qualifications process and immediate assessments to the learning. I was struck by Seamus Searson’s comment about pupils being driven into the ground by assessments and the convener’s remarks about compression being the theme. We understand that there was, in essence, less teaching across the year for a variety of reasons, particularly for some cohorts. There was less time in school, so learning was difficult to access. We have taken evidence from young people in the past couple of weeks, and it is clear that many of them feel that they have not learned as much as others have learned. What challenges will that present as young people progress to the next stages of their qualifications or, indeed, their lives?

Qualifications are a passport to the next step, and, if the qualification is based on reduced content to the extent that the person does not have the skills that the qualification should represent, that creates a challenge. In the earlier session, Tony McDaid talked about young people getting a qualification and moving on to the next step but then, because they have not consolidated their skills, struggling at the next step.

It comes back to why we said that S4 students should not be doing exams this year but should be focused on teaching and learning. One of the issues we raised in the qualifications group was that universities should be more flexible in how they accommodate school leavers coming on to courses. The whole system is focused on young people getting the relevant grades so that they can get to university and the universities can do what they do. Why do the universities not look at the fact that learning has been disrupted and decide that they should provide some consolidation work in the first two or three months, as they would for people who go to university without necessarily getting formal qualifications?

Do you think that is happening?

No, it is not happening. The response to that suggestion was, “We need to concentrate on young people getting their qualifications.”

There was a case in the papers involving a student who had not got into medicine because she had got four As and a B rather than five As. That is ridiculous. If that young person has been denied that course because she got a B instead of an A, what responsibility is the university taking? Why should the university not decide to give a good candidate some additional support initially? In an integrated education system, the universities should not be doing their own thing separately from everybody else.

I am sure that the universities would talk about the cap on student numbers that the Scottish Government put in place. Seamus Searson and Tara Lillis, do you have comments on these issues?

As Larry Flanagan said, the system needs to adjust. Assessing what the youngsters have missed is important, and teachers will be doing that, but they need support to make a difference.

Obviously, having additional teachers would help to break down the challenges that teachers have. Another obvious challenge is the multicourse teaching that is being done in the senior part of secondary schools, whereby young people studying for national 5s, highers and advanced highers are all in the same class. That means they get only a third of a lesson, if they are lucky. This year, we should have been striving to make up the work that needs to be caught up with and making sure that dedicated teachers are teaching national 5s, highers, and advanced highers separately.

At the moment, we are still in a situation in which three different courses—and they are different courses—are being taught in many subjects, particularly the ones referred to as the softer subjects, and the youngsters are not getting the dedicated time with their teacher that they deserve. We should have been trying to address that this term.

12:15  

The question is about the purpose of education in the first place. We want to move away from putting children and young people into lanes and shoving them through exams. The purpose of education is broader than that. Education should instil in an individual a love of learning that they can take with them throughout their life and career. Some of the feedback that the committee has heard today shows the stresses and strains that have been placed on fostering and embedding that love of learning within the system.

In addition to the need to look at moderation in further and higher education, which we agree with, one aspect of education that has somewhat been lost is the need for discussion of the impact on children and young people with additional support needs. Some of the feedback about 2021 said that a lot of the supports that would otherwise have been in place, such as the pupil support assistants, were diverted or missing due to Covid-related absences, which had a knock-on impact. The system needs to ensure that we look at learning in the broader sense in an inclusive way. We need to be sure that we are getting it right and that the challenges in the system are talked about not only in the broader sense but specifically in relation to pupils with additional support needs.

All of that is useful and, when we consider the design of the 2022 system, those broader impacts must be part of it. It cannot just be about the assessment model; it has to be about the reality of what teachers are facing in the classroom and the circumstances that those young people face.

My final question is about the low number of appeals. Mr Flanagan said that most pupils seem to be satisfied, but I have had representations from a significant number of pupils from across both cohorts—those who got results last year, under the algorithm, and those who got theirs this year, under the alternative certification model—and they are greatly concerned that exceptional circumstances were not accepted in their appeals. So many people faced exceptional circumstances. Should they have been included in the appeals process, and should they be included in the future? I am not too interested in additional information, because I know that provision was made for that to come through in September. That is mentioned in some of the written submissions. I am talking about the exceptional circumstances that were faced.

The EIS supported the idea that the awards should be evidence based, partly as a protection for the teacher. If there is no evidence base for a decision and a student or a parent appeals it, a professional is in a difficult situation if they just say, “It is my opinion.” There has to be an evidence base.

Even in 2020, when we had inferred progression, there was an evidence base, because the prelims were held in March and we had classwork, and the teachers inferred progression from the prelims. If the prelim was in December, you would normally expect 10 per cent improvement to the exams. People were going into schools during the Easter break to get the evidence, so there was an evidence base. The key issue is that the evidence base was not just exams. Under the ACM, not the SQA guidance, the evidence could be a class jotter, an observation that a teacher made or somebody delivering a talk. Having an evidence base is an idea that we support.

On the question of appeals, there is a challenge because the previous appeals system, which SQA operated, was evidence based. If a student could not sit their exam because of family circumstances or whatever, the school could submit alternative evidence as the basis for an appeal. However, this year’s appeals are based on a reappraisal of the existing evidence. If a school has given somebody a C rather than a B and they appeal, the school will submit the evidence it used for that judgment, and the SQA will decide whether to overrule it. That does not take account of why there is such evidence, which is the exceptional circumstances to which you allude. However, there can be a difficulty. Because somebody has had a very traumatic experience, you would be empathetic and sympathetic, but would you necessarily give them a grade that you cannot justify? There is an ethical dilemma in that.

If someone missed an exam, you could understand why alternative evidence might be looked at. However, let us say somebody got knocked down and had to spend three months in hospital and was therefore ill prepared for the exam. Could you still award what you think they might have got if they did not have the teaching and learning? There is an issue there, and we favour evidence-based approaches.

The appeals system that Larry Flanagan outlined was based on the evidence that was provided. Any chance of getting an alteration was limited because the moderation process would be against it. The problem was that teachers worried that the youngsters were not getting the evidence that they needed to the attainment levels they should have received. That was the problem for us—we wanted the teachers to be able to override the evidence.

I understand where Larry Flanagan is coming from. If you have the evidence, you cannot argue about it. However, at some point, we have to accept that teachers want to give the youngsters what they should be getting, not just what is in front of them. The ACM taught us that we need to debate the point at which a teacher’s professional judgment overrides the evidence, and we are not there yet. Teachers can override the standardised assessments in S3, but we still need to have that conversation, because teachers have gained their professional judgment through years of experience and we are not using it at the moment. We are relying on evidence because we want something to fall back on. It is a belt-and-braces approach, and we need to go beyond it. We are not there yet, but it might come in the future.

I quoted earlier the statistic from your survey that 36 per cent of teachers believed that the evidence that they collected truly demonstrated people’s attainment, which means 64 per cent did not believe that. That is a startling statistic.

We will go to Bob Doris for questions for the remaining 10 minutes.

I should put on record the fact that I am a former secondary school teacher and a former member of the EIS—in fact, the EIS might still be taking my subscriptions. I will need to check that.

All the witnesses have said that they agree that a moderation process is needed, and that that is important. The issue is to do with the extent of the moderation, the burden that we put on teaching professionals and the proportionality of the process. I think that everyone is wedded to the idea that moderation is important.

Mr Flanagan said in his submission that moderation provides teaching professionals with additional protections. Seamus Searson said that a teacher might think that a pupil was working beyond what evidence they had, but the opposite can also be true—a teacher might think that a young person is working at the level of a C grade but the young person might want to achieve a B. Unless teaching professionals can provide an evidence base for how they reach their professional judgments, I can see them getting into all kinds of difficulties. I would like Mr Flanagan to say more about the protections that moderation provides to teaching professionals.

In my constituency experience, I found that difficult conversations had to be had between teachers and young people when young people were informed what grade it was thought they were working towards, which might not have been the grade that they aspired to. There needs to be a lot more good-quality direct communication between teachers and pupils on such matters.

The key point is that teachers want to get it right for their students. Staff have made a huge commitment in investing in the quality assurance process and the estimates so that students would get the right outcomes. Every teacher knows about unconscious bias and so forth, so we routinely adopt measures to make sure that we are being as objective as possible.

By the way, Bob, there is no refund policy in the EIS—just in case you were thinking of sending me an email.

You touched on the fact that, over the course of the year, teachers give feedback to students on how they are doing. They say, “I know you want to get a B but, at the moment, your work’s indicating that you’ll get a C. Here’s what you have to do to step up if you want to improve on that.” That dialogue is not difficult—it is part and parcel of the job—but it is critical so that young people know where they are. There is no point in saying, “I think you’re capable of an A.” That does not mean that they will be given an A, unless they demonstrate the knowledge, understanding and skills that apply. There is constant dialogue.

The key point to make about protections is that parents and students must have confidence that the teacher’s final iteration of the grade is based on the dialogue across the term and the evidence that the student knows that she or he has produced. That is part of the teaching and learning process.

There have been very few appeals. I think that there have been around 4,000 appeals, which is a relatively small number given the number of presentations. I would not want us to go into an appeals system with one of our members when the evidence to support their judgment was that they thought that the candidate would do better or that they deserved an A, because it would be hard to defend that. Parents could say about their son or daughter, “I think they deserve five As and you’ve given them five Cs.” The only way such situations can be resolved is by having evidence.

I do not think that Seamus Searson and I are too far apart. A much greater range of evidence is available than just past SQA papers. That is the key point. There is a whole range of evidence available.

I can see that Seamus Searson wants to come in, and I am keen to hear from him. I will put my final question, and perhaps he could respond to both my questions, if that would be okay, convener.

Absolutely.

I want to focus on the year ahead. We know that there has been a reduction in course content to support teachers and learners in the coming year. We have heard that more details are to be provided of what alternative scenarios might look like if there is disruption to schools—that is likely to emerge in October. I want to ask our witnesses what they think should happen—God willing, this will not arise—if there is additional disruption to schools. Should more course content be taken out? Should we revert to a revised form of the alternative certification model? What would that look like? I get that teachers, learners and parents will want to have that information as soon as possible. Ross Greer mentioned that the predecessor committee was keen to get a bit of assurance on what the contingency arrangements would look like at the earliest opportunity.

Going forward, for the individual academic year that we have just commenced—I know that larger systemic changes are planned—what would you like to see happen if, unfortunately, there was additional disruption?

In response to the first question, there is a lot of information on pupils, and there is constant dialogue with pupils and parents about how they are progressing. As I said, from the day that they walk into secondary school, teachers track and monitor those youngsters. If someone is an A student throughout the first three years of secondary school, the likelihood is that they will carry on in that direction. The same is true of a youngster at a D or a C level—it is likely that they will continue to move along at that level. It is joined-up assessment that I am referring to. At the moment, it is as though it did not happen—[Inaudible.] That is something that we can be doing.

That ties into the next part of the question. If we already have that information and an understanding of where the youngster is—[Inaudible.]—understood that as they have gone through the system, it should not be a hard problem at the end of that to say—[Inaudible.]—in the past six months, “This is where we think you are and, therefore, that’s where you should be.” We should be ready to use that information should we move to a situation in which the exam system is not allowed to continue.

12:30  

I do not want to revert to an ACM just for the sake of it, but I think that the evidence that the youngsters are providing at the moment through continued teaching and learning should be continued right up until the finish. If we break from that to start gathering assessment materials, we will stop teaching and learning. We do not need that, and we cannot afford it. We should be looking at what pupils are doing at the moment and keeping that as evidence. That is the evidence—we do not need to create new evidence, which is what the ACM did at the end of last session. Although that might not have been what was intended, that is what happened.

Schools are already starting to run additional exams in the event that the exams do not take place next summer. I would argue that we need to build on the evidence that we have already got and not disrupt teaching and learning for this year, because young people have lost enough already. Cutting back on the course is not of any benefit. We need to make sure that young people cover the full area of the course as best they can in what might be a disrupted experience over the next number of months.

In looking forward to the year ahead, we are looking for reassurances that we will not face a repeat of the confusion and chaos that we had last year. We are looking for protections in the system to ensure that the workload burdens that were associated with the alternative certification model are not replicated, whether those burdens were associated with national guidance or local implementation models.

If we look backwards at the 2021 model, our position was that some form of the 2020 arrangements should have been adopted as the only practical solution to addressing the inherent difficulties with the 2021 ACM model as it ran. That is not in any way to say that that is a best-practice model; there was no off-the-shelf model. The system that was used in 2020 had its own difficulties, no matter which option was undertaken; they were all suboptimal.

In our discussions today, it is really important that we distinguish between moderation and assessment under normal circumstances and moderation and assessment while we are simultaneously contending with Covid and school closures and all the disruption that those involve. As part of that process, the key consideration is putting teachers at the heart of those discussions. The critical message that we want the committee to remember is that, if we are to ensure that we do not have a repeat of that disruption and chaos, the views and the voice of teachers must be a core consideration.

The course adjustments are marginal. They largely involve the removal of some assessment areas; they do not necessarily involve the removal of course content. There is a limit to how much can be removed before the study is invalidated. We absolutely cannot have a dual assessment approach—we have already discussed this with the SQA—whereby schools start banking materials just in case, but at the same time prepare for prelims and an exam diet. That creates a workload burden for staff and an assessment burden for students, all of which combines to detract from teaching and learning.

After some discussion, the SQA has finally put out the information that the contingency will be based on naturally produced evidence. We are content with that general phraseology, but the issue comes back to the question, what is valid evidence? In our view, at this stage in the year, naturally produced evidence will not be particularly useful for summative purposes, because students have only started the teaching and learning process.

This year’s prelim exams could be quite significant as regards any fallback that might be required post-Christmas. In 2020, a lot of students said, “If we’d known the prelims were going to be so important, we would’ve worked harder.” Of course they said that. They should know that the prelims will be important. Prelims do not test the full year; they test only what has been taught up to Christmas. All of that must combine.

We have also said clearly that, just because it is not possible to have all the students in the big hall, that does not mean that we cannot have an exam system. Even last year, we said to the SQA, “If you’re pushing for the use of past papers and what are, in effect, exams, schools can administer that in class and you can mark it all.” That would have been an exam diet, albeit that it would not have been staged in the way that we have staged exam diets previously.

All those things must be kept under consideration. I echo Tara Lillis’s point. One of the big concerns in schools is that we do not create a workload burden that will drain energy from teaching and learning. Obviously, the lockdown was precipitated by the pandemic, but we could have had decisions around the ACM much earlier. We do not want to end up with a last-minute decision. I do not know why we had to wait till August for a decision about whether there would be an exam diet this year, because it was clear what the thinking was in mid-summer, and schools could have come back to a clear decision on that. Early decision making is critical.

Thank you, Bob. Kaukab Stewart has a final question.

Since we are coming to the end of the session, I will do a quick summary as well.

I want to get a better perspective on the statistic that the convener referred to—the one about 36 per cent of teachers. I have just looked up the membership figures. Am I correct in thinking that the EIS represents about 55,000 members? I think that that is about right. I am not sure about the figure for the NASUWT and the SSTA, but I think that it is around 7,000. Therefore, the 36 per cent figure would be 36 per cent of the SSTA’s membership. Is that correct? Seamus is nodding. I just wanted to get a bit of perspective on that.

I think that what I am hearing marries quite a lot with what we have heard from other agencies, universities and a broad spectrum of young people. We have got pupils, practitioners and policy; politics is right at the bottom of that. The Organisation for Economic Co-operation and Development said that, too. Moving forward, I feel quite optimistic, because a lot of your evidence triangulates with other evidence that we are hearing. It appears that there is consistency about how to move forward; there is consistency on timescales and the need for clear consultation and communication.

Is there anything that I have missed that you would like to add with regard to how we can move forward as a committee?

I take Tara Lillis’s and Seamus Searson’s points about CERG and the NQ groups. We would welcome all trade unions being involved in those bodies. There is a significant collaborative process in place in Scotland. I spend most of my time arguing with the SQA, but collaboration does not mean constant agreement. The fact that we are having a dialogue is important, because I know from colleagues in the National Education Union, and Tara Lillis will know this from her colleagues in England, that, in England, the teacher unions are outside the discussion full stop. I think that the collaboration that we have in Scotland works to the benefit of Scottish education, even when the discussions are difficult.

Building on that would be a good way forward. When it comes to the work of Stobart and the OECD review, it is important that all the professional voices are part of the discussion because, ultimately, we need the professional associations to carry their memberships with them on the changes. If we do not carry the teachers with us on the changes, we will end up with another review in 10 years’ time looking at why we got it wrong again.

Thank you, Kaukab.

It was the teacher in you that required you to sum up at the end of the lesson, Kaukab. [Laughter.]

I thank Larry Flanagan, Seamus Searson and Tara Lillis, who have been our panellists for the past hour and a half. We really appreciate your giving us your time. We thank you for not just your oral evidence but the written evidence that you submitted in advance of the meeting, for which we are very grateful.

We will conclude our evidence taking on the alternative certification model next week, when we will take evidence from the Scottish Qualifications Authority.

The public part of today’s meeting is now at an end. We will consider our final two agenda items in private.

12:40 Meeting continued in private until 12:57.