Rafand - Red Apple Foresight
Rafand - Red Apple Foresight and Development Enterprise

How Schools’ Preparedness Related to Their Remote Instruction During COVID-19

The emergence of COVID-19 in the United States in spring 2020 forced nearly all U.S. schools to transition rapidly to remote learning.

However, a minority of U.S. public schools were prepared for a crisis on the level of COVID-19. Using responses to the American Educator Panels, researchers investigate how schools’ pre-pandemic planning translated into remote learning practices and principals’ confidence in student achievement during the COVID-19 pandemic.

Principals detailed the infrastructure preparations that their schools had made before the COVID-19 pandemic began.

Specifically, principals were asked whether, before the pandemic started, their schools had undertaken the following five practices:

  • providing devices (e.g., laptops, tablets) to, at a minimum, those students who need them
  • training teachers on delivering online instruction
  • using a learning management system
  • providing fully online or blended learning courses
  • establishing plans to deliver instruction during a prolonged school closure.

Principals also noted whether their teachers graded students’ work and whether they felt concerns about their schools’ provision of equitable instruction during the pandemic. They also gave predictions of student achievement for various student subgroups in the coming school year.

Principals whose schools were more prepared were also more comfortable continuing to assess student learning with letter grades during the COVID-19 pandemic and had fewer concerns about providing equitable instruction. It will be important to continue to document schools’ instructional practices to fully understand the conditions that are needed to ensure equitable access to high-quality instruction.

Key Findings

Most schools had some preparation indicators in place, but most did not have all indicators

  • Most principals (84 percent) reported that their school had at least one preparedness indicator in place pre-pandemic — but very few principals (7 percent) reported having all five.
  • More secondary schools than elementary schools had indicators of preparedness.
  • The level of school poverty was not correlated with the number of pre–COVID-19 preparedness indicators that schools had at the outset of COVID-19.

Most schools did not have plans in place for a prolonged school closure

  • Of the five pre-pandemic preparedness indicators, schools were most likely to have provided students with devices; they were least likely to have plans for a prolonged school closure.

The more preparedness indicators a school has, the more confident principals are in providing instruction

  • Principals in more-prepared schools (as measured by the number of preparedness indicators schools had in place pre-pandemic) were more likely to assign letter grades to students during the pandemic, even after controlling for differences in school characteristics.
  • Principals in more-prepared schools had less concern about failing to provide equitable instruction to all students.
  • Principals in more-prepared schools were less likely to predict lower future achievement for students from low-income families and students experiencing homelessness.

BEGIN

The emergence of coronavirus disease 2019 (COVID-19) in the United States in spring 2020 forced nearly all U.S. schools to transition rapidly to remote learning. Unsurprisingly, schools were unequally prepared to meet this challenge. Findings from the RAND Corporation’s nationally representative surveys of K–12 grade public school teachers and principals, as well as information from school websites collected in spring 2020, highlight substantial disparities in schools’ curriculum coverage, access to technology, and teacher training on remote learning topics (Center on Reinventing Public Education, 2020; Hamilton, Kaufman, and Diliberti, 2020; Harris et al., 2020).

A minority of U.S. public schools were prepared for a crisis on the level of COVID-19. During the 2017–2018 school year, only 46 percent of U.S. public schools had a written plan for dealing with a pandemic (Kemp, 2020). Public information was lacking as well; in 2016, only 38 states had a publicly available school health emergency plan (Uscher-Pines et al., 2018). In 2017, RAND researchers asked school and health officials how their schools could be prepared to continue teaching during prolonged building closures. Interviewees indicated that to provide quality instruction, they needed sufficient online infrastructure, including learning management systems (LMSs), home internet access for students, and teachers and families trained in using online instruction before the onset of a crisis. Interviewees also said that it would take substantial lead time for schools to shift to online instruction for the first time (Schwartz et al., 2020). (Anecdotal evidence shows that schools that used online instruction before the COVID-19 pandemic were better positioned to switch to remote learning once on-the-ground instruction stopped [Eroh, 2020].)

How did schools’ pre-pandemic preparation affect their transitions to remote learning? In this Data Note, we present descriptive evidence about how schools’ pre-pandemic planning translated into remote learning practices and principals’ confidence in student achievement during the COVID-19 pandemic. Our findings are based on a survey that we administered to a nationally representative sample of 957 public school principals via the RAND American Educator Panels (AEP).

In this survey, we asked principals about the infrastructure preparations that their schools had made before the COVID-19 pandemic (see Table 1). Specifically, we asked principals whether, before the pandemic started, their schools had undertaken the following five practices:

  1. providing devices (e.g., laptops, tablets) to at least those students who need them
  2. training teachers on delivering online instruction
  3. using an LMS
  4. providing fully online or blended learning courses
  5. establishing plans to deliver instruction during a prolonged school closure.

School Preparedness Levels Before the COVID-19 Pandemic

We also asked principals about aspects of their schools’ remote instruction during the COVID-19 pandemic, including whether teachers graded students’ work and whether principals had concerns about their schools’ equitable provision of instruction. Principals also predicted how achievement for various student subgroups would differ in the coming school year compared with fall 2019.

We then investigated the relationship between these preparedness indicators and COVID-19 remote instruction practices, as well as the relationship between preparedness indicators and principals’ predictions of student achievement. In our statistical models investigating these relationships, we controlled for school demographics, such as school grade level and urbanicity, that our previous research has shown were related to differences in COVID-19 instruction (Hamilton, Kaufman, and Diliberti, 2020).

Key Findings

Of the five pre-pandemic preparedness indicators, schools were most likely to have provided devices for students; they were least likely to have plans for a prolonged school closure. Yet having (or not having) a plan for prolonged school closure was the indicator that was most predictive of principals’ expectations about student achievement in fall 2020 (relative to fall 2019) for various student subgroups.

Most principals (84 percent) reported that their schools had at least one preparedness indicator in place pre-pandemic—but very few principals (7 percent) reported their schools had all five.

More secondary schools than elementary schools had indicators of preparedness. Only 6 percent of secondary school principals said their schools had none of the five indicators in place before COVID-19 struck, compared with 24 percent of elementary school principals.

The level of school poverty was not correlated with the number of pre–COVID-19 preparedness indicators schools had at the outset of COVID-19.

Principals in more-prepared schools (as measured by the number of preparedness indicators that schools had in place pre-pandemic) were more likely to assign letter grades to students during the pandemic, even after controlling for differences in school characteristics. As shown in Figure 1, schools with all five indicators in place pre-pandemic were 20 percentage points more likely to assign letter grades during the pandemic than schools with zero indicators in place pre-pandemic.

Principals in more-prepared schools were less concerned about failing to provide equitable instruction to all students. Some 44 percent of principals in schools that had zero preparedness indicators in place pre-pandemic reported that concern about providing equitable instruction to all students was a major limitation on the amounts or types of distance learning material that they provided. After controlling for differences in school characteristics, schools with three and four preparedness indicators were 13 and 16 percentage points, respectively, more likely to indicate that this concern was not a major limitation than schools with zero indicators in place (see Figure 2). Whether schools provided devices for students was the most predictive preparedness indicator of whether principals were concerned about providing equitable instruction.

More-Prepared Schools Were More Likely to Assign Letter Grades During the COVID-19 Pandemic

Probability of assigning students letter grades during the pandemic (relative to principals reporting zero indicators)
FIGURE 1: NOTE: This figure presents the difference in likelihood of assigning students letter grades during the pandemic relative to principals in schools reporting zero indicators in place. * Principals in schools with this number of preparedness indicators in place were significantly more likely to have their teachers assign letter grades than principals in schools with zero indicators in place (p < 0.05).

These results indicate that schools that were more prepared to deliver online instruction were also more comfortable continuing to assess student learning with letter grades during the COVID-19 pandemic and had fewer concerns about equitable instruction.

It is important to remember that, although principals’ perceptions are useful in understanding the relationships between preparedness and response, our findings are descriptive in nature and do not establish causal evidence. Our analyses specifically cannot account for unmeasured factors, such as school resource levels, that are likely related to both pre–COVID-19 preparedness and responses to the pandemic. This analysis might simply suggest that principals who were better prepared for online instruction during COVID-19 tend to be more optimistic about online instruction and student learning than other principals.

Of the five preparedness indicators included in this analysis, no single indicator of emergency preparedness fully predicted schools’ abilities to deliver remote instruction. Instead, the combination of these indicators might be most important to school success—a finding which is consistent with the feedback that we received from school and health experts in 2017. As many schools begin the 2020–2021 school year with partially or fully remote instruction, it will be important to continue to document schools’ instructional practices to fully understand the conditions that are needed to ensure equitable access to high-quality instruction. Only then will we understand what is necessary to support student learning during prolonged school closures.

Principals in More-Prepared Schools Have Less Concern About Their Ability to Provide Equitable Instruction to All Students

Probability of principals indicating that providing equitable instruction was not a major limitation during the COVID-19 pandemic (relative to principals reporting zero indicators)
NOTE: This figure presents the difference in likelihood of principals indicating that providing equitable instruction was not a major limitation during the pandemic relative to principals in schools reporting zero indicators in place. Principals in schools with this number of preparedness indicators in place were significantly more likely to say that concerns about providing equitable instruction were not a major limitation than principals in schools with zero indicators in place (p < 0.05).

Endnotes

Other important preparedness indicators—such as the percentage of students who have home internet access—exist in addition to our five preparedness indicators. In our survey, only 55 percent of principals said that 90 percent or more of their students had access to the internet at home during the pandemic. We do not have data on how many of these students had access to the internet at home before the pandemic began.

In our analysis of the relationship between schools’ preparedness and principals’ predictions of achievement in fall 2020, we generally found a consistent pattern of an inverse relationship across various student subgroups; fewer principals in schools with more preparedness indicators said that they anticipated lower or much lower student achievement in fall 2020. Although not all of these relationships were significant after controlling for school characteristics, they provide suggestive evidence that principals in more-prepared schools had fewer reservations about student achievement in the coming school year.