成人抖阴

Help fund stories like this.

When the nation鈥檚 most important test dropped in late October, the news was abysmal: Scores were among the worst in its history.

But amid the carnage, one feel-good story emerged. Los Angeles, the nation鈥檚 second-largest school district, appeared to have accomplished a reading miracle, with eighth-grade scores jumping an incredible nine points.

Peggy Carr, the nation鈥檚 top testing official, called the district鈥檚 performance one of the few 鈥渂right spots鈥� on the National Assessment of Educational Progress, otherwise known as 鈥淭he Nation鈥檚 Report Card.鈥�

The media took note. and trumpeted the development, one that Alberto Carvalho, the district鈥檚 superintendent, said 鈥渂odes very well for our L.A., and is really a testament to our strategy.鈥�

Almost as soon as the news sank in, however, a host of naysayers emerged to pop L.A.鈥檚 balloon. Among wonks on edu-Twitter, the skeptic鈥檚 case went something like this:

• Nine-point jumps on NAEP are extremely rare.
• It happened in a huge urban district and during an educationally punishing pandemic, when most school systems鈥� scores tanked or, at best, stayed flat.
• NAEP is typically harder than most state assessments. Yet L.A. scores on the national test eclipsed those on California鈥檚 Smarter Balanced assessment, where the district鈥檚 eighth-grade readers inched forward a barely perceptible half a percent. 

Speaking for many researchers, Robin Lake, director of the Center on Reinventing Public Education, took to Twitter to say the results

I really cannot make sense of the LAUSD NAEP scores given ALL other data I have seen. I want to believe they did something right and celebrate the successes, but it just does not sit right. Possible sampling issue?

— Robin Lake (@RbnLake)

At first, they didn鈥檛 sit right with the test鈥檚 administrators either. That鈥檚 why prior to NAEP鈥檚 release, the National Center for Education Statistics conducted extensive internal reviews and called on outside experts to crunch the numbers to understand what they were seeing.

鈥淭he L.A. results 鈥� certainly caught our attention, because they stood out as different from the nation as a whole and the other鈥� school systems that take NAEP, said Daniel McGrath, branch chief of the assessment division at NCES. Twenty-six districts participated in the Trial Urban District Assessment, which allows for comparisons across the nation鈥檚 large urban districts. 

The researchers 鈥渢urned over many additional rocks鈥� to back up a result they knew would raise eyebrows, he said. The American Institutes for Research conducted a separate analysis of the 2022 scores. 

The results offer the kind of frustrating nuance researchers are known for, confirming neither the unmitigated triumph of the district鈥檚 cheerleaders nor the worst-case scenarios of the skeptics.

After performing a series of statistical checks and balances, McGrath said L.A.鈥檚 schools saw a 鈥済enuine increase in student performance.鈥� Because of all the interest, NCES plans to add some technical notes to its explaining factors that affected this year鈥檚 results in Los Angeles.

A 鈥榩iece of the puzzle鈥� 

L.A. district staff participated in NCES鈥檚 Oct. 11-13 pre-release workshop to discuss the results. And on Oct. 19, the NCES team held a separate meeting on Zoom with Carvalho to answer questions about the extra steps it took to verify the data. 

The researchers explained that one factor in the district鈥檚 performance was the sample of schools in which students were tested. , the last time the nation鈥檚 students took NAEP, L.A.鈥檚 high-performing, affiliated charter schools weren鈥檛 included. This year, they were.

Including those scores 鈥� from five of the 60 schools in the sample 鈥� accounted for roughly two points in the nine-point climb in eighth-grade reading, McGrath said. That鈥檚 鈥渙ne piece of the puzzle.鈥�

Affiliated charters are district schools, but operate with some flexibility. They perform significantly higher than district schools overall, and this year鈥檚 state test scores show just how much. Forty-two percent of the district鈥檚 eighth graders met or exceeded state standards in English, compared to 66% in the district鈥檚 , according to 成人抖阴鈥檚 review of .

Since the scores鈥� release, Carvalho the district鈥檚 efforts to provide summer school, tutoring and internet connections during remote learning. 鈥淗ow about those NAEP scores!鈥� he said during public comments Friday. In addition to the nine-point increase in reading for eighth graders, he touted the district鈥檚 fourth-grade reading scores, which went up two points compared to an average three-point decline in other districts. And he offered a pointed rebuke to those he described as 鈥渋nteresting fellows鈥� on social media who 鈥渨ork at places like Harvard.鈥�

鈥淭hey do nothing but study what we do and write about it,鈥� he said. 鈥淭hat data is accurate, and to assert anything other than that is to diminish the impact that our teachers, our principals, our support staff and 鈥� our students produced.鈥�   

Carvalho doesn鈥檛 need to be reminded of NAEP鈥檚 reputation as the 鈥済old standard鈥� in testing. He鈥檚 a of the National Assessment Governing Board, which sets policy for the test.

Few things more frustrating than to hear from those on the political fringe, who often breathe the rarefied air of the ivory tower, who rarely if ever spent a day in a classroom, dealt with the issues facing our students, but so loudly opine on the work & results of those who do.

— Alberto M. Carvalho (@LAUSDSup)

While a nine-point increase is rare, leaps in that range are not unheard-of, according to examples provided by NCES. Boston Public Schools, for instance, saw a 10-point increase in fourth-grade math between 2003 to 2005. And eighth-grade reading scores in the District of Columbia schools jumped eight points between 2011 and 2013. 

Demographic shifts

Some experts noted that Los Angeles has also experienced major demographic shifts in recent years that would affect student performance. Immigration in Los Angeles County, and Carvalho has often noted that officials can鈥檛 account for thousands of . 

Between the 2019-20 and 2021-22 school years, district enrollment fell by a staggering 10%. This factor deserves more attention, said Thomas Dee, a Stanford University economist and education professor. 

Does anyone else feel like LA's NAEP numbers are too good to be true? The district lost so many students — did that affect the composition of test-takers? None of their interventions seem special compared to other big cities.

— Anya Kamenetz (@anya1anya)

鈥淭he composition of that enrollment changed by a variety of student traits and the impact of such changes on LAUSD’s recent test scores is far from clear,鈥� said Dee, who has been tracking pandemic enrollment trends. 

NCES officials thought of this, too.

It turns out that demographic changes didn鈥檛 affect the overall increases. In fact, students in each racial and ethnic group made gains between 2019 and 2022. The district鈥檚 Hispanic student enrollment fell during that time period while enrollment of white students increased. But the addition of the affiliated charters effectively canceled out the impact of that population shift, McGrath said.

NCES also considered one group of students who didn鈥檛 participate in this year鈥檚 NAEP 鈥� those who were still learning remotely in spring 2022. Last year, the district鈥檚 virtual school, City of Angels, included 76 eighth graders. If they were all lower-performing students and if they had been included in the sample, it鈥檚 possible the district鈥檚 scores would have been lower. But to drag it down even one point, those students would have had to score 20 points below the district鈥檚 average, McGrath said.

The NAEP results also didn鈥檛 jibe with typical comparisons to state tests. Usually, districts on state assessments than on NAEP. 

In this case, the overall percentage of Los Angeles students meeting or exceeding standards on the Smarter Balanced test declined since 2019. In eighth-grade English language arts, however, there was a miniscule increase 鈥� from 41.14% to 41.74%.

The discrepancy could be chalked up to the fact that the state test includes a writing section and NAEP does not, said Andrew Ho, an assessment expert at Harvard and part of the research team behind a new that uses NAEP and state test data to calculate learning loss. 

In of why these discrepancies occur, the team notes that it鈥檚 also possible for the students who took NAEP to include more high performers than the larger state sample. 

鈥楽till a full two years behind鈥�

The district鈥檚 results confused some education advocates in Los Angeles as well. They expressed doubt that the strategies Carvalho points to were robust enough to spark such an increase.

鈥淣one of it was happening at the scale any of us were hoping it would,鈥� said Hannah Gravette, regional vice president for Los Angeles with Innovate Public Schools, an advocacy organization that works with families in low-income communities. 

The district is offering and homework help, for example, but she said the type of in-person, consistent tutoring known to is still in the early stages.

But the superintendent who was in charge when the pandemic hit said he is not surprised by L.A.鈥檚 testing windfall. 

鈥淵ou can鈥檛 point to one thing. We did everything possible, and I don鈥檛 know of any other school district in the country that did,鈥� said Austin Beutner, listing factors such as the district鈥檚 extensive COVID testing program, the 140 million it provided to families and delivery of . 

鈥淲e sent trucks around the country to Apple stores to take inventory out of the back because they were closing,鈥� he said. And hiring aides and reading specialists to run breakout rooms over Zoom allowed students to get more individualized attention than some would have received even in a traditional classroom, he added.

Ray Hart, executive director of the Council of Great City Schools 鈥� the network of 78 urban school systems that helped launch the district assessment program 鈥� said looking further back than 2019 helps put the district鈥檚 performance in context.

The affiliated charters were part of that 2017 NAEP sample, but not in 2019, when the NAEP score in eighth-grade reading dropped from 254 to 248. McGrath said that just like charters didn鈥檛 account for the full increase in 2022, their absence wasn鈥檛 the only reason for the decline in 2019. Eighth-grade reading scores fell nationally that year, not just in Los Angeles.

Comparing apples to apples 鈥� 2017 to 2022 鈥� the difference in eighth grade reading is much smaller, a three-point increase from 254 to 257 on NAEP. This relatively flat performance is more in line with that of many districts this year, which both Hart and McGrath view as positive, considering the pandemic鈥檚 crushing impact nationally. 

鈥淭he interventions during the pandemic and after are starting to show signs that they are paying off,鈥� Hart said.

Regardless of the reason for the district鈥檚 bump, some advocates remain unimpressed.

鈥淚t is really not an improvement, but a return to pre-pandemic performance, which means students are still a full two years behind,鈥� said Alicia Montgomery, CEO of the Center for Powerful Public Schools, a nonprofit supporting school improvement efforts in the district. 

鈥淭hree points is nothing over a five-year period,鈥� she said. 鈥淚f there was no pandemic, students would have gained far more.鈥�

Help fund stories like this.