Note: This content was originally hosted at nobbs4ccs.com during the 2022 CCS school board election.
Academic Decline - Part 2
Brake, Brown & Sharp may have realized that their original claims about CCS’s academic decline were too extreme for Carmel voters to fall for them. I write that because it seems they:
have shifted away from spreading misinformation that incorrectly compares recent data to data from 2017 and earlier years.
have stopped leaving out the improvements seen in 2022.
are now honing in on 2019 – 2022.
They’ve also shifted away from looking at our district as a whole, and have moved on to making claims about specific schools.
I will admit that I’m pretty surprised to see that they’re still blaming CCS schools for the academic impact of the pandemic. I think the vast majority of voters in our community are smart enough to know better, but it seems the BBS campaign thinks otherwise.
Regarding the claims about academic declines at specific CCS schools, the truly magnificent response below was sent to me by another Carmel parent who has requested that I not share their name. Their collegiate education was in mathematics and their career has afforded them extensive experience in data analysis. I have downloaded the disaggregated school data they reference and verified that the numbers and analysis are correct. (If you’d like to download data sets and do your own verification, they’re on this page. You’ll want to pull the school results spreadsheets that are student group disaggregated).
Note that since we’ve already covered State of Indiana vs Carmel comparisons here, the comparisons on this page focus specifically on Carmel schools’ iLEARN data for the years 2021 and 2022.
At the end of the page, I’ll jump back in with some closing comments.
The Claim
That disaggregated school data shows specific elementary and middle schools experienced a decline of combined ELA & Math proficiency from 2019-2022.
The Short Response
Comparing the 2019-2022 year-over-year performance of individual Carmel schools is uninformed at best, and lacks a critical understanding of the schools, redistricting and what iLEARN is measuring.
There are three reasons that our schools’ year-over-year scores are an invalid way to measure results:
Incomplete 2019 data. On each table in the 2021 data reports, the State of Indiana specifically notes: ‘Data is not comparable to previous iLEARN Spring 2019 spreadsheets that reflected only those students enrolled at least 162 days.’ That is, the state specifically excluded a subset of students in 2019 and explicitly says not to compare the data from that year to data from 2021 and beyond.
The pandemic. This was a global phenomenon that negatively affected schools and students around the world.
Redistricting. Comparing 2021 vs 2022 on an individual school basis is inaccurate, as one elementary school was closed, a new one opened, and redistricting resulted in different sets of students attending each school. One would expect school board candidates to realize that this happened and the effects it had.
The Long Response
While individual year-over-year school comparisons are flawed, improvements or declines can be evaluated by using iLEARN’s disaggregated data. Note that iLEARN testing is administered to Indiana school students in grades 3 – 8, and the state provides school-specific proficiency scores for the following subgroups of students:
Paid Meal vs Free/Reduced-Price Meal
General Education vs Special Education
Non-English Language Learners vs English Language Learners
Ethnicity (categories: American Indian, Asian, Black, Hispanic, Multiracial, Native Hawaiian or Other Pacific Islander, and White)
Let’s start by taking a look at districtwide data pertaining to these subgroups. As shown in the graphs below, CCS’s iLEARN results reveal proficiency gains across ALL disaggregated subgroups.
Additionally, the graph including ethnicity demonstrates that some of the largest improvements are in subgroups that have historically had lower proficiency rates, and that DEI initiatives are intended to help. While this level of data cannot be used to definitively state that DEI initiatives are improving test scores, it does support that hypothesis.
On these graphs, the gray columns are 2021 proficiency rates for each subgroup, the blue columns are 2022 proficiency rates for the subgroups, and the red dashes are how much proficiency improved for each subgroup from 2021 to 2022. Note that the ethnic categories of American Indian and Native Hawaiian or Other Pacific Islander are not included as Indiana withholds data on subgroups containing less than 10 students.
The improvements are great to see, but relevant to the question at hand, this data also lays the groundwork for understanding why redistricting invalidates comparing 2021 results to 2022 results for individual Carmel schools.
Take notice of the proficiency gaps between subgroups in the first chart. Indiana tracks and reports that data because the state has identified three distinct subgroups that historically experience dramatically lower proficiency rates:
Students receiving free or reduced-price meals (FRL)
Students in special education (SE)
Students learning the English language (ELL)
How much lower is proficiency among these subgroups?
FRL proficiency is 16.6% statewide and 30.9% for CCS
SE proficiency is 8.9% statewide and 23.4% for CCS
ELL proficiency is 8.5% statewide and 22.1% for CCS
As previously stated, there was significant redistricting within CCS schools that occurred between 2021 and 2022 iLEARN testing. In fact, there were three schools in particular that absorbed significant amounts of students. They were the only Carmel schools to increase the number of iLEARN tests administered by over 10% from 2021 - 2022.
Carmel Elementary: 40.2% more iLEARN tests administered
Woodbrook Elementary: 20.8% more iLEARN tests administered
Mohawk Trails Elementary: 14.2% more iLEARN tests administered
That redistricting resulted in large shifts of where students in the underperforming subgroups attend elementary school. Below are the group shifts within each school from 2021 to 2022. It is instantly apparent that a few schools (Carmel Elementary, Mohawk Trails Elementary & Woodbrook Elementary) received the largest influxes of students in the underperforming subgroups.
Note that Orchard Park Elementary and Clay Center Elementary are included, despite the fact that the former closed and the latter opened between test years.
We can also chart the changes from 2021 to 2022 to identify the schools that were most affected by redistricting. The charts below show the change in subgroup distribution for each school. It is important to note that certain schools saw all three subgroups increasing (CE) or large increases in two subgroups (MTE & WBE). The large changes for these three schools are highlighted in red in the graph.
It’s very easy to see that the Mohawk Trails, Woodbrook and Carmel elementary schools received the largest influx of students in the low-performing subgroups. As would be expected, they were the three schools to experience proficiency drops from 2021 – 2022.
For each of these schools, we can also look at actual numbers to see how redistricting and shifting demographics increased the number of overall students taking iLEARN, as well as the increases to each subcategory.
Mohawk Trails Elementary, from 2021 to 2022
43 more iLEARN tests administered overall
19 more iLEARN tests administered to FRL students
3 more iLEARN tests administered to SE students
27 more iLEARN tests administered to ELL students
Woodbrook Elementary, from 2021 to 2022
49 more iLEARN tests administered overall
21 more iLEARN tests administered to FRL students
17 more iLEARN tests administered to SE students
1 more iLEARN tests administered to ELL students
Carmel Elementary, from 2021 to 2022
85 more iLEARN tests administered overall
21 more iLEARN tests administered to FRL students
15 more iLEARN tests administered to SE students
15 more iLEARN tests administered to ELL students
It is entirely unsurprising that any school’s overall proficiency rate would decline when redistricting assigns it with 14% - 40% more students and a large majority of them are in low-performing subgroups.
And About CCS’s Districtwide Proficiency…
It should also be noted that all of the above data adds some important context to claims that 41% of CCS students lack proficiency according to Indiana standards.
It is helpful to start by asking whether a full 100% proficiency should ever be expected. It may be unreasonable to expect students in special education or who are currently learning English to be fully proficient. These groups respectively account for approximately 11% and 5% of CCS elementary and middle school students.
Additionally, Indiana does not simply provide numbers for proficient vs non-proficient. For both ELA and Math, the state further breaks the data down into four categories:
Below proficiency
Approaching proficiency
At proficiency
Above proficiency
When that data for CCS is charted, it appears as follows:
Again, CCS elementary and middle schools include about 11% special education students and 5% of English language learning students. It seems reasonable to assume that those groups are disproportionately represented at the ‘below proficiency’ level for both ELA and math.
There’s another way of saying the same thing. Based on available data, it is reasonable to assume that among students not in special education or learning the English language, approximately 75% are at or above proficiency and the majority those remaining are approaching proficiency.
Additionally, the graph below shows CCS elementary and middle schools’ distributional changes in ELA and Math from 2021 to 2022 for each proficiency category.
This highlights that each category for both ELA and Math declined in 2022, with the exception of ‘above proficiency.’ This demonstrates that each proficiency level of students improved.
Some Closing Thoughts
Hi, Jim here again. Wow. I sincerely wish I could take credit for the thorough analysis provided above. The parent who provided it truly went above and beyond in helping us to even better understand how the data contradicts BBS’s narrative that CCS schools are floundering. As I pointed out in previous analysis and as has been recognized by no shortage of state and national metrics and rankings, CCS continues to achieve excellence when it comes to academic performance.
Not that I expect the BBS campaign to acknowledge this.
What’s more likely is that they’ll call the above facts ‘massaging the data.’ By their words and actions, they seem to consider it transparent to repeatedly offer various year-over-year testing comparisons that the State of Indiana explicitly says are invalid. Their statements pretend our school district is somehow unique for having to overcome negative effects from the pandemic. They call out specific schools for proficiency decreases, demonstrating that they are either ignorant of the effects of redistricting or are hoping that Carmel voters aren’t paying too close of attention.
The BBS campaign claims they’re ‘dedicated to asking complex questions,’ but again and again they show no interest in complex answers, or even simple answers that don’t align with the doom and gloom story that they’ve concocted and are trying to sell to voters. They seem to prefer misinformation and buzzword-based scare tactics over any kind of objective analysis of our school system.
Regarding CCS academics and the answers provided in this and my previous articles, I fully expect the BBS campaign to move the goalposts again and shift to making some new claim in the final stretch of the election. If and when that happens, I will do my best to investigate and write about it here.
Thanks to every voter who cares enough to be reading this site and a super special thanks to the parent who contributed the bulk of this article.
Comments