Note: This content was originally hosted at nobbs4ccs.com during the 2022 CCS school board election.
Academic Decline - Part 3
As I predicted in the conclusion to my second look at the Brake, Brown and Sharp campaign’s misinformation about Carmel schools’ academic performance, they’ve released a new round of claims. I had been worried that they would pull in some new data that I was unfamiliar with, and that it might be difficult to respond quickly. Instead, they opted to specifically come after me and my previous analysis. Fortunately, those are two topics with which I am incredibly familiar, so let’s dive in.
Note: BBS’s response to my previous analysis is so full of obvious errors that I will be quite surprised if they don’t either remove it or significantly edit it after they read this page. They’ve repeatedly used that tactic when they’ve received fact-based pushback from parents. As such, I’ve created a pdf of the page I’m responding to as it currently exists on their website. I will include a link to download it at the end of this page for those who would like to compare it to what I write here.
The Claim
That Indiana’s guidance on comparing year-over-year data should not be followed.
The Response
I’m not sure why the BBS campaign is addressing me here. Clearly their issue is with the State of Indiana, as the state issues the data and clearly states on 2021 iLEARN results:
Data is not comparable to previous ILEARN Spring 2019 spreadsheets that reflected only those students enrolled at least 162 days.
If the BBS campaign feels CCS should be exempted from this guidance, they should be explaining this to the state, not me.
I don’t know the scope of the impact of the exclusion in 2019 data.
I don’t know why it’s significant enough for Indiana to warn against comparing it to data from 2021.
I do know that if someone:
a) gives you 2021 data and says it's measured differently than and cannot be compared to 2019 data
AND
b) gives you 2022 data and it’s measured the same as 2021 data
THEN
c) it obviously follows that the 2019 data is measured differently than the 2022 data.
I also know that the BBS campaign seems to absolutely hate following any of the state’s guidance on how to review and analyze the data it gathers and reports.
Additionally, in their presentation of two sets of raw numbers that they encourage people to eyeball, they demonstrate yet again their lack of knowledge of the school system they want to run. Enrollment is measured in the fall and iLEARN is administered in the spring.
So, for example, if you want to compare enrollment and iLEARN data for the 2019-2020 school year, you use 2019 enrollment numbers and 2020 iLEARN results. The way they’ve tabled the numbers they present, they’re aligning each year’s iLEARN scores with enrollment of the following school year. I don’t believe that makes their claim any more or less wrong, but I think it’s worth noting their consistent misunderstanding or misapplication of data.
The Claim(s)
That the pandemic can’t be proven to have had a broad negative impact on learning.
That the number of COVID cases in a community is somehow more relevant to student outcomes than schools having to unexpectedly manage a mix of virtual, in-person and hybrid classes.
The Response
Let’s start with a few facts:
Due to the pandemic, CCS closed schools on March 16, 2020 and remained wholly virtual for the remainder of the 2019-2020 school year.
For the 2020-2021 school year, CCS was forced to provide a mix of in-person, virtual and hybrid instruction, subject to parental choice as well as active cases within schools.
For the 2021-2022 school year, CCS resumed in-person learning for all students.
And most importantly, so much so that I’m giving it its own paragraph instead of having it be a fourth bullet: it was the significant and sustained disruptions to in-person learning that set students back. Outside of the BBS candidates, I honestly have not encountered a single person who does not understand this.
It is absolutely insane to me that we have three candidates running for school board, seemingly with a chance of winning these seats, who consistently demonstrate that they have no understanding of how or why the pandemic impacted students.
Their ‘in-depth’ look at how community COVID cases were higher from 2021-2022 than from 2019-2021 is so irrelevant that it feels like an analysis performed decades in the future, by a high school student whose sole contextual knowledge of the pandemic of 2020-2021 is having read a couple of paragraphs about it in a history book. I take that back. Even that imaginary kid would fare better than the BBS campaign.
As for the pandemic’s effect, from 2019 – 2021, iLEARN proficiency rates declined in:
340 of 348 Indiana school districts, or 97.7%, by an average of 9.2%
1,529 of 1,728 Indiana schools, or 88.5%, by an average of 8.2%
Regarding the BBS campaign’s attempted gotcha statement of:
So, one could make the argument that the pandemic had a significant impact on the scores between 2019 and 2021, right? Well, if that was the case, wouldn’t we expect every single one of the CCS schools to demonstrate a substantial decline in the percentage of students meeting proficiency standards?
First, no. This is terrible logic. I expect that at least a significant majority of current CCS middle and high school students could do better. Let’s hypothetically say that the pandemic reduced every school’s iLEARN proficiency to 10% lower than it would have been otherwise. Then let’s say that without the pandemic, score changes would have ranged from 15% declines to 15% improvements. Under those conditions, even though the pandemic negatively affected every school by 10%, the actual range of changes would be from a 25% decline to a 5% improvement.
Second, no, no one would expect this if they’ve read my site. I noted in my first critique of BBS academic performance misinformation that 8 school districts saw increases in iLEARN proficiency from 2019-2021.
The Claim
That school’s overall iLEARN results should not be impacted when they are asked to serve significantly more or fewer students in disadvantaged subgroups that historically have dramatically lower proficiency rates.
The Response
The Brake, Brown & Sharp campaign have misused Excel to build a fatally flawed statistical model. They’ve done this in an attempt to show that changes in student population resulting from redistricting are not the cause of lowered test scores in three elementary schools. Amidst their long, condescending, incorrect explanation of statistics, a layperson only needs to pay attention to one sentence to realize just how far off base they are. From BBS campaign’s explanation of their model:
What this means is that we would REJECT the idea that there is a linear relationship between changes in iLEARN test scores and ANY of the changes in the student populations.
This is saying that according to the model they’ve built, a school’s iLEARN test scores will not be affected by:
a change in the percentage of students receiving free/reduced-price meals
a change in the percentage of students in special education
a change in the percentage of students currently learning the English language
For reference, here are the aggregate differences in proficiency rates among these subgroups both for Indiana as a whole and CCS specifically:
A model that claims proficiency rates will not be affected by increasing or decreasing the percentage of students in these categories does not pass the smell test. That is, it is so immediately, obviously wrong that the person constructing it should instantly recognize that they’ve made a serious mistake.
As for the reason my previous article ‘didn’t go deeper,’ I was attempting to explain the effect in such a way that someone with little to no knowledge of statistics would be able to understand it. What I did not count on was someone with little to no knowledge of statistics attempting to build a model to refute it.
To whomever at the BBS campaign attempted this analysis, here are two quick notes to help you develop something closer to an actual workable model:
You need to measure percentage changes in each school’s student population, not absolute numbers. With absolute numbers, the differences in school size render a comparison invalid.
You need to create a cumulative effect to sum changes across all three subgroups. There is obviously a difference between a school that increases each subgroup population by 3% and a school that increases one subgroup population by 4% while the other two decrease by 2%.
Taking those two obvious steps yields a model that shows that for every 1% increase in the portion of the student population falling into one of the three subgroups, the iLEARN proficiency level is expected to drop by .79%. Note that the p-value for this is 0.004, well below the 0.05 threshold cited by the BBS campaign.
Also, I’d like to address the BBS campaign’s claim that I am disgusting for blaming disadvantaged children for schools’ test scores decreasing. To be 100% crystal clear, I am not the one doing that, Jenny Brake, Greg Brown and Adam Sharp are. I proudly welcome a decline in a school’s proficiency rate if it’s due to that school helping a greater number of disadvantaged students. Brake, Brown and Sharp are the ones using these declines to denigrate the schools, not me.
Furthermore, I will note with no small level of irony that the BBS campaign has repeatedly misrepresented and demonized DEI initiatives. As applied by Carmel schools, such initiatives have the power to improve the school experience and academic outcomes for students who, for example, are in special education or are learning English as a second language. BBS is literally campaigning against improving things for these students.
Some Closing Thoughts
I’d like to close by addressing several other aspects of the BBS campaign’s attempts to discredit the work I’ve done over the past month.
A telling sign of which parents’ rights matter
While the BBS campaign has finally acknowledged that I actually exist and am a Carmel resident, I find it offensive and disconcerting that they omit from their page that I am a parent with children in the school system.
These are candidates running on a platform of parents’ rights, yet when confronted by an actual parent pointing out the gaps between their claims and reality, they literally frame the discussion as being between a citizen (me) and parents (them). The BBS campaign has made crystal clear that when they talk about parents’ rights, they’re only talking about the select group of parents who agree with them 100% and accept the misinformation they’re spreading without question.
This is my site with my analysis
A parent who wished to remain anonymous provided the brunt of the analysis on my second page refuting the BBS campaign’s misinformation about academic performance. They contacted me through the site's contact form and then we went back and forth for several days to review and refine the analysis they had run. This was explicitly noted in the introduction to the page.
Other than that instance, everything on this site is my own analysis and opinion. For most of the past month, I have effectively treated the refutation of BBS misinformation as a second full-time job. I haven’t tracked the exact time spent on this, but it’s over a hundred hours at this point.
I’ve downloaded, pored through and analyzed state data on test results and enrollment going back over a decade for both districts and schools.
I’ve watched CCS videos of board meetings and on specific topics related to what I’ve written.
I’ve read state law, large chunks of the Indiana Parents’ Bill of Rights and opinions issued by the office of Indiana’s Public Access Official.
I’ve reviewed what the media reported on various events over the past decade, as well as what politicians, officials and experts had to say about them at the time.
I’ve downloaded and reviewed financial disclosures for all of the current candidates.
I’ve sought out and read pro-, neutral and anti- opinions and explanations of topics like SEL and DEI.
I’ve spent hours upon hours writing the pages on this site, carefully laying out where I believe the BBS campaign is obviously distorting reality. As I’ve done this, I’ve continuously provided links to data I cite so that other voters can review these issues much more easily and quickly than it’s been for me.
And I’m sure I’ve left quite a bit off that list.
Note that the anonymous parent referenced above was neither Jeff Boldt nor Michael Gambrel. These gentlemen did reach out to me after I created this site and I have spoken to them. I had no idea who they were until after I had created the site and written multiple articles for it. Nothing on this site uses, builds upon or references the research they’ve done on their own. From what I've seen of their work, they do a great job, but they're doing their own thing that's entirely unrelated to this page.
Why I have done this
In the opening of their response, the BBS campaign rhetorically asks why I created this website and then spins off into their own misinformation without attempting to answer it. I’ll answer it.
First, I wish like hell I didn’t have to. This isn’t fun for me. While all of the long, deep dives into school data have given me an even greater appreciation for CCS, it’s honestly not how I would’ve chosen to spend such a ridiculous amount of my time over the past month.
I would much, much prefer to not have school board candidates dumping huge amounts of time and money into denigrating and tearing down our schools and their accomplishments. It would be amazing to have all of the candidates for school board living in reality and merely disagreeing on which policy approaches would allow CCS to continue to evolve and improve. Unfortunately, we’ll have to hope for that the next time around.
As for why I’ve attempted to take on some of the responsibility of fighting back for our schools, it’s not purely altruistic. I like to think that if I didn’t have kids, I would still be engaging in some level of pushback on behalf of our community, but I can't imagine that I'd have the same level of motivation.
What I can say is that a large portion of why my wife and I moved to Carmel 11 years ago was so our future kids would have access to an excellent public school education. Today, those two kids are in CCS schools and they are wonderful. It astounds me how much the school experience and approach to education has improved in the 40 years since I started attending school. And when I see a group of candidates spreading misinformation and fear out of what appears to be an effort to inject their political beliefs into the school system, you can be damned sure I’m going to speak up.
As always, thanks for reading. As promised, here’s the link to download the BBS campaign’s page as it exists at the time I’m responding to it.
Comments