Phase 2 of Selection Process
In 2023, 78 students* were invited into the advanced math selection process based on their QN CogAT scores. These students were evaluated based on a rubric to select 56 students (28 for the class at Munsey and 28 for the class at Shelter Rock.) The rubric replaced the Committee review. Students needed 16 points out of 20 for admission.
The Rubric
56 students were admitted using the rubric.**
Nearly everyone in Phase 2 started with 8 points in the rubric - 4 points from the CogAT** and 4 points from the work sample. To understand points from the work sample, click here.
Therefore, placement was determined only by the secure exam score and teacher rating. Teacher ratings mattered more than the exam score.
The rubric below was one of the rubrics applied to the 2023 cohort according to Lauren Tallarine, District Coordinator of Math and Business. The gray shows which students were admitted. The white shows which students were denied.
Revaluation for Rubric Points
The secure exam scores were revalued from 1, 2, 3 or 4. These scores were then doubled to 2, 4, 6 or 8 to give "double weight" as required by policy.
-
Over 80% was revalued to a '4' worth 8 points; students were automatically admitted (29% of students)
-
60-80% was revalued to a '3' worth 6 points
-
50-60% was revalued to a '2' worth 4 points
-
Under 50% was revalued to a '1' worth 2 points; students were automatically denied (8% of students)
The Renzulli rating scale was used to collect teacher input on each student. Click here to see the form. Scores were revalued to reflect 1, 2, 3 or 4 points in the rubric.
-
Scores of 58-60 were given a '4'
-
Scores of 55-57 were given a '3'
-
Scores of 50-54 were given a '2'
-
Scores of <50 were given a '1'
Rubric Issues
1) The rubric was designed so that teacher ratings determined what level of content knowledge was required to admit a student.
-
Students who had teacher ratings of '4' only needed a 50% on the secure exam for admission
-
Students who had teacher ratings of '1' only needed a 80% on the secure exam for admission
2) Students with very different performances were grouped together and given the same point value
-
Students with a 79% and a 60% were given the same 6 points in the rubric.
-
The "double weight" was not actually applied as data was blurred before it was doubled
3) The Renzulli rating, the most subjective criteria and the item with the most sensitivity became the driver of the rubric outcomes
-
The teacher ratings were inaccurate; they were heavily influenced by teacher behavior
-
Teacher ratings drove the outcome for ~65% of students under consideration
-
The rubric gave unintended weight to the the teacher rating because it was very sensitive to small differences in teacher scores. A very small 5% variation in teacher rating (i.e., from 55 to 58) led to an additional rubric point. In contrast, a secure exam test score had to vary by 20%, i.e., from 60 to 80%, to lead to an additional rubric point.
Students Impacted by Teacher Rating
For the majority (64%) of students invited into the process, the teacher rating they receive meaningfully changes the secure exam score needed for admission which can range from a 50% to an 80% due to the rubric applied. Since 92% of students score a 50% and only 29% score an 80%, this radically changes their probability of admission. The relevant part of the rubric for these students is shown below.
-
If a student has an easy grading teacher (gives out '4' ratings readily), he/she is nearly guaranteed admission with only a 50% required on the secure exam. Over 92% of students who take the secure exam score over 50%. nce a student gets a '4' rating from their teacher, they have a 92% chance of admission. For information on secure exam performance for the 2023 cohort, click here.
-
If a student has a hard grading teacher (gives few or no '4' ratings), he/she will need a 60% (if given a '3' or '2') or an 80% (if given a '1') on the secure exam.
Once a student gets a '2' or '3', they have a 73% chance of admission (20 points lower than the student who got a '4') because 73% of students score over a 60% on the secure exam.
Once a student gets a '1', he/she only has a 29% chance of being admitted (64 points lower than the student who got a '4') because only 29% of student scored an 80% on the secure exam.
Students with one of the 3 hard teachers have a weighted average probability of getting in of 59% compared to a 93% probability of those who had the easy teachers. That's a 34% difference, just based on the teacher you were assigned at the start of grade 5. That is patently unfair!
All students admitted to the advanced math program should be held to the same content knowledge standards. Those with easy grading teachers should not be held to a content knowledge standard 30 percentage points lower than those with easy teachers.
Prior to 2021, a secure exam and teacher ratings did not exist. All students were only tested on content taught in the classroom. Scores from unit tests and a final were combined with their aptitude test score to determine a composite score. Students were ranked based on that composite score. There was no biased teacher rating of "acceptable" 6th grade knowledge threshold that varied by 30 points from one student to another. This has resulted from a poorly envisioned 2020 policy that was compounded by poor execution via an illogical rubric.
When this was highlighted to administrators in the summer of 2023, they defended the teacher rating system as reliable and desired. This is wrong.
Before the selection process was changed in 2021, teacher ratings did not play a role in placement at all. Unless teacher ratings can be done accurately, they should be eliminated.
If you want your child in advanced math, make sure you get one of the easy graders for your 5th grader!
Additional Rubrics
There may have been additional rubrics to achieve the goal of similar class sizes in Munsey and Shelter. Click here to learn about class sizes for advanced math.
In addition, the rubrics vary from year to year. The district has stated that it does not maintain the criteria cutoffs for prior years. It seems that the rubric is not created before the process begins, it is modified based on the pool of students being considered to target certain numbers or groups.
*In 2023, the CogAT threshold was arbitrarily lowered from 98%tile to 97%tile, expanding the pool from about 66 eligible students to 78 students.
**A second rubric was created to accommodate the students with a lower CogAT threshold. A comparison of the two can be provided upon request at manhassetmath@gmail.com. These are two of the rubrics we are aware of in the process to accommodate different subgroups. Other rubrics may have been created but have not been disclosed for 2023.