Keystone Exams supplement
WEEKLY REPORT #10
Established by the ESSA
The Keystone Exam was mandated by the Every Student Succeeds Act (ESSA) was signed by President Obama on December 10, 2015. It establishes a national standard for curriculum accountability for the high school level.
It replaced a existing PSSA test for all 11th graders.
It was intended that the ESSA assessments were to be taken by all students to provide insight into their own academic proficiency and the effectiveness of the schools' teaching. It was hoped that the desire of students, parents, and schools to graduate students on schedule would drive students to achieve and schools to assist and coach them to proficiency, and to improve their curriculum and teaching so that, in time, all 11th graders would be proficient by that point in their scholastic career.
How the Keystone Exam operates
-
Content: The subjects which are tested by Keystone Exams are Algebra I, Literature, and Biology.
-
Questions are 75% which are scored, 25% which "field test" the applicability of new questions.
-
Each scored CR question may be given full or partial credit -- 4, 3, 2, 1, 0 points.
-
-
Length: There is no time limit for a student to complete the test. Each test consists of two modules, which the typical student can complete in 1 - 1.5 hours.
-
For Algebra I, there are 23 multiple-choice and 4 "constructed response (CR)" questions which test Operations and Linear Equations & Inequalities and the same number of questions which test Linear Functions and Data Organization.
-
-
Administration: School districts administer the test and schedule the modules to be given across two days, or separated across the morning and afternoon of the same day.
It is available in an online and also a pencil/paper format. -
Preparation: In order to allow students and teachers to become familiar with the tests' structure and teach or study for the Exam
-
Tutorials and online training have been developed to provide students with an opportunity to experience the online test's appearance and operation.
-
Assessment Anchor documents are available to the public which identify the concepts which will be assessed (13 pages) and with sample items (84 pages).
-
Very thorough Item and Scoring Samplers are developed for each year's Keystone Exam -- 2023's Algebra I sampler is 122 pages long.
-
-
When: Keystone Exams are considered to be end-of-course assessments, although planned for assessing during Grade 11, students may have completed the course work much earlier, or not completed it yet. As such, some students take the exam prior to their junior school year, after completing the preparatory course for a test -- ex., after they take Algebra I in 9th grade -- or in the summer following their 11th grade.
Keystone Exams are offered 4 times annually, during "windows" of around 1-2 weeks duration, including late in May and in July-August.
Students may re-take the exam any number of times.
-
Results: The results are not distributed until the following November. This reduces the time available to students to react to a result of "not proficient" and to arrange a justification to graduate the following June.
-
Manual grading of the partial credit constructed-response questions necessarily delays the completion of Keystone Exam scoring.
-
The results of Keystone Exams are published as the percent of students of the school's 11th grade who took the exam and scored as advanced, proficient, basic, or below basic. (Proficient is also referred to as "grade level".) Students who did not take the Keystone Exam are not counted. These are published in the PA Department of Education website.
-
For the purpose of assessing the school's performance for ESSA standards, school proficiency is calculated in two ways and the school is assigned the higher result. These are reported in the Future Ready PA website.
The two calculations are:-
The "# of students who achieved Advanced or Proficient on the Keystone Exam" divided by the "95% of the students enrolled in Grade 11 when testing ends".
-
The "# of students who achieved Advanced or Proficient"
divided by the "# of students who took the test".
-
-
Ex., Grade 11 has 100 students, 50 achieved Advanced or Proficient.
If every student took the test: 50 / 95 = 52.6%
If only 60 students took the test (and 40 did not): 50 / 60 = 83.3%
If the students taking the test were a random sample of the their class, if only 60 took the test, 30 of would be proficient and the school's score would be represented accurately as 50%.
Therefore, a school's Proficiency is improved if a school's weak students do not take the Keystone Exams.
-
Opting out: As the Keystone Exam was initially implemented, there were many students who were not proficient, who could not be brought up quickly to proficiency, and whose graduation was, then, in jeopardy.
-
In PA, a moratorium on enforcement of the Keystone Exam was declared until the PA General Assembly created alternate pathways for students to establish their proficiency and graduate on schedule. During the moratorium, students were permitted to opt-out of taking the Keystone Exams. The moratorium ended and taking the Keystone Exam became mandatory for the Class of 2022 -- that is, to be taken by juniors beginning with the 2020-21 school year.
-
Editor note: The moratorium was a missed opportunity to improve education in PA. By allowing students to opt-out, even though not exhibiting proficiency in the Keystone Exam subject would have no effect on their ability to graduate high school on schedule, students, parents, and schools had no visibility of academic proficiency issues and, not knowing the extent of a proficiency "gap", would not focus on improving their curricula and teaching to lift more students into proficiency.
-
-
Waiver: In March 2020, the PA General Assembly passed Act 136 which granted (Grade 11) students a "Proficient" performance level for purpose of graduation requirements until June 30, 2020, as schools were unable to administer the Keystone Exams during COVID-19 school lock-outs.
-
As shown in the examples of the ESSA proficiency calculation (above), this may have no impact on the school's calculation. The waived students would not be considered as Exam test takers (a.k.a., Number Scored on Keystone Exam results), but are considered as enrolled students.
-
It also granted waivers for students in other grades who completed a "trigger" course for a Keystone Exam during the lock-down, but no Keystone Exam were being administered following their course completion for them to take. These students may have been in grades prior to Grade 11 during this March - June 2020 period.
-
The Federal Department of Education did not provide a corresponding waiver for its ESSA accountability requirements, so both cohorts of students who were gran (2020 Grade 11 and trigger course-completing students) would not be reported to the USDE as having taken the Keystone Exam, but would be counted as class members.
-
This causes a class to appear to have fewer students who are Proficient (they never took the Keystone Exam) and affects the school's accountability score.
-
The class of 2021 (which would have been assessed in 2020) is unaffected - no Keystone Exam scores were published for 2020.
-
The classes of 2022, 2023, 2024, and 2025 (if, for example, 7th graders passed Algebra I in 2020) are affected if they claimed a waiver in 2020 through Pennsylvania's Act 136 and never took a Keystone Exam to establish their proficiency for the Federal Department of Education and, therefore, add their Proficiency to their school's accountability score.
-
In the following data table, one sees that PHS' class of 2022 (which were in Grade 11 in 2021) appear to have all taken the Algebra I Keystone Exam, either in 2021 or during 2019 or earlier.) None in this graduating class took the Literature exam and a very small number of the class took the Biology exam.
-
-
-
-
PHS' class of 2023 (which were in Grade 11 in 2022), appear to have only around half of its class take any of the Keystone Exams. This should reduce PHS' "Percent Advanced or Proficient" average because its ESSA accountability is for its entire class. Since so many did not take the exams and, so, did not have results of Proficient, PHS' Future Ready PA scores are very poor.
-
However, it appears that only PHS' Biology and Literature Keystone Exam results have been assigned using ESSA rule stated above, that the higher of two alternative calculations should be assigned for school in a given year.
-
The Future Ready PA score for PHS in Algebra I is assigned using the "test takers" formula -- 89% of PHS' 139 test takers scored as Advanced or Proficient.
-
The Future Ready PA scores for Biology and Literature can be proven arithmetically to be the results for the "95% of enrollment" formula.
-
Note: an approximation of the enrollment is used for PHS' Class of 2023 at the end of the 2021-2022 school year -- 300 students -- so, our calculations' results will be slightly different than those calculated by the Pennsylvania Department of Education.
-
This anomaly has been reported to the PDE, along with another anomaly where a Future Ready PA % of Advanced and Proficient is published where fewer than 10 students took the test and no % should be reported.
-
-
-
The practice of opting-out continues in some school districts after the cessation of the moratorium on enforcement of Keystone Exam assessments as the initial step in justifying each student's academic proficiency in Literature, Algebra I, and Biology.
Fewer than half of RTMSD's Classes of 2023 and 2024 took the Keystone Exam (in 2022 and 2023), so the Exams' cumulative scores do not provide insight into how proficient most of these classes' students are in the material.
The students, parents, and school district need to know if they have a proficiency problem -- these kids' senior year is their last opportunity to catch up on the essential skills for college and the workplace which the ESSA standards believe they need to succeed in their careers.
The school district needs to know where their curriculum and/or instruction has failed to equip students for life after RTMSD, so content and teaching methods can be improved and future classes reach their final years in the district proficient in the learning that they will need.
(The yellow-highlighting calls attention to Schools, Subjects, and Years where non-participation in the Keystone Exams is dramatically lower than all other years for the school and subject.)
We have contacted the PA Department of Education for insight into why this phenomenon occurs and received an explanation from Bruce Truesdale via the RA-ED-PSSA-KEYSTONE@pa.gov contact point.
Brian explains that a "COVID-19 pandemic waiver" was offered the 2019-2020 school year only for students who passed a "Keystone Exam trigger course" - that is, a course which has been identified as one which prepares a student to take the Keystone Exam Algebra I, Biology, or Literature assessment in their school.
Note: No Keystone Exams were performed during the 2019-2020 (pandemic) school year.
The waiver granted these students a Proficient performance level for the corresponding Keystone Exam, without needing to ever take the exam. (Also, the Federal Department of Education does not recognize waiver-granted proficiency -- it requires that the Keystone Exam be taken for certify proficiency.)
Since these students never took the Keystone Exam, they have no score which could be included in published Keystone Exams results. This adds to the anomaly of fewer students scored than there are students in the class.
(Note: there are other reasons that students in the class might not have a Keystone Exam score.)
