Sidebar

Magazine menu

19
Fri, Apr

Measuring Student Proficiency and Creating Capacity: The Move Toward End-of-Course Exams in Pennsylvania

Government/Policy
Typography

Introduction

The purpose of this article is to provide policymakers and practitioners with recommendations regarding how to maximize the benefits and minimize the risks associated with end-of-course exams. In particular, the paper provides educators in Pennsylvania with a description of the state’s new graduation policy and contextualizes the state’s move to end-of-course exams in the national context.

In January 2010, changes to Pennsylvania’s high school graduation requirements became official. When fully implemented, the new policy will require students to demonstrate proficiency in six academic content areas in order to receive a high school diploma. Students will be able to demonstrate competency on new state academic standards by passing either a set of end-of-course exams, called Keystone exams, or approved alternatives. Students in the graduating class of 2015 will be the first who must demonstrate proficiency as measured by Keystone exams or approved alternatives in Algebra 1, Literature, English Composition and Biology. The Keystone exams will be phased in over a period of several years. The timing of implementation has undergone modifications reflecting political, technical and financial considerations.

For the past decade, the state has used the Pennsylvania System of School Assessment (PSSA) to assess every Pennsylvania public school student in reading and math in grades 3–8 and grade 11. Additionally, students in 5th, 8th and 11th grades are assessed in writing, and students in 4th, 8th and 11th grades are assessed in science. According to the federal No Child Left Behind Act (NCLB), all students in a state must be 100 percent proficient in reading and math by 2014. NCLB requires states to determine annually whether schools and districts are making Adequate Yearly Progress (AYP) toward this goal in reading and math proficiency. Pennsylvania uses the PSSA assessment to determine whether districts are meeting AYP requirements. The Pennsylvania Department of Education (PDE) plans to have the Algebra 1 and Literature Keystone exams approved and used in place of the 11th-grade PSSA for federal AYP provisions during 2013–2014. The Keystone exam policy will not alter the state’s 4th- to 8th-grade PSSA program.

A Closer Look at the Keystone Exam Policy

  • When the policy is implemented fully, the state will have Keystone exams in ten subjects, and students will need to pass six exams or approved alternatives as a condition of gradation. Students will need to pass:
  • 2 English exams (Composition and Literature);
  • 2 Mathematics exams (Algebra I, Algebra II or Geometry);
  • 1 Science exam (Biology or Chemistry); and
  • 1 Social Studies exam (Civics and Government, U.S. History or World History).

(Currently, the regulations require the class of 2015 to demonstrate proficiency in Algebra 1, Literature, English Composition and biology. A social studies exam will be field tested in 2012–2013 for use in 2013–2014. The timing for the other exams is yet to be determined.)

The new graduation policy leaves room for alternatives regarding how students can demonstrate proficiency in the state standards in each subject. Proficiency in each subject can be determined by: 1) a Keystone exam counting as 33 percent of a student’s final course grade (regardless of the student’s score on the Keystone); 2) a Keystone exam as a standalone graduation requirement (in this case the student will need to score at the proficient level on the exam); 3) an independently validated local assessment; or 4) a relevant Advanced Placement or International Baccalaureate exam. A student who does not score “proficient” on a Keystone exam after two attempts will be eligible to complete a project-based assessment. The details of the project-based alternative are currently being worked out by PDE.

Pennsylvania has created a number of supports for districts and schools to utilize as they work to meet the increased demands associated with the rigor of the new graduation policy. The Keystone exams and complementary resources are embedded in the Pennsylvania’s Standards Aligned System (SAS).  SAS is a comprehensive web-based resource to assist schools and districts in aligning their curriculum and instruction to the state’s academic standards. An integral part of SAS and the new policy are Voluntary Model Curriculum (VMC) and Classroom Diagnostic Tools (CDT). CDTs are sets of online assessments, divided by content area, which are designed to offer diagnostic information to teachers and students in order to guide instruction and remediation. CDTs are currently available for math and will be available across academic subjects in grade 6 through high school in the future. VMC are online resources for schools and districts that include unit and lesson plans, as well as other content resources aligned to the Pennsylvania standards in the four content areas covered by the Keystone exams (mathematics, science, social studies and reading-writing-speaking-listening).

Introduction

The purpose of this article is to provide policymakers and practitioners with recommendations regarding how to maximize the benefits and minimize the risks associated with end-of-course exams. In particular, the paper provides educators in Pennsylvania with a description of the state’s new graduation policy and contextualizes the state’s move to end-of-course exams in the national context.

In January 2010, changes to Pennsylvania’s high school graduation requirements became official. When fully implemented, the new policy will require students to demonstrate proficiency in six academic content areas in order to receive a high school diploma. Students will be able to demonstrate competency on new state academic standards by passing either a set of end-of-course exams, called Keystone exams, or approved alternatives. Students in the graduating class of 2015 will be the first who must demonstrate proficiency as measured by Keystone exams or approved alternatives in Algebra 1, Literature, English Composition and Biology. The Keystone exams will be phased in over a period of several years. The timing of implementation has undergone modifications reflecting political, technical and financial considerations.

For the past decade, the state has used the Pennsylvania System of School Assessment (PSSA) to assess every Pennsylvania public school student in reading and math in grades 3–8 and grade 11. Additionally, students in 5th, 8th and 11th grades are assessed in writing, and students in 4th, 8th and 11th grades are assessed in science. According to the federal No Child Left Behind Act (NCLB), all students in a state must be 100 percent proficient in reading and math by 2014. NCLB requires states to determine annually whether schools and districts are making Adequate Yearly Progress (AYP) toward this goal in reading and math proficiency. Pennsylvania uses the PSSA assessment to determine whether districts are meeting AYP requirements. The Pennsylvania Department of Education (PDE) plans to have the Algebra 1 and Literature Keystone exams approved and used in place of the 11th-grade PSSA for federal AYP provisions during 2013–2014. The Keystone exam policy will not alter the state’s 4th- to 8th-grade PSSA program.

A Closer Look at the Keystone Exam Policy

  • When the policy is implemented fully, the state will have Keystone exams in ten subjects, and students will need to pass six exams or approved alternatives as a condition of gradation. Students will need to pass:
  • 2 English exams (Composition and Literature);
  • 2 Mathematics exams (Algebra I, Algebra II or Geometry);
  • 1 Science exam (Biology or Chemistry); and
  • 1 Social Studies exam (Civics and Government, U.S. History or World History).

(Currently, the regulations require the class of 2015 to demonstrate proficiency in Algebra 1, Literature, English Composition and biology. A social studies exam will be field tested in 2012–2013 for use in 2013–2014. The timing for the other exams is yet to be determined.)

The new graduation policy leaves room for alternatives regarding how students can demonstrate proficiency in the state standards in each subject. Proficiency in each subject can be determined by: 1) a Keystone exam counting as 33 percent of a student’s final course grade (regardless of the student’s score on the Keystone); 2) a Keystone exam as a standalone graduation requirement (in this case the student will need to score at the proficient level on the exam); 3) an independently validated local assessment; or 4) a relevant Advanced Placement or International Baccalaureate exam. A student who does not score “proficient” on a Keystone exam after two attempts will be eligible to complete a project-based assessment. The details of the project-based alternative are currently being worked out by PDE.

Pennsylvania has created a number of supports for districts and schools to utilize as they work to meet the increased demands associated with the rigor of the new graduation policy. The Keystone exams and complementary resources are embedded in the Pennsylvania’s Standards Aligned System (SAS).  SAS is a comprehensive web-based resource to assist schools and districts in aligning their curriculum and instruction to the state’s academic standards. An integral part of SAS and the new policy are Voluntary Model Curriculum (VMC) and Classroom Diagnostic Tools (CDT). CDTs are sets of online assessments, divided by content area, which are designed to offer diagnostic information to teachers and students in order to guide instruction and remediation. CDTs are currently available for math and will be available across academic subjects in grade 6 through high school in the future. VMC are online resources for schools and districts that include unit and lesson plans, as well as other content resources aligned to the Pennsylvania standards in the four content areas covered by the Keystone exams (mathematics, science, social studies and reading-writing-speaking-listening).

What Are End-of-Course Exams?

What Are End-of-Course Exams?

It is estimated that by next year, 74 percent of the nation’s public school students in 28 states

will be required to pass an exit exam as a condition of graduation (Center on Education Policy 2010). Increasingly, states are turning to end-of-course (EOC) exams rather than comprehensive tests as a graduation requirement. By 2015, 14 states will be using EOC exams as a graduation requirement (this includes three states that will be using both an EOC and comprehensive exam) (Center on Education Policy 2008). EOC exams differ from “comprehensive” exams such as the PSSA, which students take in a specific grade and cover material taught during several grades. EOC exams assess mastery of specific high school courses and are usually taken soon after the completion of a specific course (e.g., Algebra 1 or Biology).

Proponents argue that EOC exams are better than comprehensive exams, because the exams are taken closer to the point of instruction and measure content for a particular class. Because of this, supporters claim that EOC exams are more accurate measures of student learning and have a greater potential to improve instruction and curriculum development. Opponents, on the other hand, argue that because the EOC approach often includes numerous exams (more so than comprehensive exams) they are expensive and can impose logistical challenges to states and districts (Center on Education Policy 2010).

Maximizing Benefits While Minimizing Risks of End-of-Course Exams

Reducing the Negative Consequences

While using EOC exams to make graduation decisions for individual students may increase the motivation of educators, students and parents to make the changes necessary to improve student learning, using EOC exams to make high-stakes decisions regarding individual students may also lead to a number of unintended consequences. Negative results include narrowing the curriculum (see Au 2007), increasing dropout/pushout rates (especially for African-American, Latina/o, Native American, and English language learning students as well as students living in poverty) (see Amrein and Berliner 2002), and cheating by school- and district-level personnel (see Nichols and Berliner 2007). Additionally, states need to make sure that EOC exams are valid for the purposes of making graduation decisions for individual students. It is possible that EOC exams developed primarily to drive changes in curriculum and instruction are not valid for making inferences about student graduation (or for determining teacher performance, for that matter) (see Koretz 2008).

One way to mitigate the potential negative consequences of EOC exams is to use the assessment program as a tool to drive curricular and pedagogical improvement and not to make decisions regarding individual students or teachers. This approach is used in a number of states including Connecticut (2018), Delaware, Hawaii, Montana and South Dakota (Center on Education Policy 2010). Pennsylvania, like a number of other states, will also use a comprehensive project as an alternative pathway for students to demonstrate proficiency and receive a high school diploma. Having an alternative pathway for graduation allows students to demonstrate mastery of the same high standards set by the state in a way that is more in alignment with their own learning and response styles. Alternative paths to graduation may be particularly beneficial for students with disabilities and English language learners for whom EOC exams (or other exit exam policies) may have disproportionately negative consequences.

Additionally, it is important for states to allow for flexibility in the implementation timeline. This allows instruction to “catch up” to the increased rigor of both the content standards and assessments associated with EOC exams.  Currently, PDE has decided to delay the implementation of the Keystone exam policy. In part, this reflects the current fiscal limitations faced by the state; however, a contributing factor is students’ poor results on both Keystone field tests and CDTs (Tomalis 2011). The additional year will allow districts to align their curriculum with the content that the Keystone exams cover and to provide additional professional development. This delay in Pennsylvania demonstrates the importance of having a flexible timeline when implementing complex assessment reforms.

Developing and Utilizing Supports

An essential component of an EOC exam policy is the creation of supplemental materials and resources to assist districts and schools in meeting the demands of aligning curriculum and instruction to the new standards and rigorous exams. The development of supports should involve a range of stakeholders, especially local educators. In Pennsylvania, local educators played an important role in the creation of the VMC and CDTs. Practitioners assisted in developing model lessons, identifying relevant academic content and reviewing test items. The inclusion of district- and school-based educators in the design process will help to ensure that resources meet the diverse range of educators’ and students’ needs. Additionally, professional development for teachers should focus on the academic content and pedagogical training necessary to make sure all students are provided with the opportunity to meet the increased demands of EOC exams.

Creating supporting materials such as VMC and CDTs is necessary but not sufficient. To achieve the maximum benefit associated with EOC exams, stakeholders at the state, district and school level must collaborate on designing implementation systems that best meet the needs of local contexts. It is crucial that comprehensive training occurs to familiarize educators with the resources that are part of an EOC exam assessment policy. While PDE has an extensive implementation plan, there are early indications that CDTs and VMC are not being equally accessed in schools and classrooms across the state.

Creating the Conditions for Innovation

While EOC assessment policies may have the potential to target instruction and inform curricular development, states need to allow for local innovation. Pennsylvania has built into its new graduation policy regulations allowing for the development and validation of local assessments as a measure of student proficiency on the state standards. Districts have the option to create their own assessment program and have it validated by an approved contractor. PDE will also pay for half of the validation costs. PDE is currently developing a handbook for districts and charters to use as a guide if they choose to create a local assessment. Additionally, because PDE considers charter schools independent from local district authority, they too will be able to submit local assessment plans. The utilization of local assessment provides the opportunity for districts and charters to develop assessment systems that meet their individual needs.

If states truly want to create conditions that facilitate local innovation, they need to provide both financial and technical support to districts that want to create local assessment. Doing this will increase the potential for the creation of local innovation. States also need to work with districts and charters to make sure that the requirements of EOC exams accommodate any non-standard multidisciplinary or integrated curriculums. Local districts, charters and charter providers also need to think creatively and explore collaborative opportunities to develop local assessments and innovative curricula.

Conclusion

It is important for policymakers and practitioners to work together to make sure that EOC assessment systems live up to their potential to improve student learning and prepare all students in Pennsylvania for success in college and career. To do this, stakeholders from across the educational system need to collaborate to inform decisions across the policy-to-practice continuum. EOC exams alone are not a reform. EOC policies need to be coupled with curricular and instructional resources and training. It is also important that policymakers and practitioners learn from successful policies and programs. To do this, systems need to be put into place that allow for collaboration and the sharing of best practices across programs. If done properly, EOC policies have the potential to improve educational opportunities for all students.

Frank D. Grossman is an assistant professor of education at Swarthmore College. He is interested in the politics of large-scale assessment policy and the role of educational practitioners play in school reform. He is currently working on a project examining how actors working within the Pennsylvania Department of Education communicate and use information in support of a new graduation policy.

 

References

References

Amrein, A. L., and D. C. Berliner. (2002, March 28). High-stakes testing, uncertainty, and student learning. Education Policy Analysis Archives 10(18). Available at http://epaa.asu.edu/epaa/v10n18/.

Au, W. (2007). High-stakes testing and curricular control: A qualitative metasynthesis. Educational Researcher 36(5): 258-267.

Center on Education Policy. (2010, December). State high school tests: Exit exams and other assessments. Washington, D.C.

Center on Education Policy (2008, August). State High School Exit Exams: A Move Toward End-of-Course Exams. Washington, D.C..

Koretz, D. (2008). Measuring Up: What Educational Testing Really Tells Us. Cambridge, Mass.: Harvard University Press.

Nichols S. L., and D. C. Berliner. (2007). Collateral Damage: How High-Stakes Testing Corrupts America's Schools. Cambridge, Mass.: Harvard University Press.

Pennsylvania Department of Education. (n.d.). Keystone Exam Program Overview. Available at http://www.conneautsd.org/ourpages/auto/2010/12/21/48129931/Keystone%20Exams%20Program%20Overview%20PDE.pdf.

Tomalis, R. J. (2011, March 8). Memo to Superintendents, I.U. Executive Directors, Charter School Administrators and CTC Directors.

Issue 6 Impact