Evaluation of Implementation and Effectiveness
Overview
Inherent in the ProComp design is the need to ensure that the compensation system is working properly and is meeting the needs it was created to fulfill. Among the tests given to the system for evaluation were whether it would be fair to all teachers and SSPs, whether the system was manageable, if it was sustainable and whether it fulfilled its goal of being good for student outcomes.
Part of the ProComp agreement calls for annual reviews of the system, beginning with internal reviews and eventually providing for an external, third-party review to make certain that ProComp continues to fulfill its mission.
The Evaluation Process
The initial evaluation of ProComp was conducted through the Program Evaluation Work Group at the end of the 2004-2005 contract year. The work group included teachers, SSPs, administrators and DPS staff. During the 2005-2006 contract year, a contract was generated with the University of Colorado, Boulder, to provide research assistance for the work group.
The evaluation consisted of qualitative research conducted through focus groups of the various work groups assigned to build the structure of ProComp and the Transition Team.
Focus groups also were held with the Teaching and Learning Department and Department of Technology Services and the Human Resources Department, and Payroll/Business Services.
The first report was completed in the fall of 2005; findings were presented to the Transition Team and approved.
In the 2005-2006 contract year, the Program Evaluation Work Group again conducted research with the various work groups to determine how they were progressing and how the groups were integrating. Some of the work groups were no longer meeting on a regular basis so full scale focus groups were not conducted. In addition to the information from the work groups, an online survey was generated and conducted in the Spring of 2006 to get feedback on how teachers, SSPs and administrators viewed ProComp.
Future Evaluations
The ProComp agreement calls for the continuation of internal evaluations and for an external evaluation in the future. The results of these evaluations and recommendations will be presented to DPS and DCTA leadership in November 2006 and made available online at www.denverprocomp.org.