|Text Index||Custom Search|
Response to FADSS Press Release - Audit reports are responsive
Letter from Commissioner Smith
The Department and external auditors have worked every day for three weeks to collect and analyze FCAT processes and data in response to the concerns of school district superintendents. This has been our singular focus during these weeks. All three external audit groups have taken this very seriously and applied their expertise – which is considerable and nationally respected – to this work. Reports from these audits by Buros, HumRRO, and The Center for Assessment have been made publicly available on our website. The volume of data and analyses on the website clearly demonstrates the thoroughness and magnitude of this effort. All three audit reports clearly state that the FCAT results are valid and that the public can be assured that best practices are used throughout.
FADSS Concern 1: These reports reflect high-level, high-quality work focused on the most relevant and critical questions that might impact results, including a thorough study of learning gains. The Center’s report analyzed learning gains in pages 1 through 8 and concluded that the average change across grades and years is small. HumRRO’s analysis on pages 4 – 10 illustrate that the pattern of gains in lower achieving students across the years is more variable but conclude that this year’s results are not anomalous.
FADSS Concern 2: In equating, the Department uses passages from multiple years to the extent possible to track change in both achievement level status and learning gains. Equating only to the prior year can cause a shift in the difficulty of the test away from earlier years, distorting the meaning of “proficient”. However, as an analysis of the concern HumRRO did conduct alternative equating approaches which yielded results that were still within the normal historic range of random variation.
FADSS Concern 3: The Department sampled thousands of student test books during the scanning process. This process is described in a report of that work, as well as other quality assurance steps taken to ensure the accurate match of students to scores, which was posted with other results on Wednesday.
FADSS Concern 4: The content focus of each year’s test for each grade and content area is varied by design so that the full range of the Sunshine State Standards continues to be taught. The variation is always within specified limits and aligned with the Standards. Most importantly, the statistical characteristics of the tests are carefully controlled to ensure that the difficulties of each year’s tests are closely matched. The statistical evidence of this is posted in various documents, including test characteristic curves, at http://www.fldoe.org/SchoolGrades.asp.
FADSS Concern 5: A comparison of gains for students in different achievement levels across the years is presented in the Center’s audit report on pages 9 and 10. Dr. Hill concludes that while the improvements in past years with more students in achievement levels 2 and 3 in the bottom quarter were a factor, these changes played only a small role in the decline of learning gain scores.
Our assessment and accountability system has brought Florida’s student achievement up from the bottom a decade ago to a level that now receives national acclaim and which other states seek to replicate. This progress reflects very hard work on the part of our districts, teachers and students. Overall, the 2010 FCAT results show continued successes; these successes should be celebrated. While we must work together to address some areas where improvement has slowed, we must also refocus our conversation to give well-deserved praise for those students and schools who have made great improvements this year.
Dr. Eric J. Smith
Florida Commissioner of Education