UPDATE: Tennessee PTA responds to TNReady scoring error controversy: "Tennessee PTA encourages the Tennessee Department of Education to mandate more stringent procedures to avoid problems with TNReady outcomes and testing. Teachers, parents and students have the right to timely accurate information on academic performance in order to make informed decisions and provide student support."
About 9,400 TNReady tests across the state were scored incorrectly, according to the Tennessee Department of Education.
Hamilton County and Bradley County were among the 33 districts affected by the scoring error.
The scoring issue impacted about 70 schools statewide. According to Hamilton County superintendent Dr. Bryan Johnson, Lookout Valley Middle/High, Red Bank High and Ivy Academy were the three local schools. In Bradley County, superintendent Dr. Linda Cash said Walker Valley High, Bradley Central, and Bradley County Virtual School are the only schools affected.
Dr. Cash said, "The only subject affected was Integrated Math 2. The scoring error involved 402 students but only 55 students had corrections that changed their performance levels. Nine teachers were impacted by the recent events and we are awaiting their revised TVAAS scores to see if the changes were significant enough to make a change in their individual growth score. The district will also be reviewing revised accountability data at the teacher, school and district level for growth and achievement."
"There is no impact to the statewide results," according to an email from state spokeswoman Sara Gast. The impact on individual schools' scores, however, is unclear.
Just over 1,000 of those incorrectly scored were in Shelby County Schools, according to an email from Superintendent Dorsey Hopson to his board on Friday. Metro Nashville Public Schools was also affected. Here is the entire list:
Achievement School District
Huntingdon Special School District
Here is a statement from the Tennessee Department of Education:
The following statement is from Questar, who is the TNReady assessment vendor: “Questar takes responsibility for and apologizes for this scoring error. We are putting in additional steps in our processes to prevent any future occurrence. We are in the process of producing revised reports and committed to doing so as quickly as possible.” —Brad M. Baumgartner Chief Operating Officer
Over the past week, the Tennessee Department of Education has been updating districts about a TNReady EOC scoring issue. To be clear, less than 1 percent of student scores are affected by this issue, which is narrow in scope. Additionally, many of the impacted students will not see any material change in their scores when our final data is updated. We provided this information to be fully transparent with districts and alert them of any issues we found as soon as they were identified. The department completes a myriad of quality checks to ensure all assessment is accurate. These specific issues stem from scanning program changes and resulting delays that our assessment vendor Questar issued a statement about earlier this year – and those errors are now being addressed.
Questar understands the seriousness of this quality lapse, and we have taken additional steps to ensure that these issues are avoided in the future. Though we have reported over 99 percent of grades 3-8 and EOC score data correctly, we need to be at 100 percent accuracy. We hold our vendor and ourselves to the highest standard of delivery because that is what students, teachers, and families in Tennessee deserve. Additional context follows: We identified a data issue through conversations with districts during the TNReady embargo period. The department engaged with our vendor to research a few questions, and, ultimately, Questar determined that they had an error in their scan program, which impacted some English I & II and Integrated Math II end-of-course exams.
Questar’s scanner programming error resulted in 5 form versions of the EOC tests being scored incorrectly, out of 275 versions of the EOC tests administered across all content areas. This means there are about 8,000 tests that were initially reported with a lower scale score than was actually earned. Another 1,400 tests had scale scores that were reported with a higher scale score than was earned. Most of these scale score changes were very small, so just under 1,700 of nearly 600,000 EOC tests (or 0.3%) had any change in overall performance level (e.g. moving from approaching to on track). Questar has now correctly re-scored these tests, and they are processing new score reports for those students, which we will distribute to districts.
Additionally, we will update the individual TVAAS growth measures for those 230 teachers who may be impacted by these student score changes for EOC. We are reaching out to those educators to let them know about this issue through TNCompass. The EOC scoring issue impacted about 70 schools (out of 1,800) in 33 districts (out of 146). Because relatively few – 0.3% –tests had performance level changes, there is no impact to the statewide results. All of the updated information will be reflected in the public TVAAS site and our State Report Card released later this fall.
Final TVAAS composites for educators will also be updated in TNCompass and our TVAAS restricted site. We have been reaching out to impacted districts to ensure they understand the EOC score issue and have a chance to ask questions as we move swiftly to finalize results for district accountability determinations for 2016-17 In addition, we did an in-depth review of the tests of grades 3-8, and confirmed with Questar that there were no issues with the scanning program and resulting scores for any of those tests. The department goes through multiple quality checks to ensure the assessment data that is accurate. For context on the total scope of what we have been reviewing, the department has processed about 1.5 million linkage records to generate individual TVAAS scores for nearly 19,000 educators based on the assessment results from over 1.9 million student tests in grades 2-8 and high school.