What Does NAEP Grade 8 Data on School Experience Show about Student Learning?
NATIONAL CONFERENCE ON 
STUDENT ASSESSMENT
Los Angeles, California
June 23, 2009
•	Wendy Geiger
VA NAEP Coordinator
•	Carrie L. Giovannone
AZ NAEP Coordinator
•	Pam A. Sandoval
CO NAEP Coordinator
•	Michele Sonnenfeld
FL NAEP Coordinator

Slide2
Overview of Presentation
Grade 8 Perspective
Use of Various NAEP data results from students, teachers and schools to
•	Show new findings
•	Support current research
•	Make connections across research studies
•	Give suggestions on how to use the NAEP background data

Slide 3
NAEP Reading Grade 8 
Comparing 2003 to 2007
Only Vermont had a higher average score in 2007 than in 2003
West Virginia, Kentucky, Illinois, Missouri, and Mississippi had lower average scale scores in 2007 than in 2003

Slide 4
What Does a Grade 8 Student Look Like?

Slide 5
Media characterizes middle-school students as
•	Self-Centered
•	Hormone-Driven
•	Thoughtless
•	Undisciplined (Kinney, 2007)

Slide 6 
Middle-school students can also be
•	Concerned
•	Caring
•	Compassionate (Kinney, 2007)

Slide 7
1960s Perception
•	The brain of a child ages 12 to 14 does not grow. 
•	Teaching complex material during the middle-school years will have a damaging effect. 
•	Middle school is a time for social adjustment, individual growth, and coping with early adolescence, not academic learning or self-discipline. This is known as the Life-Adjustment Movement. (Yecke, 2005)

Slide 8
21st Century Perception
•	Focus must be on students’ acquisition of essential academic skills 
and knowledge. (ACT, 2008)
•	Talent can be practical, creative, communicative, enterprising, as well as analytical intelligence. (Prime Minster Brown, 2009)
•	Students must be instilled with a love of learning for its own sake. (Rose, 2009)

Slide 9
The Forgotten Middle 
Middle schools should be dedicated to the goals of high academic standards and 
achievement, challenging curriculum, results-based accountability, and sound discipline. (ACT, 2008)

Slide 10
“…eighth-grade achievement is the best predictor of students’ ultimate level 
of college and career readiness by high school graduation–even more than students’ family background, high school coursework, or high school grade point average.”
-	The Forgotten Middle
(ACT, 2008)

Slide 11
What are the students saying?
What students said:
•	92% indicated they would definitely or probably attend college.
•	93% said there was no chance they would drop out of high school and not graduate.
What really happens:
•	Only 66% begin college directly after high school.
•	Only 83% actually graduate from high school. (NASSP/PDK, 2007)

Slide 12
Why do Grade 8 students who say they will attend college fail to enroll?
They do not:
•	have the knowledge and skills needed to enter and succeed in high school;
•	have the academic and behavioral discipline to be successful in high school; 
•	know which courses are required for college entry;
•	have access to adequate funds to pay for college;
•	think the university is for them;
•	have encouragement from home.        (ACT, 2008)

Slide 13
What Does the NAEP Data Say about Grade 8 Student Learning?
•	Main NAEP
•	Long Term Trend
•	High School Transcript Study
•	Questionnaire Data
•	Average Scale Scores
•	Percentages

Slide 14
NAEP Questionnaires
Pictures of covers of Sample Questions, School Questionnaire, Teacher Background Questionnaire, and Language Arts Teacher Background Questionnaire. 

Slide 15
NAEP Student Questionnaires 
Picture of cover of Sample Questions 2007 Mathematics, Reading, Writing Grade 8:
•	Completed by students and used to collect background information.
•	Collected information on students' demographic characteristics, classroom experiences, and educational support.

Slide 16
NAEP Teacher Questionnaires 
Pictures of covers of Teacher Background Questionnaire and Language Arts Teacher. Background Questionnaire.
•	Completed by teachers and used to collect background information.
•	Collected data on teachers’ background, training, and classroom-by-classroom information.

Slide 17
NAEP School Questionnaires 
Picture of cover of School Questionnaire 2007 Grade 8.
•	Completed by school principal or other head administrator.
•	Collected information about school policies that relate to or describe characteristics of schools.
•	Investigated the relationship between student achievement and school factors that may influence achievement.

Slide 18
Uses for the NAEP Background Data
•	Gives us “rich” data that can be linked to students’ NAEP scores
•	Can be corroborated with other research
•	Can reveal areas of interest for further research

Slide 19
Limitations of the NAEP Background Data
•	Does not determine causality
•	Must keep in mind the standard errors
•	Many explanations can play a role – only used to raise “red flags” in the data

Slide 20
Affective Disposition/Difficulty of Assessment 
“How hard was this test compared to most other tests you have taken this year in school?”  
Students chose from the following options:
•	Easier than other tests
•	About as hard as other tests
•	Harder than other tests
•	Much harder than other tests

Slide 21
Difficulty of Assessment
Student Questionnaire, Grade 8 NAEP 2007 Reading.
The data verifies that the students’ answers on the background questionnaires are valid.
These data show that the easier the test was perceived to be by the students, the higher their scale scores.
•	Easier than others – 62%, average scale score of 265
•	As hard as others – 30%, average scale score of 261
•	Harder than others – 6%, average scale score of 245
•	Much harder – 2%, average scale score of 226

Slide 22
Importance of Success on NAEP Reading Assessment
Student Questionnaire, Grade 8 NAEP 2007 Reading.
As students placed greater importance on the NAEP test, the lower their scale scores.
•	Not very important – 16%, average scale score of 259
•	Somewhat important – 34%, average scale score 265
•	Important – 30%, average scale score 263
•	Very important – 20%, average scale score 257

Slide 23
Cross Tab - Difficulty and Importance
Student Questionnaire, Grade 8 NAEP 2007 Reading
The majority of the students are in the “easier than others” and “somewhat important”/”important” cross-tab cells.
The NAEP data shows that the more value the students placed on the test, the lower they scored.
•	Students who thought the test was not very important and easier than other tests had an average scale score of 263
•	Students who thought the test was not very important and as hard as other tests had an average scale score of 259
•	Students who thought the test was somewhat important and easier than other tests had an average scale score of 268
•	Students who thought the test was somewhat important and as hard as other tests had an average scale score of 264
•	Students who thought the test was important and easier than other tests had an average scale score of 266
•	Students who thought the test was important and as hard as other tests had an average scale score of 262
•	Students who thought the test was very important and easier than other tests had an average scale score of 261
•	Students who thought the test was very important and as hard as other tests had an average scale score of 256

Slide 24
Teachers Play a Role in Student Learning
Teachers help
•	Develop a students character and mind
•	Instill an ambition to achieve and a love of life-long learning
•	Offer friendship to the students
•	Build student confidence, resilience, and determination
•	Encourage students to plan ahead, work with others, and stay on task
•	Emphasize the importance of healthy living

Slide 25
“Teachers who can create a  climate for learning and focus on relationship-building with their students will encourage their students to reach their future goals and dreams.” (ICLE, 2008)
“A curriculum is only as good as those who teach it.” (Alexander, 2009)

Slide 26
Pages Read in School and for Homework
Student Questionnaire, Grade 8 NAEP 2007 Reading
Two graphs show this correlation:
•	29% of Grade 8 students selected to take the NAEP Reading assessment reported that their teachers only asked them to read 5 or fewer pages in school and for homework.
•	The more pages Grade 8 students are required to read for school or homework (up to 20 pages), the higher their NAEP Reading test score.
•	The fewer pages Grade 8 students are required to read for school or homework (up to 20 pages), the lower their NAEP Reading test score.                                                          

Slide 27
The 15 Key Elements of EffectiveAdolescent Literacy Programs 
(Biancarosa & Snow, 2006)
What does the NAEP data tell us about 4 out of the 15 elements? 
Direct, Explicit Comprehension Instruction.
•	Effective adolescent literacy interventions must address reading comprehension.
•	Effective Instructional Principles Embedded in Content.
Motivation and Self-Directed Learning.
•	Students choose the books they want to read in school
Extended Time for Literac.y
•	Need 2 to 4 hours of literacy-connected learning daily

Slide 28
Direct, Explicit Comprehension Instruction
Teacher Questionnaire, Grade 8 NAEP 2007 Reading
Language Arts Classes
Ask students to explain or support their understanding of what they have read.
Ask students to make generalizations and draw inferences based on what they have read.
Ask students to describe the style or structure of the text they read.

Most-Advanced Students
•	Those students asked to do this task “more than once a week” scored higher than those asked to explain what they read “only once a month”
•	No Difference in Scores
•	Students that were asked to complete this task at least “once a month” scored higher than those “never or hardly ever” asked to describe the style of the text they read

Least-Advanced Students
•	No Difference in Scores
•	Students that were asked to complete this task at least “once a month” scored higher than those “never or hardly ever” asked to make generalizations based on what they read
•	Students that were asked to complete this task at least “once a month” scored higher than those “never or hardly ever” asked to describe the style of the text they read

Slide 29
Effective Instructional Principles Embedded in Content
Teacher Questionnaire, Grade 8 NAEP 2007 Reading
“Which best describes how language arts instruction is organized for 8th grade students at this school?”  (Choose only one option.)
•	Language arts is taught primarily as a discrete subject with little or no integration with instruction in other subjects.
•	Some language arts instruction is integrated with other subjects, and some language arts instruction is presented as a discrete subject [combined]. 
•	Language arts lessons are primarily integrated with instruction in other subjects.

Slide 30
Effective Instructional Principles Embedded in Content Language Arts Class Organization
Teacher Questionnaire, Grade 8 NAEP 2007 Reading
NAEP 2007 data shows that 65% of Grade 8 students selected to take NAEP Reading were in “Combined” language arts classes. (average scale score of 263).
The students in “Integrated” courses scored lower than their peers. (10% of Grade 8 students selected) (average scale score of 256).
The students in “Discrete” courses scored lower than their peers. (25% of Grade 8 students selected) (average scale score of 263).

Slide 31
Motivation and Self-Directed Learning
Teacher Questionnaire, Grade 8 NAEP 2007 Reading.
NAEP 2007 data shows that “least advanced” Grade 8 students scored significantly lower when asked to read “almost every day” even though they chose their books. (32%, average scale score of 261).

There is no difference in the average scale scores of those who are given time in class to read books they chose when asked to read “once or twice a month” (22%, average scale score of 265) or “once or twice a week.” (31%, average scale score of 264).

Students who “never or hardly ever” read in class were 15% of the respondents and had an average scale score of 262.

Slide 32
Extended Time for Literacy
Teacher Questionnaire, Grade 8 NAEP 2007 Reading
Total time teacher spends with one Grade 8 Language Arts class in typical week.
•	2 percent spend less than 3 hours, 
•	28 percent  spend between 3 and 4.9 hours, 
•	31 percent spend between 5 and 6.9 hours, 15 percent spend between 7 and 9.9 hours, and 4% spend at least 10 hours a week

Slide 33
Extended Time for Literacy
Teacher Questionnaire, Grade 8 NAEP 2007 Reading
Language Arts Instruction Time
•	1 to 10 percent of Language Arts instructors spend 9 percent of their time on Reading Skills and Strategies, 11 percent on Appreciation and Analysis of Literature, and 7 percent on Student Writing
•	11 to 40 percent of Language Arts instructors spend 46 percent of their time on Reading Skills and Strategies, 49 percent on Appreciation and Analysis of Literature, and 47 percent on Student Writing
•	41 to 60 percent of Language Arts instructors spend 25 percent of their time on Reading Skills and Strategies, 26 percent on Appreciation and Analysis of Literature, and 27 percent on Student Writing
•	61 to 90 percent of Language Arts instructors spend 13 percent of their time on Reading Skills and Strategies, 10 percent on Appreciation and Analysis of Literature, and 13 percent on Student Writing
•	91 + percent of Language Arts instructors spend 6 percent of their time on Reading Skills and Strategies, 4 percent on Appreciation and Analysis of Literature, and 5 percent on Student Writing

Slide 34
How to Use the NAEP Data
•	Within State Departments
•	To Support State Initiatives for Grant Proposals
•	For Teacher Professional Development
•	To Guide Policy Development
•	As an Integral Part of the Overall State Assessment Program

Slide 35
Opportunities in the 21st Century
“Education is no longer just a pathway to opportunity; it’s a prerequisite for success.”

President Barack Obama’s speech 
to a joint session of Congress 
February 24, 2009
http://www.cbsnews.com/stories/2009/02/24/politics/main4826494.shtml 

Slide 36
References
NAEP Data Explorer: http://nces.ed.gov/nationsreportcard/naepdata/ 
NAEP Questionnaires: http://nces.ed.gov/nationsreportcard/bgquest.asp
•	ACT. (2008). The forgotten middle: Ensuring that all students are on target for college and 	career readiness before high school. Iowa City: ACT.
•	Alexander, R.J. (2009) Towards a New Primary Curriculum: a report from the Cambridge Primary 	Review. Part 2: The Future. Cambridge: University of Cambridge Faculty of Education.
•	Biancarosa, C., & Snow, C. E. (2006). Reading next: A vision for action and research in middle 	and high school literacy. (2nd Ed.). Washington, D.C.: Alliance for Excellent Education.
•	International Center for Leadership in Education. (2008). Florida middle school mathematics 	initiative, institute 6. Elementary-Middle School Transition Student Engagement Initiative in 	Review.
•	Kinney, P. (October, 2007). A voice from the middle: Middle level students care about their 	education and their future. Principal Leadership, 35-36. 
•	National Association of Secondary School Principals & Phi Delta Kappa. (2007). A voice from the 	middle: Highlights of the 2007 NASSP/PDK middle school student poll. Bloomington, IN: 	NASSP/PDK. 
•	Prime Minister Brown. (2009). PM’s speech on education for the new global age. Retrieved from 	the internet June 18, 2009 http://www.number10.gov.uk/Page19209 
•	Rose, J. (2009). Independent review of the primary curriculum: Final report. Retrieved from the 	internet June 18, 2009 http://publications.teachernet.gov.uk/default.aspx?PageFunction=productdetails&PageMode=publications&ProductId=DCSF-00499-2009&  
•	Yecke, C. P. (2005). Mayhem in the middle: How middle schools have failed America and how to 	make them work. Washington, D.C.: Thomas B. Fordham Institute.

Slide 37
Thank you!
•	Wendy Geiger
Virginia NAEP Coordinator
•	Carrie L. Giovannone
Arizona NAEP Coordinator
•	Pam A. Sandoval
Colorado NAEP Coordinator
•	Michele Sonnenfeld
Florida NAEP Coordinator
Contact your State NAEP Coordinator for the latest NAEP results