Slide 1 Civics End-of-Course Assessment (EOCA) Overview
Dr. Stacy Skinner, Social Studies Coordinator, 
Test Development Center
Michael DiPierro, Social Studies Education Specialist, Bureau of Standards and Instructional Support   
Mike: Welcome and thank you for attending. My name is Michael DiPierro and I am joined by Dr. Stacy Skinner. I am the Social Studies Education Specialist and Dr. Skinner is the Social Studies Coordinator at the Test Development Center. 
Dr. Skinner:  
•	Thank you.  
•	I appreciate the opportunity to participate in this professional learning opportunity.
Mike:
The content provided in this presentation is informational and is not intended to replace or supersede local guidance. Additionally, the information provided is current as of today, 9/14/16.

Slide 2 Desired Outcomes
Locate information and resources related to the teaching and learning of M/J Civics EOCA courses. 
Understand and apply content-specific skills that may support student performance with curricula associated with these courses. 
Mike: The first part of this presentation includes information and resources related to courses corresponding to the Civics End-of-Course Assessment.  The second portion of this presentation will highlight some content-specific strategies.
Slide 3 Course Descriptions and Standards
Mike: CPALMS is the State of Florida’s official source for standards information and course descriptions. For the most current course information, be sure that you are on the 2015 – And Beyond tab as seen in this screen capture.  
Slide 4 Course Standards and Coding Scheme 
Mike: When viewing a course standards you will see additional information. Information related to this standard includes Access Points, Related Courses, Related Resources, Remarks/Examples, and Related Resources. You can also print, export, and share. 

Slide 5 1003.4156 General requirements for middle grades promotion. - 
(c) Three middle grades or higher courses in social studies. Beginning with students entering grade 6 in the 2012-2013 school year, one of these courses must be at least a one-semester civics education course that includes the roles and responsibilities of federal, state, and local governments; the structures and functions of the legislative, executive, and judicial branches of government; and the meaning and significance of historic documents, such as the Articles of Confederation, the Declaration of Independence, and the Constitution of the United States. Beginning with the 2013-2014 school year, each student’s performance on the statewide, standardized EOC assessment in civics education required under s. 1008.22 constitutes 30 percent of the student’s final course grade. 
http://www.fldoe.org/academics/standards/subject-areas/social-studies.stml 
Mike: As seen in Florida Statute 1003.4156,  at least a one-semester civics education course is required for middle grades promotion and the Civics End of Course Assessment will count for 30 percent of the students final course grade. 

Slide 6 1008.22 Student assessment program for public schools. -  

(3)(b)End-of-course (EOC) assessments.—EOC assessments must be statewide, standardized, and developed or approved by the Department of Education as follows:
(3)(b)1. EOC assessments for Algebra I, Geometry, Algebra II, Biology I, United States History, and Civics shall be administered to students enrolled in such courses as specified in the course code directory.
http://www.fldoe.org/academics/standards/subject-areas/social-studies.stml 
The Civics EOCA was field tested in 2013.
The baseline administration of the Civics EOCA was in 2014. 
Dr. Skinner: 
•	Assessment and Accountability are outlined in the Florida Statute’s K-20 Education Code.
•	Specifically, Chapter 1008 Section 22 provides legislation about the student assessment program for public schools including end of course assessments correlated to social studies standards-based curricula for middle school civics and high school U.S. History.
Slide 7 NGSSS EOC Assessments Fact Sheet 
Mike: The images displayed are from the NGSSS EOC Assessment Facts Sheet. As you can see, this document provides information on courses corresponding to the United States History End-of-Course Assessment, the test administration schedule, and the additional information related to NGSSS EOC assessments. 
Dr. Skinner:
•	FDOE’s Office of Assessment Administration updates this document at least annually preceding each Spring test administration for the subsequent administration year. 
•	It is important to access this document online to ensure access to the most current information.
•	School-district and school-site assessment contacts may offer additional, specific insights as to testing administration policies and procedures in any particular school district.
Slide 8 State EOC Assessment Results 
Mike: Results from past End of Course Assessment administrations can be found on the Results section of the FDOE End-of-Course (EOC) Assessments webpage. 
From the main web page, select Results from the menu on the left side of the screen.
After the Results web page opens, select the test administration year of the results you wish to view.  For example, to view results associated with the Spring 2016 administration, click 2016.
Click or scroll to view Social Studies Assessments, and then select the particular results you wish to view.
You may have to scroll through the page to find the information you are looking for because Civics is not the only assessment found on this page. 
Slide 9 Content Focus Reports
Dr. Skinner: 
•	Content Focus Reports can be accessed in the What’s on the Assessment section of the FDOE End-of-Course (EOC) Assessments webpage. 
•	Content Focus Reports are generated after reporting is completed for each assessment administration. 
•	Because some reporting categories have too few test items to report at the student level, 
•	Content Focus Reports should NOT be used to make decisions about instruction at the individual student level. 
Slide 10 Achievement Level Descriptions
Dr. Skinner: 
•	The Achievement Level Descriptions, or ALDs, for Florida EOC Assessments outline the specific student expectations for each of the five Achievement Levels for each EOC assessment. 
•	The content of each statewide assessment is organized by reporting categories that are used for test design, scoring, and reporting purposes, and the ALDs express what students at each Achievement Level know and can do for each reporting category.
 
Slide 11EOCA – Item Specs 
Mike: The Civics End-of-Course Test Item Specifications contains a wealth of information to support the teaching and learning of related courses as seen in the screen capture of its table of contents. In the next few slides we will highlight some information from this document.
Slide 12 EOCA –Test Item Specifications  
Dr. Skinner:
•	The Civics End-of-Course (EOC) Assessment Test Item Specifications were published in 2012 and remain available on FDOE’s End-of-Course (EOC) Assessments web site.
•	This document provides details about the portion of the standards assessment designed to assess Civics and includes information about the benchmarks, the stimulus types, and the test items.
•	The Florida Department of Education (FDOE) and committees of experienced Florida educators developed and approved the Specifications., a resource that defines the content and format of the test and test items for item writers and reviewers. 
•	The Specifications indicates the alignment of test items with the Next Generation Sunshine State Standards adopted by the Florida State Board of Education in 2008.
Slide 13 Cogitative Complexity 
Mike: On the Civics EOCA, cognitive complexity is based upon Dr. Norman L. Webb’s Depth of Knowledge (DOK) levels. The categories are low complexity, moderate complexity, and high complexity and form an ordered description of the demands an item may make on a student. For example, low-complexity items may require a student to solve a one-step problem. Moderate-complexity items may require multiple steps. High-complexity items may require a student to analyze and synthesize information. 
Notice the Percentage of Points by Cognitive Complexity Level. It is important that students are exposed to course content at each level of complexity. 
Dr. Skinner:
Additionally, as outlined in the Cognitive Complexity section of the Civics EOCA Test Item Specifcations, 
•	The rationale for classifying a test item by its DOK level of complexity focuses on the expectations made of the test item, not on the ability of the student. 
•	When classifying a test item’s demands on thinking (i.e., what the test item requires the student to recall, understand, analyze, and do), it is assumed that the student is familiar with the basic concepts of the task. 
•	Test items are chosen for the EOC assessments based on the NGSSS and their grade-level appropriateness, but the complexity of the test items remains independent of the particular curriculum a student has experienced. 
Slide 14 EOCA - Cognitive Complexity 
Dr. Skinner:
•	Let’s take a look at the items provided on pages 9 – 11 of the Test Item Specifications to clarify some examples of cognitive complexity levels.  
•	And let’s review those items using the Examples of Civics Activities across Cognitive Complexity Levels chart provided on page 12 of the Test Item Specifications.
•	The item presented on page 9 asks, “In which document are these statements found?”
•	This item calls on the student to recognize information presented in the historical document that appears as this item’s stimulus attribute.
•	Using a graphic to recall or recognize information is a low complexity task.
•	Before categorizing an item as “recall,” it is important for the item writer and/or review to explicitly identify what is being recalled.
•	For this item, the student is called on to recall the name of a historical document based on recognition of the excerpts presented.
•	On page 10 of the Civics EOCA Test Item Specifications we find a moderate complexity item.
•	This item asks, “How did the opinions expressed in these statements help influence the American political system in the United States?”
•	Typically, when an item asks “How?” the student is called on to describe a process.
•	When asked “Why?” students should often be prepared to explain a reason.
•	This item is essentially asking how one historical event (grievances stated in the Declaration of Independence) caused another historical event (influence on the American political system).
•	In doing so, this item calls on the student to identify the significance of historical actions and concepts. 	
•	Because this thought pattern engages the student in this level of mental processing, it is a moderate complexity item.
•	Page 11 offers a high complexity item.
•	High complexity items call on the student to engage in more abstract reasoning, planning, analysis, judgment, and creative thought. 
•	High complexity items require abstract and sophisticated thinking, often involving multiple steps. 
•	By asking, “How does the modern political system in the United States uphold the principles expressed in these statements?” this item calls on the student to generalize or draw a conclusion when presented with historical information, excerpts from the Declaration of Independence.	
•	Note the growth of cognitive engagement through these sample cognitive complexity item.  
•	To respond to the high complexity item, students must first recognize the historical document (a low complexity task), 
•	and then correlate the grievances to the American political system (a moderate complexity task)
•	before selecting an appropriate response to the high complexity item.
Slide 15 EOCA – Definitions of Benchmark Specifications
Dr. Skinner:
•	On this slide, we see page 16 of the Test Item Specifications, the Definitions of Benchmark Specifications.
•	The Individual Test Item Specifications identify how Florida’s NGSSS benchmarks are assessed on the Civics EOCA. 
•	For each benchmark assessed in Civics, information is provided in each Individual Benchmark Specifications section as defined on page 16.
Slide 16 EOCA – Benchmark Specifications
Dr. Skinner:
•	Individual Benchmark Specifications sections include the Standard, Benchmark, and Benchmark Clarifications.
•	There are a variety of implications exist regarding how to implement the Test Item Specifications as an instructional resource when planning, teaching, and assessing Civics curricula.
•	One approach includes understanding
•	the BENCHMARK as WHAT we teach,
•	the STANDARD as WHY we teach the BENCHMARK,
•	and the BENCHMARK CLARIFICATIONS as pathways to how once could teach a BENCHMARK
•	This approach supports standards-based instruction to avoid the problematic practice of teaching benchmarks in isolation.
•	
Slide 17Civics Item Rating Form 
Dr. Skinner: 
•	In addition to the information shared thus far, the Test Item Specifications include Appendices.
•	Here, we see a page from Appendix A – Directions for Item Review.
•	Appendix A offers a series of questions designed to assist in evaluating the quality of the Civics EOC Assessment items.
•	Each question correlates to one or more segments of the Test Item Specifications addressed earlier in this presentation.  This correlation promoted evidence-based decision making during an item review process.
•	On page A-2, we find the CIVICS END-OF-COURSE ASSESSMENT ITEM RATING FORM shown on this slide.
•	Each rating category correlates to a question on page A-1 and its answer’s corresponding evidence within the Test Item Specifications.
•	This is the form used by the Content Expert and Item Review Committee when evaluating items for the Civics EOCA.
•	This form can also be used to review alignment to the Test Item Specifications for any test item resources. 
Slide 18 Test Item Specifications – Appendices
Mike:
•	Additional appendices in the Test Item Specifications offer information about:
•	the Content Assessed by Civics End-of-Course Assessment, 
•	a Civics Item Writer Glossary, 
•	It is important to remember that this glossary is ONLY a reference list provided for the Item Writers and is not intended to comprise a comprehensive vocabulary list for students.
•	This reminder also provides an opportunity to remind our audience that Content Focus terms presented in Individual Item Specifications are given in addition to those found in the standards, benchmarks, and benchmark clarifications. Content Focus terms may include, but are not limited to, the terms listed.
•	and, as seen on CPALMS, Course Descriptions for Civics courses.
Slide 19 Civics EOCA – Reporting Categories 
Reporting Category 1 (25%)
Origins and Purposes of Law and Government
Reporting Category 2 (25%)
Roles, Rights, and Responsibilities of Citizens
Reporting Category 3 (25%)
Government Policies and Political Processes
Reporting Category 4 (25%)
Organization and Function of Government
Dr. Skinner: 
•	In addition to specifying the duration and length of test as noted on the aforementioned Fact Sheet,
•	APPENDIX D presents the CIVICS END-OF-COURSE ASSESSMENT TEST DESIGN SUMMARY
•	On page D-2 we find the Reporting Categories for the Civics EOCA.
Mike:
•	Civics EOCA Reporting Categories are uniquely aligned to the four Civics Standards in the Civics curriculum.
Dr. Skinner:
•	That’s right, Mike.  And each Reporting Category is equally weighted for scoring and reporting purposes.
Slide 20 Sample Questions for NGSSS EOCA
Mike:
•	We noted that each Individual Benchmark Specification includes a Sample Item.
•	The sample items are presented in a format similar to the one used in the test. 
This is a sample question that can be found in the test item specifications book on page 21 and on the online practice test. We are showing you a sample item with some questions to consider for a few reasons. First, these questions to consider, can generate discussion within a Professional Learning Community about the content and skill demands of a test item. Second, by using these questions as a guide, teachers can deliberately plan instruction to support the content knowledge and skills demanded on the end of course assessment. Last, it illustrates the cognitive demand an item, which can be used to reflect on the level of instruction taking place in the classroom. Ultimately, think about ways to align course standards, instruction, and assessment. 
Slide 22 Resources for Locating Sources and Stimulus 
DocsTeach, National Archives
https://www.docsteach.org/ 
Document Analysis Worksheets
https://www.archives.gov/education/lessons/worksheets/ 
Google Image Search – “Labeled for Reuse”  
https://images.google.com/ 
Library of Congress 
https://www.loc.gov/
TeachingHistory.org
http://teachinghistory.org/ 
Mike: Displayed are some resources for locating sources and stimulus attributes. I seek out sources in the public domain when developing resources. The document analysis worksheets are a great way to build student capacity to analyze documents, but keep in mind that these worksheets are not permitted on the EOCA. We must use these resources as appropriate for our students and work to develop the habits of mind in our students so that they can work independently. 
Slide 23 and 24 Political Cartoon Analysis 
Mike: One strategy to help students independently analyze sources is the WAVES strategy. This method helps students break a political cartoon down into individual parts in order to draw conclusions about the cartoon. By examining the Words, Actions, Visuals, and Emotions in an image, students are provided a framework for deconstructing the information provided and putting it back to gather to determining its significance. 
Over the years I have seen different versions of the WAVES strategy used in Broward County classrooms, for example, I have seen “so what?” replace significance, however, I used it as presented here. This strategy can be used with photographs as well.
Slide 25 Document Analysis 
Mike: SOAPStone and APPARTS are two strategies to help students analyze documents. You can see some of the over lap and differences in these strategies. For example, some similarities include: speaker/author, audience/audience, purpose/reason, where as addressing the tone is unique to SOAPStone. 
Dr. Skinner: 
When identifying the audience associated with any particular primary source, it may be beneficial to identify BOTH the authoring entity’s Intended audience as well as any collateral audiences.
Social studies curricula call on teaching and learning that probes not only the cause and consequence of events in our nation, but also the course of actions that resulted from a cause and how those actions led to a consequence.  It is in that investigation that we may find unintended or collateral audiences that influenced events and outcomes.
Slide 26 Document Analysis 
Dr. Skinner:
The AUTHOR is identified as Thomas Paine.  I don’t remember him as a Founder in so far as signing the Declaration but based on this piece, he at least intended to influence because the TIME is 1776.
Mike:
And why is 1776 important?
Dr. Skinner:
The year is associated with the Declaration of Independence.
I also have a thought about SIGNIFICANCE.
Colonists suffered constraints (to say the least) under British rule.  This document appears authored by someone who (to my knowledge) was NOT an elected official but was still allowed to express his opinion.
Slide 27 Categorization 
PERSIA and SPRITE are strategies to help students gain a deeper understanding of their discipline specific vocabulary, by associating terms with one or more categories. 
Slide 28 Categorization Practice  
PERSIA and SPRITE can be used at various complexity levels. For example, at the low level, students may be tasked with identifying economic terms associated with the rule of law. At the moderate level students may be asked to explain how intellectual causes lead to political outcomes. For example, may be asked to describe how Enlightenment ideas are evident in modern day applications of the United States Constitution. Additionally, students can complete a word sort activity, where they are tasked with categorizing words based on PERSIA or SPRITE along with providing justification their categorizations. Students may notice that some words can be categorized into more than one category, which provides teachers the opportunity to have students explain their thinking. For example, think about how students can justify the First Amendment as Political, Social, and Religious. At the high complexity level, students can use the terms in each category to draw conclusions about an event or  concept. For example, students can use the terms classified as intellectual to draw conclusions about the ideas in the Founding Documents. 
Note that the word sort activity can be done with or without providing students categories. With out categories is an open sort and with categories, such as  PERSIA or SPRITE is an open sort. 
Slide 29 FDOE Resource – Florida Students  
The Florida students website has a wide range of resources organized by the content standards. 
Slide 30 FDOE Resource – Toolkits
Dr. Skinner: The ePAT items are sample items from the Test Item Specifications.
FDOE Resource - ePATs
Slide 31 Computer-Based Practice Tests (ePATs) 
Dr. Skinner:
An online practice tests (ePATs) is currently available for the Civics EOCA.  ePAT items are Sample Items included in the Test Item Specifications.
Slide 32 Reflection
Which instructional resources can I utilize to support standards-based instruction?
How do I implement instructional resources to plan, teach, and assess my course’s standards-based curriculum?
Mike: As you start to reflect on the information provided consider these questions. Which instructional resources can I utilize to support standards-based instruction? How do I implement instructional resources to plan, teach, and assess my course’s standards-based curriculum?
Slide 33 For more information, contact
Michael DiPierro, Social Studies Education Specialist
Phone: 850-245-9773
Michael.DiPierro@fldoe.org 
Mike, thank you for attending this presentation.