Chapter 6 Test Administration
6.1 Introduction
“The usefulness and interpretability of test scores require that a test be administered and scored according to the developer’s instructions” (American Educational Research Association [AERA], American Psychological Association [APA], & National Council on Measurement in Education [NCME], 2014, p. 111). Smarter Balanced maintains a customizable Test Administration Manual to ensure standardized test administration procedures and, thus, uniform test administration conditions for all students in Smarter Balanced member states (Smarter Balanced, 2018f). Version 3.0 of this manual is applicable to the 2018-19 summative administration. This chapter provides information about testing windows and testing times and presents an overview of the customizable Test Administration Manual. Brief descriptions of, and references to, other manuals and guidelines for test administration are also included. For complete and detailed test administration rules and procedures, please see the Member Procedures Manual (Smarter Balanced, 2018g).
6.2 Test Administration Window
Students in Smarter Balanced member states participated in the 2018-19 test administration when a specified percentage of the school year had occurred. Each state established a schedule for the administration of the Smarter Balanced summative assessments using a testing window, as defined below.
Grades 3–8
- Testing shall not begin until at least sixty-six percent (66%) of a school’s annual instructional days have been completed.
- Testing may continue up to and including the last day of school.
High School
- Testing shall not begin until at least eighty percent (80%) of a school’s annual instructional days have been completed.
- Testing may continue up to and including the last day of school.
Exceptions and updates to these rules are described in the Member Procedures Manual (Smarter Balanced, 2018g, pp. 26–27). Consortium members may elect to provide flexibility in establishing high school testing schedules that better align with their academic calendar and their students’ course enrollment, or to address conflicting testing requirements or technology resource constraints. The flexibility may include extending the grade 11 window to begin after 66% of instruction is completed by course or instructional calendar, as appropriate. States may elect to administer the high school assessment in grades other than grade 11 (e.g., grades 9, 10, or 12).
6.3 Duration and Timing Information
The scheduling recommendations for each of these components was provided in the 2018-19 Online Summative Test Administration Manual (TAM), available to members. Duration, timing, break/pause rules, and session recommendations vary for each content area and component. Table 6.1 provides estimated testing times based on the scheduling recommendations in the TAM.
| Subject | Grades | CAT (hours:minutes) | PT (hours:minutes) | Total (hours:minutes) |
|---|---|---|---|---|
| ELA/L | 3 to 5 | 1:30 | 2:00 | 3:30 |
| 6 to 8 | 1:30 | 2:00 | 3:30 | |
| HS | 2:00 | 2:00 | 4:00 | |
| Mathematics | 3 to 5 | 1:30 | 1:00 | 2:30 |
| 6 to 8 | 2:00 | 1:00 | 3:00 | |
| HS | 2:00 | 1:30 | 3:30 |
Table 6.2 shows the time students spent responding to test items within the computer adaptive test (CAT) and performance task (PT) sections of each grade-level test, according to data received from states participating in the 2018-19 summative administration. Times are based only on the states that used the Smarter Balanced test blueprints for 2018-19 and administered tests at the grades indicated in the table. They include only students who took the minimum number of test items specified for the session (CAT or PT). Times are the average, or mean, across all students included for the given test section, subject, and grade. Due to rounding, session times may not sum to total time.
| Subject | Grade | States CAT | States PT | CAT (hours:minutes) | PT (hours:minutes) | Total (hours:minutes) |
|---|---|---|---|---|---|---|
| English Language Arts/Literacy | 3 | CA,DE,HI,ID,OR,SD,VI,VT,WA | CA,DE,HI,ID,OR,SD,VI,VT,WA | 1:37 | 2:01 | 3:39 |
| 4 | CA,DE,HI,ID,OR,SD,VI,VT,WA | CA,DE,HI,ID,OR,SD,VI,VT,WA | 1:42 | 2:11 | 3:53 | |
| 5 | CA,DE,HI,ID,OR,SD,VI,VT,WA | CA,DE,HI,ID,OR,SD,VI,VT,WA | 1:42 | 2:09 | 3:52 | |
| 6 | CA,DE,HI,ID,OR,SD,VI,VT,WA | CA,DE,HI,ID,OR,SD,VI,VT,WA | 1:56 | 2:04 | 4:01 | |
| 7 | CA,DE,HI,ID,OR,SD,VI,VT,WA | CA,DE,HI,ID,OR,SD,VI,VT,WA | 1:40 | 1:57 | 3:38 | |
| 8 | CA,DE,HI,ID,OR,SD,VT,WA | CA,DE,HI,ID,OR,SD,VT,WA | 1:42 | 1:52 | 3:34 | |
| 11 | CA,HI,ID,OR,SD,VI,WA | CA,HI,ID,OR,SD,VI,WA | 1:24 | 1:28 | 2:53 | |
| Mathematics | 3 | CA,DE,HI,ID,OR,SD,VI,VT,WA | CA,DE,ID,OR,SD,VI,VT,WA | 1:25 | 0:46 | 2:12 |
| 4 | CA,DE,HI,ID,OR,SD,VI,VT,WA | CA,DE,ID,OR,SD,VI,VT,WA | 1:33 | 0:46 | 2:19 | |
| 5 | CA,DE,HI,ID,OR,SD,VI,VT,WA | CA,DE,ID,OR,SD,VI,VT,WA | 1:36 | 1:06 | 2:43 | |
| 6 | CA,DE,HI,ID,OR,SD,VI,VT,WA | CA,DE,ID,OR,SD,VI,VT,WA | 1:45 | 0:58 | 2:44 | |
| 7 | CA,DE,HI,ID,OR,SD,VI,VT,WA | CA,DE,ID,OR,SD,VI,VT,WA | 1:33 | 0:35 | 2:09 | |
| 8 | CA,DE,HI,ID,OR,SD,VT,WA | CA,DE,ID,OR,SD,VT,WA | 1:43 | 0:42 | 2:25 | |
| 11 | CA,HI,ID,OR,SD,VI,WA | CA,ID,OR,SD,VI,WA | 1:15 | 0:36 | 1:51 |
Table 6.3 shows the test start day. Day 1 is January 1, 2019; Day 2 is January 2, 2019; Day 46 is February 15, 2019; etc. Negative days are the number of days prior to January 1, 2019. States included in this table by grade were: Grades 3–8 — CA, DE, HI, ID, MT, NV, OR, SD, VT, WA; High School — CA, HI, ID, MT, OR, SD, VT, WA.
| Subject | Grade | Test Start Day Mean | Test Start Day Min | Test Start Day Max |
|---|---|---|---|---|
| ELA/Literacy | 3 | 120.2 | 52 | 189 |
| 4 | 120.2 | 51 | 192 | |
| 5 | 120.0 | 50 | 196 | |
| 6 | 119.9 | 50 | 191 | |
| 7 | 119.0 | 51 | 182 | |
| 8 | 118.8 | 26 | 190 | |
| HS | 108.2 | -34 | 180 | |
| Mathematics | 3 | 128.2 | 51 | 187 |
| 4 | 128.5 | 52 | 192 | |
| 5 | 128.4 | 59 | 196 | |
| 6 | 127.5 | 50 | 182 | |
| 7 | 125.8 | 59 | 183 | |
| 8 | 125.8 | 26 | 192 | |
| HS | 115.4 | -34 | 180 | |
| * Day 1 = January 1, 2019 |
6.4 Test Administration Manual
The Smarter Balanced Member Procedures Manual (Smarter Balanced, 2018g) provides a high-level overview of the assessment system, including expected policies and procedures for administration, required trainings, general information about the open-source platform, information about the evidence states must provide to Smarter Balanced annually, procurement information, and links to resource documents. This document provides the core responsibilities that member states must assume in order to obtain Smarter Balanced test results that are generalizable across states.
Specific instructions for member states to administer Smarter Balanced summative assessments are included in the Test Administrator User Guide (Smarter Balanced, 2016e); the Administration and Registration Tools (ART) User’s Guide (Smarter Balanced Assessment Consortium, 2016b); the Online Summative Test Administration Manual (TAM); the Paper Pencil Test Administration Manual for ELA (Smarter Balanced, 2018a); and the Pencil Paper Test Administration Manual for Mathematics (Smarter Balanced, 2018b). Specific components of these user guides and manuals require customization to meet unique needs in each member state. These components include:
- Help desk information
- Test expiration dates
- Administration and registration tools (ART) user roles
- State user roles
- Test security policy
- Links to where materials and modules are posted
- Test security/administration training policy
- Instructions for identifying and retrieving the classroom activity
- Role-specific checklists
The development of the Smarter Balanced Test Administration Manual was guided by the AERA, APA, and NCME 2014 Standards. In regard to test administration, the Standards provide guidance to test developers that the directions for test administration should be sufficiently clear to allow for standardized implementation in a variety of conditions (see Standard 4.15). In addition, the Standards provide guidance that test developers should provide sufficient detail so that test takers can respond to items and tasks in the manner intended by the test developer (see Standard 4.16).
6.5 Clear Directions to Ensure Uniform Administration
Smarter Balanced test administration manuals include instructions that clearly articulate various aspects of the administration process. The Online Summative Test Administration Manual (TAM) covers an extensive amount of material for events that occur before, during, and after testing. In addition, the Online Summative TAM points the user to training materials that further provide detail and clarity to support reliable test administration by qualified test administrators. The details provided in the Online Summative TAM describe the general rules of online testing, including pause rules; scheduling tests; recommended order of test administration; classroom activity information; assessment duration, timing, and sequencing information; and the materials that the test administrator and students need for testing. All work together to ensure uniform test administration conditions across Smarter Balanced member states.
Section 8 of the Online Summative TAM provides an overview of the universal tools, designated supports, and accommodations. All are further explicated in the Smarter Balanced Assessment Consortium Usability, Accessibility, and Accommodations Guidelines (Smarter Balanced, 2018c) and the Usability, Accessibility, and Accommodations Implementation Guide (Smarter Balanced, 2016).
6.6 Detailed Instructions for Test Administrators and Test Takers
Section 10 of the Online Summative TAM provides step-by-step instructions to test administrators (TA) on how to start a test session, monitor a test session, and end a test session. Throughout the steps, Smarter Balanced has embedded scripts that TAs are instructed to read to students. Test administrators are instructed to strictly adhere to scripts, use professional judgment when responding to student questions, and refrain from reading test items, suggesting answers, or evaluating student work during testing. See Section 10 of the Online Summative Test Administration Manual for the script (Smarter Balanced, 2018i).
Smarter Balanced provides practice and training test packages for each grade and content area. The practice and training tests are available on each member’s assessment portal to familiarize students with how to navigate the online test delivery system and to allow practice with the item types and the functionality of the testing environment. The practice test resources ensure that students are able to answer the items and tasks in the manner intended by Smarter Balanced. Smarter Balanced also makes available a sample test for public access to the practice test items. The sample test is not intended to be used by students to prepare for testing.
In addition, Smarter Balanced developed nine training modules that address various topics and support processes that those involved in test administration will need to know. These modules are listed below and available online to members.
Module 1: What Is a CAT?
Module 2: Technology Requirements for Online Testing
Module 3: Performance Task Overview
Module 4: Student Interface for Online Testing
Module 5: Test Administrator Interface for Online Testing
Module 6: Administration and Registration Tools (ART)
Module 7: Accessibility and Accommodations
Module 8: Embedded Universal Tools and Online Features
Module 9: Test Administration Overview
6.7 Responsibilities of Test Administrators
The AERA, APA, and NCME Standards (2014) also provide guidance to test administrators and test users. Test administrators are guided to carefully follow the standardized procedures (Standard 6.1); inform test takers of available accommodations (Standard 6.2); report changes or disruptions to the standardized test administration (Standard 6.3); furnish a comfortable environment with minimal distractions (Standard 6.4); provide appropriate instructions, practice, and other supports (Standard 6.5); and ensure the integrity of the test by eliminating opportunities for test taker malfeasance (Standard 6.6). In addition, test users are responsible for test security at all times (Standard 6.7). To align with these guidelines, the Online Summative TAM provides:
- careful direction to TAs to strictly adhere to the directions in the TAM;
- available universal tools, designated supports, and accommodations;
- requirements of the test environment, including student seating, a quiet environment, and access to allowable universal tools, designated supports, and accommodations; and
- descriptions of testing improprieties, irregularities, and breaches.
Deviations from standardized online testing procedures (specifically testing irregularities and improprieties) are handled at the local and/or state level, per the guidelines in the Online Summative TAM. Depending on the nature and severity of the incident, a student’s test may be reset, invalidated, reopened, or restored. All such incidents must be reported by authorized administrators at the local and state levels (Smarter Balanced Assessment Consortium, 2018g).
6.8 Universal Tools, Designated Supports, and Accommodations
To enhance student access to the assessment content during test administration, Smarter Balanced developed a conceptual model that includes universal tools, designated supports, and accommodations (Smarter Balanced, 2016c). Universal tools are access features of the Smarter Balanced assessment that are either provided as digitally delivered components of the test delivery system (embedded) or provided separately from the test delivery system (non-embedded). Universal tools are available to all students based on student preference and selection. Embedded universal tools include (but are not limited to) features such as a “pause” function that allows the student to take a break of 20 minutes or less during the assessment; a digital calculator that the student may access by clicking on a calculator button; and a digital notepad. Non-embedded universal tools include (but are not limited to) provision of an English dictionary for the full-write portion of the ELA/literacy performance task and the provision of physical scratch paper for all content area tests.
Designated supports for the Smarter Balanced assessments are embedded and non-embedded features that are available for use by any student for whom the need has been indicated by an educator or team of educators (along with the student and his/her parent/guardian) familiar with the student’s instructional needs. Embedded designated supports include (but are not limited to) features such as color contrast, which enables students to adjust background or font color; translated test directions; translated glossaries; and stacked translations for mathematics items. Non-embedded designated supports include (but are not limited to) provision of color overlays, printing test content with different colors, use of magnification devices, and use of noise buffers.
Accommodations are changes in procedures or materials that increase equitable access during the Smarter Balanced assessments. Students receiving accommodations must have a need for those accommodations documented in an Individualized Education Program (IEP) or 504 accommodation plan. Like universal tools and designated supports, accommodations may be either embedded or non-embedded. Examples of embedded accommodations include (but are not limited to) closed captioning and test content translated into American Sign Language (ASL) video. Non-embedded accommodations include (but are not limited to) use of an abacus, print on demand, and use of an external communication device (speech-to-text). Universal tools, designated supports, and accommodations all yield valid scores that count as participation in assessments that meet the requirements of ESEA when used in a manner consistent with the Smarter Balanced Usability, Accessibility, and Accommodations Guidelines (Smarter Balanced, 2018c). A complete summary of all embedded and non-embedded universal tools, designated supports, and accommodations is included in the Usability, Accessibility, and Accommodations Guidelines.
Please see Chapter 3 for additional information about Smarter Balanced tools, designated supports, and accommodations.