Chapter 6 Test Administration
6.1 Introduction
“The usefulness and interpretability of test scores require that a test be administered and scored according to the developer’s instructions” (American Educational Research Association, American Psychological Association, & National Council on Measurement in Education, 2014, p. 111).
Smarter Balanced maintains several manuals and user guides with policies and guidelines for test administration that are available to members and their service providers. These are available on the Smarter Balanced portal, and include:
Member Procedures Manual: provides a high-level overview of the assessment system for member leadership, including core responsibilities that member states must assume in order to obtain Smarter Balanced test results that are generalizable across states, expected policies and procedures for administration, required trainings, a description of the evidence states must maintain and provide to Smarter Balanced annually, procurement information, and links to resource documents.
Implementation Guide for States and Service Providers: describes critical elements of Consortium policies and practices that states and service providers may consider in developing or responding to Requests for Proposals (RFP) regarding the Smarter Balanced Assessment System.
Summative Test Administration Manual: provides instructions and guidance for the Smarter Balanced online test delivery system that members customize to meet the needs of their specific state or territory.
Remote Summative Testing Implementation Guidance: intended to be used as a supplement to the member-customized Online Summative Test Administration Manual and includes additional protocols and procedures specific to the remote administration of the Smarter Balanced summative assessments.
6.2 Remote versus In-Person Administration
Beginning with the 2020-21 administration year, Test Administrators (TAs) may administer Smarter Balanced tests with the TA and the student(s) in different locations (i.e., remotely), consistent with the established policies for in-person test administrations. Members work with their service providers to determine whether and how the remote option will be offered in their state/territory.
6.3 Test Duration
Estimated testing times for the adjusted and full blueprint are provided in Table 6.1 for both the computer adaptive test (CAT) and the performance task (PT). For the adjusted blueprint, the CAT portion is expected to take about half the time as the full version, with the same amount of time needed for the PT portion.
| Blueprint | Subject | Grades | CAT (hours:minutes) | PT (hours:minutes) | Total (hours:minutes) |
|---|---|---|---|---|---|
| Adjusted | ELA/Literacy | 3 to 5 | 0:45 | 2:00 | 3:30 |
| 6 to 8 | 0:45 | 2:00 | 3:30 | ||
| HS | 1:00 | 2:00 | 4:00 | ||
| Mathematics | 3 to 5 | 0:45 | 1:00 | 2:30 | |
| 6 to 8 | 1:00 | 1:00 | 3:00 | ||
| HS | 1:00 | 1:30 | 3:30 | ||
| Full | ELA/Literacy | 3 to 5 | 1:30 | 2:00 | 3:30 |
| 6 to 8 | 1:30 | 2:00 | 3:30 | ||
| HS | 2:00 | 2:00 | 4:00 | ||
| Mathematics | 3 to 5 | 1:30 | 1:00 | 2:30 | |
| 6 to 8 | 1:30 | 1:00 | 3:00 | ||
| HS | 2:00 | 1:30 | 3:30 |
Table 6.2 shows the time students spent responding to test items within the CAT and PT sections of each grade-level test, according to data received from states participating in the 2020-21 summative administration. Times are based only on the states that used the Smarter Balanced test blueprints for 2020-21 and administered tests at the grades indicated in the table. They include only students who took the minimum number of test items specified for the session (CAT or PT). Times are the average, or mean, across all students included for the given test section, subject, and grade. Due to rounding, session times may not sum to total time.
| Blueprint | Subject | Grade | States CAT | States PT | CAT (hours:minutes) | PT (hours:minutes) | Total (hours:minutes) |
|---|---|---|---|---|---|---|---|
| Adjusted | ELA/Literacy | 3 | CA,DE,ID,VI | CA,DE,HI,ID,VI | 0:44 | 1:26 | 2:10 |
| 4 | CA,DE,ID,VI | CA,DE,HI,ID,VI | 0:46 | 1:34 | 2:21 | ||
| 5 | CA,DE,ID,VI | CA,DE,HI,ID,VI | 0:48 | 1:40 | 2:28 | ||
| 6 | CA,DE,ID,VI | CA,DE,HI,ID,VI | 1:06 | 1:35 | 2:42 | ||
| 7 | CA,DE,ID,VI | CA,DE,HI,ID,VI | 1:03 | 1:32 | 2:36 | ||
| 8 | CA,DE,ID,VI | CA,DE,HI,ID,VI | 1:02 | 1:30 | 2:33 | ||
| 11 | CA,ID,VI | CA,HI,ID,VI | 0:56 | 1:08 | 2:05 | ||
| Mathematics | 3 | CA,DE,ID,VI | CA,DE,ID,VI | 0:41 | 0:28 | 1:09 | |
| 4 | CA,DE,ID,VI | CA,DE,ID,VI | 0:43 | 0:28 | 1:12 | ||
| 5 | CA,DE,ID,VI | CA,DE,ID,VI | 0:46 | 0:42 | 1:28 | ||
| 6 | CA,DE,ID,VI | CA,DE,ID,VI | 0:49 | 0:39 | 1:29 | ||
| 7 | CA,DE,ID,VI | CA,DE,ID,VI | 0:49 | 0:26 | 1:16 | ||
| 8 | CA,DE,ID,VI | CA,DE,ID,VI | 0:52 | 0:31 | 1:23 | ||
| 11 | CA,ID,VI | CA,ID,VI | 0:47 | 0:29 | 1:17 | ||
| Full | ELA/Literacy | 3 | SD,VT | SD,VT | 1:17 | 2:16 | 3:33 |
| 4 | SD,VT | SD,VT | 1:18 | 2:20 | 3:38 | ||
| 5 | SD,VT | SD,VT | 1:15 | 2:10 | 3:25 | ||
| 6 | SD,VT | SD,VT | 1:22 | 1:50 | 3:12 | ||
| 7 | SD,VT | SD,VT | 1:19 | 1:50 | 3:09 | ||
| 8 | SD,VT | SD,VT | 1:15 | 1:47 | 3:02 | ||
| 11 | SD,VT | SD,VT | 1:08 | 1:27 | 2:35 | ||
| Mathematics | 3 | MT,SD,VT | MT,SD,VT | 1:03 | 0:26 | 1:30 | |
| 4 | MT,SD,VT | MT,SD,VT | 1:07 | 0:26 | 1:34 | ||
| 5 | MT,SD,VT | MT,SD,VT | 1:08 | 0:36 | 1:45 | ||
| 6 | MT,SD,VT | MT,SD,VT | 1:10 | 0:32 | 1:43 | ||
| 7 | MT,SD,VT | MT,SD,VT | 1:12 | 0:20 | 1:32 | ||
| 8 | MT,SD,VT | MT,SD,VT | 1:16 | 0:25 | 1:41 | ||
| 11 | SD | SD,VT | 1:02 | 0:26 | 1:29 |
6.4 Test Start Dates
Because of the COVID-19 pandemic, several states altered or extended their testing window for the 2020-21 administration year compared to previous years.
Table 6.3 shows the test start day. Day 1 is January 1, 2021; Day 2 is January 2, 2021; Day 46 is February 15, 2021; etc. Negative days are the number of days prior to January 1, 2021. States whose data are included in this table by grade are: Grades 3-8 — California, Delaware, Hawaii, Idaho, Montana, Nevada, Oregon, South Dakota, US Virgin Islands, Vermont; High School — California, Hawaii, Idaho, Oregon, South Dakota, US Virgin Islands, Vermont.
| Blueprint | Subject | Grade | Test Start Day Mean | Test Start Day Min | Test Start Day Max |
|---|---|---|---|---|---|
| Adjusted | ELA/Literacy | 3 | 122.0 | 53 | 209 |
| 4 | 120.2 | 47 | 207 | ||
| 5 | 119.9 | 53 | 202 | ||
| 6 | 122.5 | 52 | 199 | ||
| 7 | 120.4 | 46 | 208 | ||
| 8 | 119.2 | 47 | 207 | ||
| HS | 113.1 | -31 | 209 | ||
| Mathematics | 3 | 126.0 | 55 | 209 | |
| 4 | 127.3 | 47 | 208 | ||
| 5 | 126.0 | 55 | 208 | ||
| 6 | 125.7 | 62 | 210 | ||
| 7 | 124.3 | 46 | 206 | ||
| 8 | 124.4 | 47 | 207 | ||
| HS | 116.0 | -32 | 209 | ||
| Full | ELA/Literacy | 3 | 108.6 | 59 | 159 |
| 4 | 109.8 | 59 | 159 | ||
| 5 | 108.9 | 60 | 158 | ||
| 6 | 107.7 | 59 | 158 | ||
| 7 | 107.1 | 59 | 159 | ||
| 8 | 106.9 | 52 | 160 | ||
| HS | 104.7 | 66 | 159 | ||
| Mathematics | 3 | 112.5 | 59 | 157 | |
| 4 | 113.6 | 59 | 159 | ||
| 5 | 113.0 | 63 | 160 | ||
| 6 | 110.9 | 59 | 160 | ||
| 7 | 110.3 | 59 | 159 | ||
| 8 | 110.4 | 52 | 159 | ||
| HS | 106.4 | 66 | 160 | ||
| * Day 1 = January 1, 2021 |
6.5 Test Administration Manual
The Smarter Balanced Member Procedures Manual provides a high-level overview of the assessment system, including policy topics such as test security and test scheduling. In addition, this document describes the core responsibilities that member states must assume in order to obtain Smarter Balanced test results that are generalizable across states.
Specific instructions for member states to administer Smarter Balanced summative assessments are included in the Test Administrator User Guide (Smarter Balanced, 2016d); the Administration and Registration Tools (ART) User’s Guide (Smarter Balanced, 2016c); the Online Summative Test Administration Manual (TAM); the Paper Pencil Test Administration Manual for ELA (Smarter Balanced, 2018a); and the Pencil Paper Test Administration Manual for Mathematics (Smarter Balanced, 2018b). Specific components of these user guides and manuals require customization to meet unique needs in each member state. These components include:
- Help desk information
- Test expiration dates
- Administration and registration tools (ART) user roles
- State user roles
- Links to the posted materials and modules
- Test security/administration training policy
- Instructions for identifying and retrieving the classroom activity
- Role-specific checklists
The development of the Smarter Balanced Test Administration Manual was guided by the AERA, APA, and NCME 2014 Standards. In regard to test administration, the Standards provide guidance to test developers that the directions for test administration should be sufficiently clear to allow for standardized implementation in a variety of conditions (see Standard 4.15). In addition, the Standards provide guidance that test developers should provide sufficient detail so that test takers can respond to items and tasks in the manner intended by the test developer (see Standard 4.16).
6.6 Clear Directions to Ensure Uniform Administration
Smarter Balanced test administration manuals include instructions that clearly articulate various aspects of the administration process. The Online Summative Test Administration Manual (TAM) covers an extensive amount of material for events that occur before, during, and after testing. In addition, the Online Summative TAM points the user to training materials that further provide detail and clarity to support reliable test administration by qualified test administrators. The details provided in the Online Summative TAM describe the general rules of online testing, including pause rules; scheduling tests; recommended order of test administration; classroom activity information; assessment duration, timing, and sequencing information; and the materials that the test administrator and students need for testing. All work together to ensure uniform test administration conditions across Smarter Balanced member states.
Section 8 of the Online Summative TAM provides an overview of the universal tools, designated supports, and accommodations. All are further explicated in the Smarter Balanced Assessment Consortium Usability, Accessibility, and Accommodations Guidelines (Smarter Balanced, 2022d) and the Usability, Accessibility, and Accommodations Implementation Guide (Smarter Balanced, 2022e).
6.7 Detailed Instructions for Test Administrators and Test Takers
Section 10 of the Online Summative TAM provides step-by-step instructions for test administrators (TA) on how to start a test session, monitor a test session, and end a test session. Throughout the steps, Smarter Balanced has embedded scripts that TAs are instructed to read to students. Test administrators are instructed to strictly adhere to scripts, use professional judgment when responding to student questions, and refrain from reading test items, suggesting answers, or evaluating student work during testing. See Section 10 of the Online Summative Test Administration Manual for the script (Smarter Balanced, 2021b).
Smarter Balanced provides practice and training test packages for each grade and content area. The practice and training tests are available on each member’s assessment portal to familiarize students with how to navigate the online test delivery system and to allow practice with the item types and the functionality of the testing environment. The practice test resources ensure that students are able to answer the items and tasks in the manner intended by Smarter Balanced. Smarter Balanced also makes available a sample test for public access to the practice test items. The sample test is not intended to be used by students to prepare for testing.
In addition, Smarter Balanced developed nine training modules that address various topics and support processes that those involved in test administration will need to know. These modules are listed below and available online to members.
Module 1: What Is a CAT?
Module 2: Technology Requirements for Online Testing
Module 3: Performance Task Overview
Module 4: Student Interface for Online Testing
Module 5: Test Administrator Interface for Online Testing
Module 6: Administration and Registration Tools (ART)
Module 7: Accessibility and Accommodations
Module 8: Embedded Universal Tools and Online Features
Module 9: Test Administration Overview
6.8 Responsibilities of Test Administrators
The AERA, APA, and NCME Standards (2014) also provide guidance to test administrators and test users. Test administrators are guided to carefully follow the standardized procedures (Standard 6.1); inform test takers of available accommodations (Standard 6.2); report changes or disruptions to the standardized test administration (Standard 6.3); furnish a comfortable environment with minimal distractions (Standard 6.4); provide appropriate instructions, practice, and other supports (Standard 6.5); and ensure the integrity of the test by eliminating opportunities for test taker malfeasance (Standard 6.6). In addition, test users are responsible for test security at all times (Standard 6.7). To align with these guidelines, the Online Summative TAM provides:
- careful direction to TAs to strictly adhere to the directions in the TAM;
- available universal tools, designated supports, and accommodations;
- requirements of the test environment, including student seating, a quiet environment, and access to allowable universal tools, designated supports, and accommodations; and
- descriptions of testing improprieties, irregularities, and breaches.
Deviations from standardized online testing procedures (specifically testing irregularities and improprieties) are handled at the local and/or state level, per the guidelines in the Online Summative TAM. Depending on the nature and severity of the incident, a student’s test may be reset, invalidated, reopened, or restored. All such incidents must be reported by authorized administrators at the local and state levels (Smarter Balanced, 2022b).
6.9 Universal Tools, Designated Supports, and Accommodations
To enhance student access to the assessment content during test administration, Smarter Balanced developed a conceptual model that includes universal tools, designated supports, and accommodations (Smarter Balanced, 2016b). Universal tools are access features of the Smarter Balanced assessment that are either provided as digitally delivered components of the test delivery system (embedded) or provided separately from the test delivery system (non-embedded). Universal tools are available to all students based on student preference and selection. Embedded universal tools include (but are not limited to) features such as a “pause” function that allows the student to take a break of 20 minutes or less during the assessment; a digital calculator that the student may access by clicking on a calculator button; and a digital notepad. Non-embedded universal tools include (but are not limited to) provision of an English dictionary for the full-write portion of the ELA/literacy performance task and the provision of physical scratch paper for all content area tests.
Designated supports for the Smarter Balanced assessments are embedded and non-embedded features that are available for use by any student for whom the need has been indicated by an educator or team of educators (along with the student and his/her parent/guardian) familiar with the student’s instructional needs. Embedded designated supports include (but are not limited to) features such as color contrast, which enables students to adjust background or font color; translated test directions; translated glossaries; and stacked translations for mathematics items. Non-embedded designated supports include (but are not limited to) provision of color overlays, printing test content with different colors, use of magnification devices, and use of noise buffers.
Accommodations are changes in procedures or materials that increase equitable access during the Smarter Balanced assessments. Students receiving accommodations must have a need for those accommodations documented in an Individualized Education Program (IEP) or 504 accommodation plan. Like universal tools and designated supports, accommodations may be either embedded or non-embedded. Examples of embedded accommodations include (but are not limited to) closed captioning and test content translated into American Sign Language (ASL) video. Non-embedded accommodations include (but are not limited to) use of an abacus, print on demand, and use of an external communication device (speech-to-text). Universal tools, designated supports, and accommodations all yield valid scores that count as participation in assessments that meet the requirements of ESSA when used in a manner consistent with the Smarter Balanced Usability, Accessibility, and Accommodations Guidelines (Smarter Balanced, 2022d). A complete summary of all embedded and non-embedded universal tools, designated supports, and accommodations is included in the Usability, Accessibility, and Accommodations Guidelines.
Please see Chapter 3 for additional information about Smarter Balanced tools, designated supports, and accommodations.