Chapter 6 Reporting and Interpretation
6.1 Introduction
This chapter provides an overview of the Smarter Balanced reporting system. The use of the Smarter Reporting System is optional and configurable. Members can choose to utilize the system hosted by the Consortium, deploy their own instance of the open source system and adapt to their needs, or utilize a different system.
6.2 Types of Scores
As noted in Chapter 4 on test design, the ICAs provide an overall indicator of proficiency and a set of claim indicators corresponding to more specific areas of content (claims) within the broader content areas (ELA/literacy and mathematics). The ICAs are built on similar blueprints to the full-form summative assessments. They are designed to measure a similar range of content as the summative assessments and assess similar claims, targets, and standards. Information from IABs and FIABs is reported in the same manner as claim-level information on the ICAs and summative assessments: “above,” “near,” or “below” standard, where the standard is the level 3 cut score. The following section describes how overall performance is reported on an ICA.
It should be noted that the hand scoring of student answers to constructed-response items in interim assessments is a state vendor task and is not subject to the rigorous controls used in the summative assessment. Local results may show some variations from standardized hand-scoring results. Even when an interim is administered under standardized conditions, local hand scoring may not be standardized. Also, if an interim assessment is administered to students more than once, familiarity with test questions may affect student performance and the accuracy of results as related to past and future performance. For these reasons and the fact that teachers may view the items and student responses, the interims are not intended to be used for accountability purposes.
6.2.1 Overall Test Scores
Scale scores are the basic means of reporting a student’s overall performance on summative and ICA tests. These scores fall along a continuous vertical scale that increases across grade levels from approximately 2000 to 3000. Scores are used to describe an individual student’s current level of achievement. They can also be used to track growth over time (Martineau, 2016a, 2016b). The achievement of groups of students differentiated by administration year, district, school, classroom, or demographics is described by the group mean and standard deviation, the percent of students at each achievement level, and the percent of students at or above proficient, where the level 3 cut score is the lower boundary of proficient.
For the ICA, the Smarter Balanced reporting system communicates an individual student’s overall scale score in relation to achievement levels using graphics similar to Figure 6.1. By default, the system uses terms relating to the level 3 standard for the achievement levels (Did Not Meet, Nearly Met, Met, Exceeded), but members may customize them using terms such as “novice, developing, proficient, advanced” or others.
Figure 6.1: PORTRAYAL OF SCORE REPORTING LEVELS AND ERROR BAND
The achievement levels in Figure 6.1 and in Smarter Balanced reports are illustrated by colored bands that collectively extend from the lowest obtainable scale score (LOSS) to the highest obtainable scale score (HOSS). The LOSS (2265), HOSS (2802), and cut scores separating the achievement levels in Figure 6.1 are those for grade 8 mathematics.
ICA scale scores are reported with an error band based on the standard error of measurement (SEM) (refer to Chapter 2). In Figure 6.1, the overall score is 2647, which is in the category: Met Standard (Level 3). However, it should be noted that the score’s error band encompasses Exceeded Standard (Level 4). Smarter Balanced reporting provides information to help users understand the meaning of the error bands.
Smarter Balanced has developed a set of customizable achievement level descriptors for ELA/literacy and mathematics that are aligned with the state standards and Smarter Balanced assessment claims. The intent of these descriptors is to specify, in content terms, the knowledge and skills that students may display at four levels of achievement. The full set of customizable reporting achievement level descriptors are available to the public online (Smarter Balanced, 2017c, 2019a, 2019b).
6.2.2 IAB/FIAB Overall Scores and ICA Claim Scores
Student performance on IABs and FIABs and on claims of the ICAs are reported in terms of scale scores and their standard errors. In addition, student performance is classified into one of three reporting categories: “above,” “near,” or “below” standard. The standard is the level 3 cut score (the lower boundary of level 3 achievement level). Table 6.1 and Table 6.2 show the claims for which ICA (and summative) claim scores are reported in the subject areas of ELA/literacy and mathematics. Not showing in Table 6.2 is the fact that in mathematics, claims 2 and 4 are combined to produce a single subscore. In all other cases, a subscore corresponds to one and only one claim.
| Claim | Description |
|---|---|
|
Students can read closely and analytically to comprehend a range of increasingly complex literary and informational texts. |
|
Students can produce effective and well-grounded writing for a range of purposes and audiences. |
|
Students can employ effective speaking and listening skills for a range of purposes and audiences. At this time, only listening is assessed. |
|
Students can engage in research/inquiry to investigate topics and to analyze, integrate, and present information. |
| Claim | Description |
|---|---|
|
Students can explain and apply mathematical concepts and interpret and carry out mathematical procedures with precision and fluency. |
|
Students can solve a range of complex, well-posed problems in pure and applied mathematics, making productive use of knowledge and problem-solving strategies. |
|
Students can clearly and precisely construct viable arguments to support their own reasoning and to critique the reasoning of others. |
|
Students can analyze complex, real-world scenarios and can construct and use mathematical models to interpret and solve problems. |
Other than the level 3 cut score serving as the standard for the reporting categories (“above,” “below,” or “near” standard), achievement levels per se are not used in reporting ICA claim scores and IAB/FIAB overall scores.
The measurement error associated with ICA claim scores and IAB/FIAB overall scores is used to classify the student’s performance into the reporting categories. To classify a student’s performance into one of the three reporting categories, an error band is defined that extends 1.5 times the SEM of the student’s subscore above and below the student’s subscore. The subscore is located on the same reporting scale that is used to report the summative and ICA overall scores. Table 6.3 shows how the student’s performance is then classified based on where the standard (the level 3 cut score) sits with respect to this error band.
| Category | Description |
|---|---|
| Above Standard | The Level 3 cut score is below the error band. |
| Near Standard | The Level 3 cut score falls on or within the boundaries of the error band. |
| Below Standard | The Level 3 cut score is below the error band. |
Although IAB/FIAB overall scores and ICA claim scores are portrayed in Smarter Balanced reports using the three reporting categories, claim-level scale scores and SEMs are available to members in the data provided from their vendors. Members may use these in local reporting systems.
Reporting for IABs and FIABs is focused on communicating the block-level results for a list of students by assessment grade and for individual student reporting. Educators can review individual student responses to each interim item, including hand-scored or written responses. A single individual student report is provided for all the interim block assessments that a student has taken in a given academic year.
Educators, students, parents, and guardians may use the individual student reports as one indicator to understand a student’s achievement, progress toward mastery of the state standards, and attainment of the academic knowledge and skills required to be college and career ready. The report may provide context for a parent-teacher conference or, together with other instructional data, may help to identify areas for instructional focus.
ICA claim scores are reported as shown in Figure 6.2 below.
Figure 6.2: ILLUSTRATION OF ICA CLAIM-SCORE REPORTING ON INDIVIDUAL STUDENT REPORTS
6.3 Types of Reports
The Smarter Reporting System allows teachers and administrators to access detailed student performance information for interim and summative assessments. Authorized users of the system with assgined permission may be able to view student data at the district, school, and individual levels. Users may also view, download and print individual student reports.
6.3.1 Individual Student Reports
There is one type of printable report available for the interim assessments: individual student reports (ISRs). This type of report is available for summative assessments as well. The ISR will be described briefly here, but the reader is urged to consult the Smarter Balanced Reporting System User Guide for more detail. The ISRs present individual student assessment scores, SEMs, and achievement levels. They also display the reporting categories for claim-score results, as illustrated above. The scores and descriptions provide context for understanding what the assessment has measured and how to interpret the scores and subscores. The reports are designed for teachers, students, and parents to use in understanding a student’s achievement and progress toward mastery of the state standards. The report may be combined with other information that provides context for instructional focus. Individual student reports can be downloaded as PDF files for easy printing and distribution to parents.
6.3.2 Data Downloads
In addition to the predesigned reports, the Smarter Reporting System offers authorized users the ability to download data for distribution or further review and analysis in external systems. User authorization is closely controlled for ISRs and personally identifiable information (PII) in files. The list of available data downloads appears in Table 6.4.
| Download Type | Description |
|---|---|
| Student Assessment Results Export | This is a download of the assessment results for the selected assessment, with one row per student. The resulting files contain all of the data for the overall scale score, error band, achievement level, and claim scores for ICA and summative, as well as all the student data (e.g., demographics, grade/school/district/state attribution, etc.) for the specific summative or interim assessment being viewed. |
| Printable Student Reports | Printable versions of the individual student report. |
| Custom Aggregate Report Export | This is a download of a customized aggregate report of assessment results for a selected assessment type (summative, IAB, ICA), with one row per student. The resulting files contain all of the data for the overall scale score, error band, achievement level, and claim scores for ICA and summative, as well as all the student data (e.g., demographics, grade/school/district/state attribution, etc.) for the selected assessment type. |
| District/School Export | This is a bulk download of all assessment results for the selected school/district, with one row per student. The resulting files contain all of the data for the overall scale score, error band, achievement level, and claim scores for ICA and summative, as well as all the student data (e.g., demographics, grade/school/district/state attribution, etc.) for all assessments administered to date in the district/school. |