Data Gap Analysis Final Report (Example)

Download as pdf or txt
Download as pdf or txt
You are on page 1of 66

180 Grand Avenue, Suite 995

Oakland, California 94612


Tel. 510 444 0400 Fax 510 444 5855
PublicConsultingGroup.com

Washington State KK-12 Education


Data Gap Analysis

June 2010
Washington State K–12
K Education
Data Gap Analysis

CONTENTS

Executive Summary................................
............................................................................................................................
............................ 3
Methodology ................................
................................................................................................................................
.......................................... 3
Summary Recommendations ................................
................................................................................................
................................................. 3

Introduction: Background and Purpose of the Project ................................................................


......................................... 7

Methodology ................................
................................................................................................................................
................................... 10
Stakeholder Interviews................................
................................................................................................
......................................................... 10
Washington Metadata Workbook ................................................................................................
........................................ 11
National Education Data Model ................................................................................................
........................................... 13
NEDM 2.0 ................................................................
................................................................................................
............................................. 13

Connection to Research and Policy Questions ................................................................


..................................................... 15

Gap Analysis ................................................................


................................................................................................
.................................... 17
Analysis of ESHB 22611 Expectations and Gaps ................................................................
..................................................... 17
Analysis of American Recovery and Reinvestment Act Expectations and Gaps .................................. ................................ 26
Analysis of Data Dictionary Gaps ................................................................................................
.......................................... 29
EDFacts Granular Data Gaps ................................
................................................................................................
................................................ 30

Research and Policy Questions Gaps ................................................................................................


................................... 32

Summary Recommendations................................
................................................................................................
............................................ 52

APPENDIX ................................................................
................................................................................................
........................................ 55
A. Excerpts from ESHB 2261 ................................
................................................................................................
................................................. 55
B. List of Interviewees ................................
..........................................................................................................................
.......................... 60
C. Data System Gap Analysis Project Description ................................................................
................................................ 61
D. Inventory of Existing Data Sources ................................................................................................
.................................. 62

June 2010 Page 1


Washington State K–12
K Education
Data Gap Analysis

TABLES
Table 1. Twelve Components of the Washington State Data System ..........................................................
................................ 7
Table 2. Tasks of the Data Governance Group .............................................................................................
............................. 8
Table 3. Washington Metadata Workbook Description and Contents.......................................................
................................ 11
Table 4. NEDM Gaps – Specific Data Element List ................................................................
...................................................... 19
Table 5. Example Key Performance Indicator Model for At Risk Students .................................................
................................ 28
Table 6. NEDM Data Element Gaps by Entity .............................................................................................
............................. 29
Table 7. EDFacts Data Element Gaps by Entity ...........................................................................................
........................... 30
Table 8. EDFacts Gaps – Specific Data Element List ................................................................
.................................................... 30
Table 9. Count of Research
esearch and Policy Questions Gaps by Category .........................................................
................................ 32
Table 10. Research and Policy Questions Gaps ..........................................................................................
.......................... 33
Table 11. Summary Recommendations ................................................................................................
...................................... 52

Page 2 June 2010


Washington State K–12
K Education
Data Gap Analysis

EXECUTIVE SUMMARY
In 2009, the Washington State Legisla
Legislature established a vision for a comprehensive K–12
12 education
improvement data system. The overall intent of this system is to provide Washington stakeholders with
information that addresses critical questions about student progress and the quality and costs
cost of
education in the state of Washington. The system should also incorporate data that allow the state to
address the state’s prioritized research and policy questions.

To assist with the design and operation of the data system, the Legislature created a Data Governance
Group within the Office of Superintendent of Public Instruction (OSPI) with responsibility for
implementing key tasks with consultant support. Steps included: 1) the identification of a priority list of
research and policy questions the sta
state
te data system should provide educators with the capacity to
address; 2) a gap analysis comparing the current status of the state’s data system with the information
needs associated with the research and policy questions, the legislative expectations in ESHB
ES 2261, and
the data system requirements in the federal American Recovery and Reinvestment Act of 2009 (ARRA);
and 3) a technical capabilities gap analysis at the classroom level to help ensure that data from the
state’s statewide longitudinal data system are accessible to key stakeholders including principals,
teachers, and other district leaders.. OSPI contracted with PCG Education to assist in implementing these
critical tasks.

Methodology
PCG Education’s methodology for identifying the data system gaps included the following components:

• Interviews with 34 stakeholder group representatives identified by OSPI. The interview process
provided an overall view of the data collected and available throughout the department. The
interviewees were asked questions on the sources and uses of data, specific key questions those
individuals have been asked but are unable to address due to lack of data or data connections,
and validation of existing documented metadata
metadata.
• Development of Washington Metadata Workbook designed d to capture metadata about the
appropriate people, systems, data items, and data dictionary elements necessary for the gap
analysis. The workbook provided the normative list of data elements, or data dictionary, across
the enterprise from which data requi
requirements
rements and availability were compared.

Summary Recommendations
Discussions with OSPI data managers and well as key state stakeholders interviewed through the
Research and Policy Questions portion of the project revealed a consistent focus on the need and an desire
for the ability to collect, retrieve, and analyze quality data in order to guide instruction and improve
student achievement as well as meet the reporting requirements of the state legislature and federal
government. To do this will require consol
consolidation
idation of many of the agency’s disparate data collections into
a comprehensive longitudinal data system. This comprehensive data system, along with a rigorous and

June 2010 Page 3


Washington State K–12
K Education
Data Gap Analysis

structured metadata documentation process, will allow for uniformity in definition, standards,
standard and use.
Washington has a robust student data collection system in CEDARS but no data warehouse or reporting
solution. Washington is currently in the process of releasing an RFP to procure and develop the data
warehouse in accordance with state requirem
requirements
ents and vision specified in their successful 2009 State
Longitudinal Data System (SLDS) grant award.

The following table displays recommendations gathered and synthesized through the data gap analysis
and validated against the data dictionary.

Summary Recommendations
ID Recommendation / Gap Discussion
1 Use the SharePoint workbook created OSPI and ERDC now have a significant resource available
through this project as the common data through the metadata mapping contained in the
dictionary to guide development of the OSPI Workbook. Both agencies would benefit from the
K-12 and ERDS P-2020 SLDS data warehouses continued development of the workbook and data
and data marts. roadmap.

2 Enable valid teacher effect calculations Although Washington is moving ahead with plans to
based on student growth percentiles
percentiles. implement a student growth model based on the Colorado
Student Growth Percentile approach,, include explicit plans
to link to teacher for the purpose of providing additional
additio
insights and evaluation models supported in Race to the
Top.
2.1 Calculate and load student growth Include in data warehouse in order to expose
xpose to reporting
percentile into CEDARS data warehouse capabilities once built.
once built
2.2 Establish section entrance and exit for Currently course attendance is snapshot based.
class roster in CEDARS. Class schedule by
course by date.
2.3 Create Current, Prior Year 1 assessment Support longitudinal growth structure recommended by
score growth. NEDM.

3 Develop student drop-out


out / early warning Washington is examining this issue through the Building
prevention and reporting module using the Bridges Workgroup.
rkgroup. Incorporation of at risk factors in a
ABC indicators recommended in the NGA state longitudinal data system offers distinct advantages
report (Absence, Behavior, Course Grade, over local systems for understanding risk at the state level.
and Over Age for Grade) Washington should examine drop-out out early warning
systems in the context of response to intervention
tervention and
positive behavior solutions to provide the necessary
support for at risk students.
3.1 Collect student and incident level This was a theme echoed consistently throughout the
discipline data through CEDARS. project in order to establish critical cross linkage of data and
answer Research and Policy questions of interest.

Page 4 June 2010


Washington State K–12
K Education
Data Gap Analysis

3.2 Improve student attendance attributes OSPI has the foundation in place to collect count of days
to enable accurate accounting of student attended but lacks the ability to determine an excused
excused absences and school calendars. absence. Either define excused versus unexcused absence or
collect school calendar to determine attendance. Create
physical database structure to allow collection of daily
attendance in the future.
3.3 Extend course classification to all grades. OSPI has intentions to “turn on validation” thus improving
the use of the codes.

4 Replace teacher certification system with The certification system currently lacks many of the
one capable of collecting all required features requested via research and policy questions as
educator information including postpost- well as requires error-prone
prone manual intervention.
secondary performance and relevant major.
4.1 Develop plans to phase out paper
systems / collections: CTE, eCert, Special
Education discipline, e.g.
4.2 Data in eCertification is not connected to Data is manually entered twice.
Certificate DB; data not directly used.
4.3 Collect degree information and Significant interest was expressed in having more clear
institution related to certification. information on teacher education background
4.4 Extend system to maintain professional Vision for system extends to include tracking a teacher’s
growth plans connecting specific course entire history and their academic credentials including their
schedules
hedules and student outcomes with course, continuing education, degree, certificates,
teacher qualifications. endorsements, etc.

5 Commit to a feasibility study to use CEDARS Recommend detailed studies of variance of possible
data to drive apportionment. Run multiple funding using CEDARS as first step in determining district
models approximating Apportionment FTEs level differences between accounting methods.
with CEDARS head counts. Determine
variance. Design legislative action as
needed.
5.1 Washington should expand its chart of
accounts for all school financial
transactions and report the transaction
data to OSPI for analysis and
comparisons within the state data
warehouse once built.

6 OSPI should establish a database of record Although the CEDARS data ta warehouse does not yet exist,
for each data element in the EDFacts when established it should contain data snapshots for all
collections depending on the required official EDFacts reports.
reporting period. Those data can then be
published to the data warehouse as the
official record of the submission.

June 2010 Page 5


Washington State K–12
K Education
Data Gap Analysis

6.1 Build EDFacts data mart as part of data


warehouse.

Page 6 June 2010


Washington State K–12
K Education
Data Gap Analysis

INTRODUCTION: BACKGROUND
OUND AND PURPOSE OF THE PROJECT
In 2009, the Washington State Legislature established a vision for a comprehensive KK–12
12 education
improvement data system. The overall intent of this system is to provide Washington stakeholders with
information that addresses critical questions about stud
student
ent progress and the quality and costs of
education in the state of Washington. The system should also incorporate data that allow the state to
address the state’s prioritized research and policy questions.

According to ESHB 2261, the objectives of the d data


ata system are to monitor student progress; have
information on the quality of the educator workforce; monitor and analyze the program costs; provide
for financial integrity and accountability; and have the capability to link across these various data
components
onents by student, by class, by teacher, by school, by district, and statewide (Washington State
Legislature, 2009). The intended audiences for reports from the data system “include teachers, parents,
superintendents, school boards, legislature, OSPI, and the public” (OSPI, December 2009). Information
regarding the legislation is available in Appendix A.

The vision of the Washington Legislature anticipates emerging data system capacities that allow for the
linkage of student level data with educator and fi financial data and calls for a transformation from a state
level “allocation and compliance” data system to an “education improvement” data system—a system system
that will facilitate decision making at all levels (OSPI, November 2009). As shown in Table 1, Part 2 of
ESHB 2261 specifies the 12 components to be included in the data system.

Table 1. Twelve Components of the Washington State Data System


1. Comprehensive educator information, including grade level and courses taught, job assignment, years of
experience, higher education institution for degree, compensation, mobility, and other variables
2. Capacity to link educator assignment information with educator certification
3. Common coding of secondary courses and major areas of study at the elementary level or standard coding
of course content
4. Robust student information, including student characteristics, course and program enrollment, state
assessment performance, and performance on college readiness tests
5. A subset of student information elements to serve as a dropout early warning system
6. The capacity to link educator information with student information
7. A common standardized structure for reporting the costs or programs at the school and district level with a
focus on the costs of services delivered to students
8. Separate accounting of state, federal, and local revenues and costs
9. Information linking state funding formulas to school and district budgeting and accounting procedures
10. The capacity to link program cost information with student performance information to gauge the cost
effectiveness of programs
11. Information that is centrally accessible and updated regularly
12. An anonymous, non-identifiableble replicated copy of data that is updated at least quarterly and made
available to the public by the state

June 2010 Page 7


Washington State K–12
K Education
Data Gap Analysis

To assist with the design and operation of the data system, the Legislature created a Data Governance Group
within the OSPI responsible for implementing the tasks delineated below with consultant assistance.
Table 2. Tasks of the Data Governance Group
• Identify critical research and policy questions.
• Determine new reporting needs— —identify
identify the reports and other information that meet user needs.
• Create a comprehensive needs requirement document detailing the specific information and technical
capacity needed by school districts and the state.
• Conduct a gap analysis of current and planned information.
• Focus on financial and cost data necessary to support the new K K–12
12 financial models and funding formulas.
• Define the operating rules and governance structure for K K–12 data collection.

Data Governance Group members were selected by State Superintendent Randy Dorn in July and August
2009 and the group began meeting monthly in August. After its formation,
ion, the Data Governance Group
completed several activities to accomplish the tasks described in Table 2. Since that time OSPI has
reported that the Data Governance Group has:
• Held ten meetings since August 2009 hearing from teachers, principals, counselors,
counselor business
officials, superintendents as to their unique data needs and the utility of current OSPI systems.
• Adopted Implementation Guidelines for the K K-12 Data Governance System (available
available at
http://www.k12.wa.us/K12DataGovernance/pubdocs/DataGovernanceManualV
http://www.k12.wa.us/K12DataGovernance/pubdocs/DataGovernanceManualV-1.pdf ) during
the December 16, 2009 meeting
meeting. This document outlines
utlines the data management processes,
policies, and
nd priorities for all K
K-12 data.
• With the assistance
nce of PCG Education
Education,, identified the research and policy questions of interest to
state stakeholders. The research and policy questions report are available on the data
governance web site at: http://www.k12.wa.us/K12DataGovernance/Objectives.aspx .
• Reviewed the current status of Washington’s K K–12
12 education data system, including the status
of systems such as the Comprehensive Education Data and Research System (CEDARS),
(CEDARS) a
student information data collection begun in August 2009, and eCert (an educator database),
database)
Apportionment re-hosting
hosting project, and a review of plans for data system enhancements.
• Initiated work on the fiscal, student, and class size reports OSPI is to post on the Internet,
I
including processes to ensure data accuracy and compliance.
• Created a website to share information about the Group’s responsibilities and activities with the
general public.
In designing the education improvement data system, the task of identifyi
identifying
ng a priority list of questions
followed by a gap analysis represented critical first steps. In December 2009, Public Consulting Group
(PCG) was retained by the Office of Superintendent of Public Instruction on behalf of the Data
Governance Group to engage in a short term project. OSPI contracted with PCG Education to assist in
implementing a process to:
1. Identify the priority research and policy questions the state data system should provide
educators with the capacity to address based on a review of the momostst current national literature

Page 8 June 2010


Washington State K–12
K Education
Data Gap Analysis

on state data systems and input from the Washington stakeholders who would be using the
system. Stakeholders included legislators, advocacy groups, researchers, the State Board of
Education, the Professional Educator Standard
Standardss Board, teachers, parents, and district and school
administrators.
2. Conduct a data gap analysis comparing the current status of the state’s data systems with: 1)
the information needs identified in the prioritization of research and policy questions; 2) the
legislative expectations in ESHB 2261; and 3) the data system requirements in the federal
American
merican Recovery and Reinvestment Act of 2009 (ARRA) and subsequent grant programs.
3. Conduct a technical capabilities gap aanalysis at the school and classroom level to assess whether
data from the state’s statewide longitudinal data system are accessible to key stakeholders
including principals, teachers, and other district leaders.
PCG Education assisted OSPI in indentifying and prioritizing research and policy questions of interest as
described above in task number 1. That report is available on the OSPI Data Governance website at
http://www.k12.wa.us/K12DataGovernance/default.aspx
vernance/default.aspx

This report presents the results of the data system gap analysis conducted by PCG Education (task
number 2 described above).Through
Through the course of the engagement, the individuals and groups that PCG
Education spoke to more thoroughly d defined
efined the vision for state data system, as well as the interim
initiatives proposed to address several of the gaps. In a series of interviews and conversations, key
questions emerged that needed to be addressed in order to move the longitudinal data system syste towards
concrete action steps in implementing this vision. PCG Education collected feedback from participants
about what data systems and collections were already in place, what types of data are available, and the
goals in connecting data systems toward an integrated data warehouse. The result of those interviews,
analysis of OSPI’s data systems, and recommendations are presented below.

PCG Education also assisted OSPI in performing the technical gap analysis at the school and classroom
level as described by task 3 above. That report is available on the OSPI Data Governance website at
http://www.k12.wa.us/K12DataGovernance/default.aspx

June 2010 Page 9


Washington State K–12
K Education
Data Gap Analysis

METHODOLOGY
The methodology for identifying the data system gaps centered on two primary activities: 1) interviews
and discussions with key OSPI information technology and business stakeholders; 2) the creation of a
Washington Metadata Workbook.

Stakeholder Interviews
At the start of the project, OSPI develop
developed a list of internal stakeholders to participate in the interview
process. Interviews were conducted with each of the stakeholders to gather information about their use
and need for data. These interviews wer were conducted March through May 2010 with 34 stakeholder
representatives. The 34 interviewees consisted primarily of individuals within OSPI who are members of
the Data Management Committee, three of whom also sit on the Data Governance Group. As “Data
Stewards”
rds” and “Data Owners,” this group represented most program areas within OSPI including
student, educator, financial, and cross
cross-sectional federal reporting. The
he IT Project Management Director,
Enterprise Architect,, and Data Governance Coordinator also played ed critical roles in providing system and
data expertise throughout the process. PCG Education also interviewed two individuals from the
Education Research and Data Center (ERDC)
(ERDC), which is Washington’s P-2020 statewide longitudinal
database, housed in the Office
fice of Financial management
management.. For a complete list of interviewees, please see
Appendix B.

The interview protocol included an explanation of the goals of the project and metadata workbook,
questions about the interviewee’s sources and uses of data, specifi
specificc key questions those individuals
have been asked but are unable to address due to lack of data or data connections,, and validation of
existing documented metadata. Appendix C includes the project description and interview protocol
given to all interviewees.

All interviews were conducted by phone using an Internet hosted WebEx session to view the metadata
workbook and share other documentation. Members of the IT Project Management Office or Enterprise
Architecture attended the majority of interviews. PCG EEducation
ducation set the context for the interview and
led a brief introduction to the metadata workbook at the start of each interview.. The interview notes
were typed as the session was in progress as well as edits made directly to the workbook to help ensure
the accuracy and timeliness of the information. The interviews provided a critical opportunity to validate
and refine data in the workbook as well as discover additional data collections and systems. Follow up
information including the incorporation of additional data elements, systems, or collections,
collections as well as
the synthesis and integration of the notes
notes, was done following the interview. PCG Education followed up
with several individuals to clarify specific points and gather additional information.

Because
cause of the open ended nature of the interviews, each one was different and focused on the unique
aspects of the program or domain. This allowed the interviewer to more thoroughly discuss the area of
greatest interest or importance to them
them. The notes and metadata from these interviews was captured in
the Washington Metadata Workbook.

Page 10 June 2010


Washington State K–12
K Education
Data Gap Analysis

Washington Metadata Workbook


The collection and documentation of OSPI metadata is at the heart of the data system gap analysis
process. The identification of a data gap ultimately
mately occurs by comparing between data desired
desire and data
collected and stored.. However, it is also important that the elements being compared are normalized in
order for the process to yield meaningful results. That is, one needs to compare apples to apples.
apple
Establishing a consistent process and format for documenting metadata is important not just to tell if a
desired data element is collected, but also to compare definitions, allowable values, frequency of
collection, etc. Thus gaps may expose themselves not just as the absence of data collected, but also in
terms of timing or level of aggregation. For example, in Washington suspensions / expulsions data are
collected, but not as a student level attribute but instead an aggregate number of incidents at the
district level are reported to OSPI,, therefore preventing student level associations with these data.

To assist in the documentation of OSPI metadata, PCG Education developed a Microsoft Excel
documentation template designed to capture metadata about the appropriate people, systems, data
items, and data dictionary elements necessary for the gap analysis. The workbook provided the
generalized framework for the metadata inventory process and was customized to suit the OSPI working
environment through conversations
ations and review with OSPI staff. The OSPI “Data Owners” were all asked
to comprehensively review the workbook as well as the preliminar
preliminarily identified gaps. Their edits and
findings are all incorporated into the delivered version of the workbook.

While PCG
CG Education would recommend OSPI consider adopting a more formal metadata
documentation
entation tool and process, the w workbook
orkbook serves as a key starting point for developing a data
roadmap and a more formal comprehensive metadata library. The ultimate goal for the workbook is to
produce a normative list of data elements, or data dictionary, across the enterprise that can serve as the
foundational description of all data collected and reported, with common definitions and option sets.

The Washington Metadata Workbo


Workbookok provided the framework for performing the data gap analysis and
as such the PCG Education process closely mirrored the tabs contained within the workbook. The
process for documenting this metadata did not always follow a linear path, but instead tended to be
iterative. For example, the identification of aan additional system led to an interview in which an
additional collection was identified for which there were additional people to interview, and so forth.
The following table summarizes the content and results of the interview and metadata documentation
process. The workbook itself is not suited to be included as an appendix but is a significant deliverable
provided separately to Washington. The workbook is available at
http://www.k12.wa.us/K12DataGovernance/default.aspx

Table 3. Washington Metadata Workbook Description and Contents

Overview An overview of the metadata documentation process flow and definitions of each tab and
intended purpose.
Glossary A glossary of terms used throughout the workbook, organized by tab.
People A list of individual stakeholders throughout OSPI with department, titles, and contact
information. The proper identification of data sources throughout OSPI starts with

June 2010 Page 11


Washington State K–12
K Education
Data Gap Analysis

people. One of the critical purposes of the interview process was to identify all
authoritative data sources. By talking to the technical and business resources, PCG
Education was able to identify additional people, systems, and data collections that are
documented in the workbook.

In total, 34 individual stakeholders were formally interviewed as part of the


documentation gathering and validation process.
Systems A list of systems containing information on system name, office responsible for data, list
of sub-systems,
systems, basic description, business and technical owners, and reference to item
level repository.

In total, 17 columns of information on 67 distinct systems and 174 iGrants packages were
identified and documented. Please see Appendix D for a complete list of systems
reviewed.
Items List of all items collected through systems, assessments, spreadsheets, and external
exte
vendor hosted systems
systems.. Includes name, definition, data type, and references to original
source.

Starting with a list of 56,013 data elements, PCG Education identified 16,269
16,2 of those
which are collected from districts. The remaining 39,744 data items are not collected
from districts but instead serve the internal operations of OSPI. Of those 16,269 data
elements collected, 15,645 (96%) come from iGrants.
Data Dictionary List of all data elements necessary for the data gap analysis. Provides name, definition,
data types, option values, and mapping
mappings to the National Education Data Model and
EDEN/EDFacts collections
collections.

PCG Education mapped most major OSPI systems to the National Education Data Model,
v. 2.0.: CEDARS, Certif
Certificate, eCert, EDS/EMS, and SAFS. Approximately 26 columns of
information with 465 element level mappings were completed.
Interview Notes The chronological log of all interview notes categorized by topic. The interview notes
were reviewed for identified gagaps and integrated into other partss of the workbook as
necessary and are preserved for reference.

In total, there were 397 individual free form text line items from the 34 interviews.
Questions Deliverable of this work: an analysis of the data necessary and data gaps for the high
priority research and policy questions as identified by part one of this project.

See Research and Policy Questions Gaps discussion below.


2261 Deliverable of this wo
work:
rk: an analysis of the legislative expectations on data and gaps.

See Analysis of ESHB 2261 Expectations and Gaps discussion below.


ARRA Deliverable of this work: an analysis of the data requirements to fulfill the ARRA
assurances.

See Analysis of ARRA Expectations and Gaps discussion below.


Gaps Deliverable of this work: an analysis of data gaps to the National Education Data Model.
Model

Page 12 June 2010


Washington State K–12
K Education
Data Gap Analysis

See Analysis of Data Dictionary Gaps discussion below.


Reference An inventory of other sources consulted as part of the data system gap analysis.
Indicator Model A sample of Key Performance Indicators suggested by PCG Education which includes
specific statistics for determining risk,, warning, neutral, good, and exemplary status for
Student Engagement, Academic Engagement, and Students at Risk. These indicators were
not reviewed or suggested by OSPI but can be built from the data elements specified by
National Education Data Model and mapped to Washington data elements. elements
Assessments A list of assessments by grade and content area with notes on dates administered and
score type.

In total, 10 columns of information on 68 assessments were identified and documented.

National Education Data Model


The National Education Data Model ((NEDM) is a project funded by the US Department of Education and
coordinated by the Council of Chief State School Officers. Its mission is to create an open framework
based on current standards for education data systems to:

• describe relationships between and among data sets; and


• create an open framework based on current data standards to build education data systems.
NEDM provides a P – 20 data resource and common framework and language for collecting, comparing,
and using data to improve schools and answer important research and policy questions. It also supports
a blueprint of data available for current and future collection and reporting. This includes a set of
consistent data definitions and an architecture that will all
allow for improved data quality
ity as well as
interoperability from multiple perspectives:

• Educators: Use the data model to identify requirements


• Vendors: Extract a software
software-specific conceptual model
• Researchers: Prepare a research design
The development of NEDM involved taking important education questions, issues, or processes, and
identifying the data that need to be tracked in order to answer the questions, address the issues, or
reflect the processes involved.

NEDM 2.0

The Washington Metadata Workbook is based on the second version of NEDM “State Core” data
elements, officially released March 2010. Extending the questions based approach taken with the initial
development of NEDM, version 2.0 explicitly included federal reporting requirements and other national
n
standards:

• EDEN/EDFacts (federal compliance reporting) record level elements


• National Center for Education Statistics (NCES) Handbooks

June 2010 Page 13


Washington State K–12
K Education
Data Gap Analysis

• School Interoperability Framework (SIF) v2r3


• Post-secondary
secondary Electronic Standards Council (PESC)
• Data assurance
ce called out in the American Recovery and Reinvestment Act (ARRA)
• The ten “essential elements” of the Data Quality Campaign (DQC) for statewide longitudinal
data systems
The result was a merged set of common elements for students, programs, school districts,
distri and post
secondary institutions. PCG Education led the State Core Team, a group focused on building out and
validating the core of the model by:

• Mapping all 86 EDEN/EDFacts collections to the data element list


• Mapping 33 state longitudinal data system
systems to the data model.
• Interviewing 19 state departments of education
The following are several key insights gained during the development of NEDM 2.0 applicable to
Washington:

Insight #1: A national standard should be used to create comparable types of enro enrollment
llment. One of the
earliest insights that helped direct the development of the initial version of the State Core Data Set was
the recognition that all states are dealing with three primary types of school and district enrollment
attributions. While each state
ate may call it something different, the archetypical case involves a student
resident in one district, enrolled as a member in a school in the same or in a second district, and serviced
by either of those or by a third district for special education or other
her services. Mapping each state to
these three enrollment types is necessary to establish data comparability.

Gap: No gap. Washington is able to distinguish between these three entity types using a Primary School
indicator in the CEDARS School Student Fil
File (C).

Recommendation: Washington could consider using the NEDM State Core naming convention for
enhanced clarity and comparability with other states. Consider the use of,, “Resident”, “Member”, and
“Serviced by” enrollment types to distinguish the multiple levels of enrollment.

Insight #2: The creation of standardized data sets is important. It is impossible to properly document a
data set without first distinguishing certain key factors to establish the context of the data. Primary
among these factors are the time and type of the data set. For example, there is a large difference in the
creation
ion and usage of a snapshot, current, or other specialized data set such as a student cohort. A
snapshot data set often must be created for EDEN/EDFacts and other federal reporting. It involves a
known set of transformations from source systems into a str structure
ucture that is flattened to a particular point
in time. This is how the CEDARS collections currently function. This structure is also useful for Online
Analytic Processing (OLAP) cube development and other analytic structures. Current data sets come
much closer
loser to the structure of normalized and operational structures. They always contain the most
current data available for the given attributes. That is, some data within the data set may have been
updated within the past several days and some may not have b been
een updated for several months. They
are more flexible and accommodate more frequent updates and heterogeneous data sets, but are more
complex to use properly for reports and aggregate analysis. Additional specialized data sets must be

Page 14 June 2010


Washington State K–12
K Education
Data Gap Analysis

created to establish the unique context for NNational Governors Association graduation
raduation rate cohorts,
assessment, discipline incidents, special education, organization scorecards, and directories. Each of
these data sets is included in the State Core and carried through the mod
model.

Gap: There is not yet a standard practice within Washington with regards to identifying dataset
metadata.

Recommendation: Adopt NEDM State Core entity.attribute structure for datasets:

DataSet.Data_Set_ID
DataSet.Data_Set_Name
DataSet.Data_Set_Description
ption
DataSet.Data_Set_Version
DataSet.Data_Set_Type
DataSet.System_Date
DataSet.Reporting_Date
DataSet.Timeset
DataSet.Reporting__School_Year

Insight #3: It is necessary for NEDM to add “Dimensions.” In developing the State Core taxonomy and
snapshot dataset, it became useful to group student and other attributes by type and establish a
standard, non-alphabetical
alphabetical presentation order. While many terms could be used (i.e. attribute type,
group, category),
ry), the term “dimension” was selected to describe this grouping after conversations and
interviews with state data architects confirmed the importance of this structure to facilitate data
management, reporting, and analytic cube development.

Gap: Washington
on does not yet have a data dictionary that describes data in the OSPI or ERDC enterprise
by primary entity and attribute.

Recommendation: Adopt the Data Dictionary in the Washington Metadata Workbook as a standard for
classifying all core data elements.

Connection to Research and Policy Questions


Phase one of PCG Education’s engagement with OSPI resulted in a report detailing the high priority
research and policy questions that stakeholders throughout the State of Washington want the
longitudinal data system
stem to be capable of addressing. Please see OSPI Data Governance website at
http://www.k12.wa.us/K12DataGovernance/default.aspx for a copy of the report. The questions were
derived from a combination
ombination of interviews with key stakeholders, a national literature review, and the
development and analysis of three targeted surveys at the district, school, and state level. This approach
enabled respondents to answer questions appropriate for their p position
osition and level and allowed an
analysis of the varying data priorities of each group of stakeholders.

This process identified 48 research and policy questions where there was high consensus about the
priority of the questions. While reflecting a compreh
comprehensive
ensive array of educational issues, these 48

June 2010 Page 15


Washington State K–12
K Education
Data Gap Analysis

questions represent a relatively modest set of high priority research and policy questions, given the
hundreds of questions a state data system might answer, and the fact that the questions represent nine
categories
ries of information, as well as linkages across the nine categories. Within this set of 48 questions,
18 were in the top ten rated questions of one or more of the stakeholder groups surveyed.

With a well documented set of OSPI metadata and mapping to NE NEDM,


DM, PCG Education was able to
identify what data are immediately available to answer the 48 research and policy questions by
decomposing the questions into their component data elements. This decomposition resulted in a list of
data elements that would be necessary to answer each question. These data elements are documented
in the Workbook and mapped to their NEDM entity / attribute identification. With a specific list of data
elements needed to answer the questions and a list of data elements available wit
within
hin OSPI, the gaps
become apparent. See Research and Policy Question Gap Analysis for further detail.

Page 16 June 2010


Washington State K–12
K Education
Data Gap Analysis

GAP ANALYSIS
The following section provides highlights from the Washington Metadata Workbook which was provided
to OSPI as a separate deliverable. The reader is strongly encouraged to review the workbook for
additional detail supporting the data element gaps and recommendations.

Analysis of ESHB 2261 Expectations and Gaps


In November 2009, OSPI submitted a preliminary report to the Legislature on the current capacity of
school districts and the state to implement each of the specific components required to meet ESHB 2261
objectives. In several cases the requirements center on developing additional capabilities, systems, or
processes, and not necessarily data. However, where possible, P PCG Education has developed a gap
analysis on the key data elements and linkages necessary to meet each legislative expectation using the
Washington Metadata Workbook.

1. Comprehensive educator information including: grade level taught, courses taught, building or
location, program, job assignment, years of experience, the institution of higher education from
which the educator obtained his or her degree, compensation, class size, mobility of class population,
socioeconomic data of class, number of languages and which languages are spoken by students,
general resources available for curriculum and other classroom needs, number and type of
instructional
nal support staff in the building

Gap: Although most components identified as comprehensive educator information are currently
collected, in order to successfully
essfully meet the expectation several new elements must be collected.
collected

Recommendation:
Data Element Gaps:
The institution of higher education Gap: In some instances Washington can determine the institution
from which the educator obtained from which an educator received their certification, but there is
his or her degree not a field to account for institution of higher education.
Recommendation: Collect Staff.Degree Granting Institution.
Institution
Number of languages and which Gap: Washington does collect native language and language that
languages are spoken by students is spoken at home, however, does not currently capture data for
students that speak multiple languages. For example, a student
who speaks Spanish, French, and English
nglish is a native French
speaker and communicates in English at home. WA does not
capture that the student can also speak Spanish.
Recommendation: Either collect multiple home language codes
per student or seek legislative change.
General resources available for Gap: There is currently no Washington data element nor a NEDM
curriculum and other classroom attribute that accounts for this expectation.

June 2010 Page 17


Washington State K–12
K Education
Data Gap Analysis

needs Recommendation: Legislature clarify intent (see findings from


research and policy questions analysis).

2. Capacity to link educator assignment information with educator certification including: type of
certification, route to certification, certification program, certification assessment, evaluation scores

Gap: Because staff certification number is collected across each system (CEDARS, eCert, and S-275),
S
certification information can be linked to ed
educator assignment information. However, not all
certification items identified by the Legislative expectations are currentlyy collected.

Recommendation:
Data Element Gaps:
Route to Certification Gap: If the intention of the legislature is to collect an education
profile, there
here is currently not a WA data element that accounts
for this expectation.
Recommendation: Collect Staff.Certification Path
Certification Program Gap: Currently, WA has certification program data available only
for in-state certifications.
Recommendation: Collect Staff.Certification Program upon
u initial
application or renewal.
Evaluation Scores Gap: There is currently not a Washington data element that
accounts for this expectation.
Recommendation: Collect Staff.Evaluation Score in accordance
with the implementation of SB 6696.

3. Common coding of secondary courses and major areas of study at the elementary level or standard
coding of course content

Gap: While a common coding scheme of secondary courses has been implemented this school year
for high school courses, there is currently no collection of major areas of study at the elementary
level besides general
al “Elementary Curriculum”.

Recommendation: To meet this expectation, elementary schedules must be consistently broken


down to their major areas or standard coding. Expand course classification to all grades.
grades

4. Robust student information inclu


including:
ding: student characteristics, course and program enrollment, state
assessment performance, and perform
performance on college readiness tests

Page 18 June 2010


Washington State K–12
K Education
Data Gap Analysis

Gap: Many student characteristics are obtained at the individual student level through CEDARS data
collections but there are gaps according to the National Education Data Model and the research and
policy questions analysis.

Recommendation: Expand collection to include elements necessary to meet Legislative


expectations. The following table lists all data eleme
element
nt gaps. While Washington meets its federal
reporting requirements via EDFacts, not all data are collected at the student level but instead are
collected as aggregate counts by the district
district.. Those elements are collected but are included below as
suggestionss of additional student level attributes. In addition, NEDM exposes the best practices as
validated with other state departments of education. Many of the following data elements may not
be appropriate for Washington but are presented here with a justifica
justification
tion for consideration.

Table 4. NEDM Gaps – Specific Data Element List


Entity Category Element Justification
Student Identity Generation Code Generation Code (Jr., III, etc.) should be
separated into its own field so that is not
mistakenly added to last name.
Student Identity Personal Title/Prefix Profile
Student Identity Other Name Profile
Student Demographic City of Birth Used for identity verification
Student Demographic State of Birth Used for identity verification
Student Demographic Family Size Profile
Student Enrollment Address Type Profile
Student Enrollment Street Number/Name Profile
Student Enrollment Apartment/Room/Suite Number Profile
Student Enrollment City Profile
Student Enrollment Name of County Profile
Student Enrollment State Abbreviation Profile
Student Enrollment Zip Code Profile
Student Enrollment Telephone Number Type Profile
Student Enrollment Telephone Number Profile
Student Enrollment Primary Telephone Number Profile
Indicator
Student Enrollment Electronic Mail Address Type Profile
Student Enrollment Electronic Mail Address Profile
Student 504 504 Accommodation plan Necessary to track students covered under
Section 504 to ensure student needs are met
Student SpEd IEP Start Date Identifies which students have an active IEP for
child count dates
Student SpEd IEP End Date Identifies which students have an active IEP for
child count dates
Student SpEd Secondary Disability Type Identifies students with more than one disability
Student SpEd Awaiting Initial Evaluation for Used for federal reporting and to monitor local
Special Education compliance for evaluating students
Student SpEd Evaluated for Special Education Used for OSEP compliance processes
but Not Receiving Services

June 2010 Page 19


Washington State K–12
K Education
Data Gap Analysis

Student Title I Title I Participant Type Used in EDFacts reporting.


Student Title I NCLB Title I School Choice Applied Used in EDFacts reporting.
Student Title I NCLB Title I School Choice Offered Used in EDFacts reporting.
Student Title I Title I Supplemental Services Used in EDFacts reporting.
Eligible
Student Title I Title I Supplemental Services Used in EDFacts reporting.
Applied
Student Title I Title I Supplemental Services Used in EDFacts reporting.
Offered
Student Title I Supplemental Service Provider Used in EDFacts reporting.
Student Title I Title I Support Services Received Used in EDFacts reporting
Student CTE Displaced Homemaker Needed for the Perkins CTE Act
Student Immigrant Country of Citizenship Profile
Student Homeless Homeless Unaccompanied Youth Used in EDFacts reporting
Status
Student Homeless Homeless Served Status Used in EDFacts reporting
Student Homeless Homeless Services Received Used to determine whether student is
participating in a McKinney-Vento
Vento program
Student Homeless Homeless Primary Nighttime Necessary to provide transportation to school
Residence
Student Neglected and Neglected or Delinquent Program Used in EDFacts reporting
Delinquent Participant
Student Neglected and Length of Placement in Neglected Used in EDFacts reporting
Delinquent and Delinquent Program
Student Neglected and Neglected or Delinquent Program Used in EDFacts reporting
Delinquent Type
Student Neglected and Pre-Post
Post Test Indicator (N and D) Used in EDFacts reporting
Delinquent
Student Neglected and Pretest Results Used in EDFacts reporting
Delinquent
Student Neglected and Progress Level (N and D) Used in EDFacts reporting
Delinquent
Student Assessment Technology Literacy Status in 8th Used in Growth Calculations and student profile
Status Grade reports. Very useful in analytics as a dimension
for analysis.
Student Discipline # Days Suspended in a School Student suspension is a clear sign that the
Year (Total) student may be at risk for dropout.
Student Discipline Number of Days Expelled In a Used in EDFacts reports and an important
School Year indication of serious behavior problems.
Incident Instance Student Unique ID Connecting the Incident to the Student enables
analysis and is necessary for data management.
CEDARS collects, but not linked to student in
Attendance and Weapons system.
Incident Instance Student Role The student’s role in the incident is important.
Incident Instance Date Data should be kept for analysis.
Incident Instance Discipline Reason Used for EDFacts reports that require a count of
incidents rather than a count of students.
Incident Instance Discipline Method - Firearms Used for EDFacts reports that require a count of
(IDEA) incidents rather than a count of students.

Page 20 June 2010


Washington State K–12
K Education
Data Gap Analysis

Incident Instance Interim Removal (IDEA) Used for EDFacts reports that require a count of
incidents rather than a count of students.
Incident Instance Interim Removal Reason (IDEA) Used for EDFacts reports that require a count of
incidents rather than a count of students.
Incident Instance Educational Services Used for EDFacts reports that require a count of
incidents rather than a count of students.
Staff Identity Name Prefix Used to establish the identity of staff members.
Staff Identity Generation Code/Suffix Used to establish the identity of staff members.
Staff Assignment Contract Beginning Date Used to establish teacher assignment to a
school or district.
Staff Assignment Secondary Teaching Assignment Used in EDFacts reporting.
(Academic Subject)
Staff Assignment MEP Session Type Used in EDFacts reporting.
Staff Credential Paraprofessional Qualification Used in EDFacts reporting.
Status
Staff Credential Degree Granting Institution Teacher experience.
Staff Credential Technology Skills Assessed Used in EDFacts reporting.
Staff Credential Technology Standards Met Used in EDFacts reporting.
Section Section Location/Room # Used to establish a student's relationship to a
teacher in a particular section.
Section Section Session Name Used to establish a student's relationship to a
teacher in a particular section.
Section Course Available Credit Used to establish a student's relationship to a
teacher in a particular section.
Section Course Course Level Used to establish a student's relationship to a
teacher in a particular section.
Section Staff Section Entry Date Used to establish a student's relationship to a
teacher in a particular section.
Section Staff Section Exit Date Used to establish a student's relationship to a
teacher in a particular section.
School AYP AYP Status Profile
School AYP Alternate Approach Status Profile
School AYP Improvement Status Used in EDFacts reporting.
School Assessment Advanced Placement (AP) Profile
Mathematics Program Offered
School Assessment Advanced Placement (AP) Other Profile
Program Offered
School Assessment Advanced Placement (AP) Science Profile
Program Offered
School Type Availability of Ability Grouping Profile
School Type Distinguished School Status Profile
School Type Focus of Alternative School Profile
School Type Magnet Status Used in EDFacts reporting.
School Type Corrective Action Used in EDFacts reporting.
School Type Restructuring Action Used in EDFacts reporting.
School Type School Improvement Funds Used in EDFacts reporting.
Allocation
School Type Shared Time Indicator Profile

June 2010 Page 21


Washington State K–12
K Education
Data Gap Analysis

School Type AMAO Progress Attainment Used in EDFacts reporting.


Status for LEP Students
School Type AMAO Proficiency Attainment Used in EDFacts reporting.
Status for LEP Students
School Type Elementary/ Middle Additional Used in EDFacts reporting.
Indicator Status
School Type GFSA Reporting Status Used in EDFacts reporting.
School Type REAP Alternative Funding Used in EDFacts reporting.
Indicator
School Type Supplemental Services Provided Profile
School Indicator High School Graduation Rate Profile
Indicator Status
School Indicator Persistently Dangerous Status Profile
School Indicator Number of Computers with High Used in EDFacts reporting.
Speed Ethernet or Wireless
Connectivity
School Indicator Number of Computers with Less Used in EDFacts reporting.
than High Speed Connectivity
School Indicator Total Number of Schools Used in EDFacts reporting.
School Indicator Truancy Rate Used in EDFacts reporting.
School Indicator Boys Only Interscholastic Athletic Profile
Sports
School Indicator Girls Only Interscholastic Athletic Profile
Sports
School Indicator Boys Only Interscholastic Athletic Profile
Teams
School Indicator Girls Only Interscholastic Athletic Profile
Teams
District Directory D-U-N-SS Number Directory
District Directory Supervisory Union Identification Directory
Number
District Directory Education Agency Type Directory
District Directory Title I District Status Directory
District Directory Operational Status Directory
District Directory Grades Offered Directory
District Sup Official Title of LEA Directory
Superintendent
District AYP AYP Status Profile
District AYP Alternate Approach Status Profile
District AYP Improvement Status Profile
District Indicator Federal Programs Offered Used in EDFacts reporting.
District Indicator Funding Allocation Type Used in EDFacts reporting.
District Indicator Integrated Technology Status Used in EDFacts reporting.
District Indicator Federal Funding Allocations Used in EDFacts reporting.
District Indicator Number of Schools Classified as Profile
Persistently Dangerous

Page 22 June 2010


Washington State K–12
K Education
Data Gap Analysis

5. A subset of student information elements to serve as a dropout early warning system

Gap: Assuming Washington chooses to implement the National Governors Association (NGA)
recommended early warning dropout model, daily attendance and student level discipline are
required and currently not available.

Recommendation: Washington should move forward ward with the NGA model and collect daily
attendance and student level behavior data from all districts. Student course grades, grade level,
and age are already available. Washington needs to define what constitutes an excused versus
unexcused absence or collect
ollect district calendar information. Student behavior / discipline incidents
are reported in aggregate by the district but should be collected and reported on a student basis.

n states across the nation, drop out early warning and intervention systems (DEWIS) are emerging
In
as one of the most valuable applications of state longitudinal data systems to support school
operational issues. Washington is also currently examining this iissue
ssue through ESSB 6403. While
school districts will always have the most up
up-to-date
date attendance and granular local assessment data,
a state longitudinal data system can provide a strong foundation of near-real-time
time data integrated
across districts and schooll years to provide an effective data set to screen students most at risk.

The National Governors Association (NGA) nicely summarizes near consensus conclusions on
appropriate state actions synthesized from the growing national body of research, “[E]arly
“[E] warning
data systems are neither expensive nor difficult to build because they are based on basic academic
information already collected at the school and district levels
levels:: attendance, behavior, course
achievement, and student age and grade. In numerous studies,dies, indicators based on these data have
been shown to be highly predictive of dropping out. Several studies suggest that grades are more
highly predictive than test scores for graduation, but states with graduation tests should consider
including low testt scores as an indicator.
indicator.”” (“Achievement for All” NGA, December, 2009).

6. The capacity to link educator information with student information

Gap: Capacity to link education information with student information takes place through the
Washington field Course ID.. This element is collected in both the Student Schedule File and the Staff
Schedule File within CEDARs to provide the necessary linkage. However the course schedule is
snapshot based – an indication of a student’s schedule at the time of the file upload.

Recommendation: Establish section entrance and exit for student and staff schedules in CEDARS.

7. A common standardized structure for reporting the costs of programs at the school and district level
with a focus on the costss of services delivered to students

June 2010 Page 23


Washington State K–12
K Education
Data Gap Analysis

Gap: A standardized
tandardized structure for reporting the expenditures by school is nott yet in place.
place

Recommendation: Washington
ashington should expand its chart of accounts for all school financial
transactions and report the granular transaction data to OSPI for analysis and comparisons within
the state data warehouse once built
built. Washington should continue to move forward to address the
legislative requirement for school level expenditure accounting.

8. Separate accounting of state, fed


federal, and local revenues and costs

Gap: A method for connecting costs to specific revenue streams is not in yet place
place,, although OSPI is
currently exploring options that would align each expenditure codi
coding
ng to a specified revenue stream.

Recommendation: OSPI should continue their exploration of this area. If adopted, thet accounting
manual should include appropriate guidance on methodologies and practices for capturing this
linkage within detailed accounting records. OSPI should evaluate the cost associated
associate with this effort
in light of new funding formulas based on prototypical school structure as this requirement may
become less important.

9. Information linking state funding formulas to school and district bud


budgeting
geting and accounting
procedures

Gap: The method for collecting data to link state funding formulas to district budgeting does not yet
exist.

Recommendation: Commit to a feasibility study to use CEDARS data to drive apportionment and
create a standard chart of accounts for building and progr
program level accounting. Conduct detailed
studies of variance of possible funding using CEDARS as first step in determining district level
differences between accounting methods. Run multiple models approximating Apportionment FTEs
with CEDARS unduplicated head d counts. Design legislative action as needed
needed.

Creating a closed loop system, where apportionment is driven from an unduplicated headcount of
students as reported through the state SLDS, will provide districts a powerful incentive to accurately
and timely report their data, leading to an overall increa
increase
se in quality and usability. However,
Washington currently maintains distinct systems for these functions and must proceed cautiously
when considering the implications of altering a funding approach develope
developed d over decades.

10. The capacity to link program cost information with student performance information to gauge the
cost effectiveness of programs

Page 24 June 2010


Washington State K–12
K Education
Data Gap Analysis

Gap: Before linking program cost information, the effectiveness of a pprogram


rogram alone must be
measured. Further, one definition of “program” at the state level tends to include items like Title I,
LEP, and Special Education.. CEDARS collects this type of program participation. There are also, of
course, many smaller initiatives such as an after school reading program
program, curricular software
packages, etc. that may also need to be considered for cost effectiveness
effectiveness.

Recommendation: Making the assumption that this expectation is for state programs only, the
collected codes must be expanded to include a compcomplete list of programs
grams that the State
S wishes to
evaluate. Students can then be associated with these expenditure categories through the CEDARS
program enrollment file. The generic program enrollment file in CEDARS provides a very flexible and
forward thinking interface to expand data for future programs. The State tate will also need to define the
entities that will be used to measure the effectiveness. Can state assessments be used
longitudinally? Does each program have a diagnostic and exit assessment? The State tate will want to
ensure
nsure these means of measurement are valid and acceptable. To link program cost information,
eparate accounting of state, federal, and local costs) must be achieved and the State
Expectation 8 (Separate S
must be able to associate a total cost with each specific pro
program.

11. Information that is centrally accessible and updated regularly

Gap: Washington does not have a centralized data warehouse.

Recommendation: Washington is proceeding with plans to procure a data warehouse and reporting
solution. Physically moving or replicating all data within OSPI, even if required for reporting, to a
central data warehouse is unnecessary so long as all the sources are known and well documented in
the metadata documentation tool. OSPI has indicated its intention to create a database of record
and schedule for each data element required for reporting. This would allow Washington the
flexibility to report from a number of transactional systems as well as the data warehouse
depending on the timing and scope of the report. It also supports the model of using the data
warehouse for analytic reporting, thereby committing OSPI undertake a careful evaluation of the
data elements stored in the data warehouse versus other transactional systems.

In terms of regularly updating the data, the State receives monthly (often more frequent) updates
from all districts for the required CEDARS elements. The State should also establish data sets as
recommended in the key insights with the development of NEDM as discussed above. Namely, OSPI
will want to establish documented and standard logical data views for every official reporting period
as well as current and cohort data sets.

12. An anonymous, non-identifiable


identifiable replicated copy of data that is updated a
att least quarterly and made
available to the public by the State
tate

June 2010 Page 25


Washington State K–12
K Education
Data Gap Analysis

Gap: Many types of aggregate


ggregate data are available via the OSPI website and de de-indentified
indentified individual
student level data is available by request for several specific report types. However, the state lacks a
general mechanism by which to publish all its data in an anonymous, non non-identifiable
identifiable form as
specified by this legislative requirement.

Recommendation: Develop a de de-indentified


indentified data mart with appropriate suppression rules and
refreshed periodically following official submission snapshot datasets, primarily from CEDARS using
NEDM as the starting point. This data can then made available either directly or indirectly to the
requestor via a web-based
based business intelligence tool or a delimited file format. In general, the more
data that is published for each student, the more likel
likelyy that student is uniquely identifiable.
Washington will need to determine the minimum student count for each individual category of
information published to prevent the identification of students. For example, if there are fewer than
10 special education students per school should those records be removed from the data set or not
marked as special education?

Analysis of American Recov


Recovery and Reinvestment Act Expectations and Gaps
As stated by the U.S. Department of Education, the “overall goals of ARRA aare
re to stimulate the economy
in the short term and invest in education and other essential public services to ensure the long-term
long
economic health of our nation.” During tthe
he development of NEDM, the detailed ARRA assurances were
initially incorporated into the Standards Comparison Report, which formed the basis of NEDM 2.0. PCG
Education used this baseline to map Washington’s data systems, thereby creating the link to data
necessary to fulfill the requirements of ARRA.

There are four assurances that states are required to address in order to improve student achievement
through school improvement and reform:

1. Increase teacher effectiveness and address inequities in the distri


distribution
bution of highly qualified teachers

Gap: Washington does not yet have a method to calculate teacher effectiveness.

Recommendation: OSPI should enable valid teacher effect calculations based on student growth
percentile models. Calculate student assessm
assessmentent elements: Prior Year 1 [Subject] Student Growth
Percentile for each year of assessment data available. Loading the student growth scores into the
data warehouse, once built, will provide critical linkages between the teacher and financial data
domains. Washington will need to develop the appropriate reports and professional development
required on the proper use of growth data.

Within Washington and nationally there is great interest in examining methods for linking student
performance to teacher evaluation
uation models
models. However, this approach requires stakeholders to
fundamentally change the way in which they judge education quality from status to progress and

Page 26 June 2010


Washington State K–12
K Education
Data Gap Analysis

this change is non-trivial.


trivial. For example, evaluating teacher
teachers requires development of principals in the
area of using evidence and data. Many states are grappling with developing models for teacher
evaluation:
• Colorado recently passed SB10SB10-191,
191, part of which establishes a governor’s Council for
Educator Effectiveness; the bill redefines how teachers are awarded tenure
• Rhode Island is producing its Rhode Island Educator Evaluation Model and hopes to be
operational for teachers and principals by 2011
2011-12
• New Hampshire (SB 180) requires the development of a “performance
“performance-based
based accountability
system” that includes
udes measures of student growth to judge whether schools provide all
students with the “opportunity for an adequate education”
• Some states (e.g., Virginia) are interested in using end
end-of-course
course assessments; issues arise
with multiple-testing
testing occurrences an
and other idiosyncrasies

Other key considerations and challenges when considering the limits of student growth percentile
evaluation models:
• Roughly 70% of teachers DO NOT participate directly in large scale state assessment from
which student growth percentiles are calculated
• Student
tudent growth percentiles CAN be calculated across different assessment forms, so long as
the construct measured is similar and the student pool is large and enough and similar
enough; constructs betwe
between
en assessments must be well correlated over time (at least 0.7
correlation needed).

Finally, 14 of the 48 (29%) Washington Research and Policy Questions specifically address teacher
effectiveness in the classroom. Building out the data elements necessary to answer those research
and policy questions will provide additional insight into this assurance. See Research and Policy
Questions Gaps below for the detailed data elements.

2. Establish and use a pre-K-through


through-college-and-career data system to track progress and foster
continuous improvement

Gap: No gap.

Discussion: Throughout the interview process both in this project and the Research and Policy
Question interviews, the interest and importance of tracking students from early childhood to post-
post
high school graduation was clearly expressed. This assurance has been met by the establishment of
the ERDC. Washington has indicated its strong support of this capability through the development of
the SLEDS system at ERDC via their successful 2009 ARRA SLDS ggrant
rant awarded May 2010. This work
will “extend those K-12
12 capabilities by incorporating longitudinal early
early-learning,
learning, post-secondary,
post and
workforce information into a unified, comprehensive, and efficient P P-20 system”” (Washington State
Application for Grants under the SLDS Recovery Act Grant)
Grant).

June 2010 Page 27


Washington State K–12
K Education
Data Gap Analysis

3. Make progress towards rigorous college


college- and career-ready standards and high-quality
quality assessments
that are valid and reliable
le for all students, including LLimited
imited English proficient students and students
with disabilities

Gap: While Washington does have a valid and reliable assessment system, it lacks the ability
a to link
student growth to other educational entities and subgroups such as Limited English Proficient
students and students with disabilities to determine the ef
effectiveness
fectiveness of programs, evaluation on
assessment, and reviews of the characteristics of high performing schools.

Recommendation: See discussion related to student growth percentiles in assurance number one
above.

In addition, 27 of 48 (56%) of the Res


Research
earch and Policy Questions link student subgroups to the
effectiveness of programs, evaluation on assessments, and review of the characteristics of high
performing schools. Building out the data elements necessary to answer those research and policy
questions
ons will provide additional insight into this assurance. See Research and Policy Questions Gaps
below for the detailed data elements.

4. Provide targeted, intensive support and effective interventions to turn around schools identified for
corrective action and restructuring

Gap: There are three accountability models requiring the determination of specific indicators for
Washington districts: the School Improvement Grant model, Adequate Yearly Progress, and the
State Board of Education’s new accountability model. However, Washington lacks the th ability to
calculate key performance indicators for all schools for at risk students and other operational
metrics of interest.

Recommendation: Develop key performance indicators and statistics for determining specific risk,
warning, neutral, good, and exemplary status for Student Engagement, Academic Engagement, and
Students at Risk. These indicators can be built from the data elements specified by National
Education Data Model and mapped to Washington data elements in the CEDARS data warehouse,
once built. A sample of a potential indicator model is included below. Please see Washington
Metadata Workbook for a complete list of sample indicators and required data elements.
elements

Table 5.. Example Key Performance Indicator Model for At Risk Students
Indicator Risk Warning Neutral Good Exemplary
Attendance
Index
Current YTD Attendance Rate <90% 90-95% 95-99%
99% 100%
Last 7 Days Attendance Rate <90% 90-95% 95-99%
99% 100%
Last 30 Days Attendance Rate <90% 90-95% 95-99%
99% 100%

Page 28 June 2010


Washington State K–12
K Education
Data Gap Analysis

Prior Year Attendance Rate <90% 90-95% 95-99%


99% 100%
Current YTD Tardy Count >10 5-10 2-4 1 none
Current YTD Attendance Rate + Low <90% + low
Income income
Behavior
Index
Current YTD # Days Suspended Suspended Not
Suspended
Current YTD # Incidents
Last 30 Days # Incidents
Course Grades/Credits
Index
[Section] Term Grade F D C B A
[Section] Year Grade F D C B A
YTD # Ds or Fs in Core Classes 2+ Ds or Fs 1 D or F No Ds or Fs
PY1 # Ds or Fs in Core Classes 2+ Ds or Fs 2 D or F No Ds or Fs
Current GPA <1.0 1.0-2.5 >2.5 >3.5
% Credits vs. On Track <80% 80-95% 95-105% 105-120%
120% >120%

In addition, 18 of the 48 (38%) Research and Policy Questions compare data between schools and
districts to determine the most effective schools and programs
programs.. Building out the data elements
necessary to answer those research and policy questions will provide additional insight into this
assurance. See Research and Policy Questions Gaps below for the detailed data elements.

Analysis of Data Dictionary Gaps


NEDM includes the organization of data by entity. An entity reflects the real
real-world
world function of the
object. There are seven entity types defined in NEDM 2.0: Student, Incident, Staff, Section, School,
District, and State. Each entity contains one or more categories to add further organization and
hierarchy to the data model. The following table shows the number of categories and distinct data
elements per entity and the overall number of Washington gaps to the National Education Data Model.
Please see Table 4. NEDM Gaps – Specific Data Element List for the detailed data elements associated
with this table.

Table 6.. NEDM Data Element Gaps by Entity


Entity Number of Number of Elements Number of Percent Collected
Categories Within the Entity Washington Element
Gaps
Student 15 213 48 77%
Incident 1 13 8 38%
Staff 5 45 9 80%
Section 6 33 6 82%
School 8 59 30 49%
District 4 27 15 44%

June 2010 Page 29


Washington State K–12
K Education
Data Gap Analysis

State 3 13 0 100%

The fewest number of gaps in absolute terms are within the Student and Staff entities reflecting their
relative maturity developed through the implementation of CEDARS and fulfilling federal reporting
requirements. Included within the Data Dictionary mapping to NEDM is an element
element-level
level linkage to the
EDEN/EDFacts collections, providing Washington with a direct link between what is federally required
and what is currently collected.

EDFacts Granular Data Gaps

Gap: OSPI currently runs many separate data collections, each with its own data definitions.
defini From these
collections, OSPI submits the nearly 90 EDEN/EDFacts files required yearly. As these collections are
largely separate and have limited interoperability, the data collected is often redundant and
contradictory. For example, the count of fr free
ee and reduced lunch students is via CEDARS but the official
snapshot is collected via the child nutrition systems.

Recommendation: OSPI should establish a database of record for each data element in the EDFacts
collections depending on the required report
reporting
ing period. Those data can then be published to the data
warehouse as the official record of the submission. As summarized in the following table, a total of 51
data elements would need to be incorporated to build an EDFacts data mart within the OSPI data
warehouse, once built.

Note, while Washington meets its federal reporting requirements via EDFacts, not all data are collected
at the student level but instead are collected as aggregate counts by the district. Those elements are
collected but are includedd below as suggestions of additional student level attributes or attributes that
are not collected via CEDARS but would need to be included in the data warehouse to build out an
EDFacts data mart.

Table 7.. EDFacts Data Element Gaps by Entity


Entity Number of Number of EDFacts Number of Washington Percent
Categories Elements Within the Entity Element Gaps for EDFacts Available
Student 15 100 21 79%
Incident 1 9 5 44%
Staff 5 21 5 76%
Section 6 5 1 80%
School 8 17 15 12%
District 4 4 4 0%
State 3 11 0 100%

Table 8. EDFacts Gaps – Specific Data Element List


Entity Category Element
Student Title I Title I Participant Type
Student Title I NCLB Title I School Choice Applied
Student Title I NCLB Title I School Choice Offered

Page 30 June 2010


Washington State K–12
K Education
Data Gap Analysis

Student Title I Title I Supplemental Services Eligible


Student Title I Title I Supplemental Services Applied
Student Title I Title I Supplemental Services Offered
Student Title I Supplemental Service Provider
Student Title I Title I Support Services Received
Student Homeless Homeless Unaccompanied Youth Status
Student Homeless Homeless Served Status
Student Homeless Homeless Primary Nighttime Residence
Student Neglected and Delinquent Neglected or Delinquent Program Participant
Student Neglected and Delinquent Length of Placement in Neglected and Delinquent Program
Student Neglected and Delinquent Neglected or Delinquent Program Type
Student Neglected and Delinquent Pre-Post Test Indicator (N and D)
Student Neglected and Delinquent Pretest Results
Student Neglected and Delinquent Progress Level (N and D)
Student Assessment Status Technology Literacy Status in 8th Grade
Student Discipline # Days Suspended in a School Year (Total)
Student Discipline Number of Days Expelled In a School Year
Incident Instance Discipline Reason
Incident Instance Discipline Method - Firearms (IDEA)
Incident Instance Interim Removal (IDEA)
Incident Instance Interim Removal Reason (IDEA)
Incident Instance Educational Services
Staff Assignment Secondary Teaching Assignment (Academic Subject)
Staff Assignment MEP Session Type
Staff Credential Paraprofessional Qualification Status
Staff Credential Technology Skills Assessed
Staff Credential Technology Standards Met
Section Course Course Level
School AYP Improvement Status
School Type Magnet Status
School Type Corrective Action
School Type Restructuring Action
School Type School Improvement Funds Allocation
School Type AMAO Progress Attainment Status for LEP Students
School Type AMAO Proficiency Attainment Status for LEP Students
School Type Elementary/ Middle Additional Indicator Status
School Type GFSA Reporting Status
School Type REAP Alternative Funding Indicator
School Indicator High School Graduation Rate Indicator Status
School Indicator Number of Computers with High Speed Ethernet or Wireless
Connectivity
School Indicator Number of Computers with Less than High Speed Connectivity
School Indicator Total Number of Schools
School Indicator Truancy Rate
District Indicator Federal Programs Offered
District Indicator Funding Allocation Type
District Indicator Integrated Technology Status

June 2010 Page 31


Washington State K–12
K Education
Data Gap Analysis

District Indicator Federal Funding Allocations

Research and Policy Questions Gaps

17 of 48 (35%) high priority Washington R


Research and Policy Questions are currently able to be
answered with the data available via existing collections.

The research and policy questions were designed to be inclusive of the information priorities and the
different categories of information cited in OSPI documents, the national literature, and by
stakeholders. The survey items were organized around nine per
pertinent categories:
1. District and School Enrollment Trends
2. Program and Course Enrollment Trends
3. Student Achievement
4. Attendance, Discipline, Dropout, and Graduation Rates
5. Success and Risk Indicators, and Transitions
6. Program Outcomes
7. Teacher Workforce and d Student Achievement
8. Cost Effectiveness
9. Cost Analyses
The following table shows the distribution of data gaps across the defined categories:

Table 9.. Count of Research and Policy Questions Gaps by Category


Question Category Questions Able to be Questions with Percent
Answered Element Gaps Answerable
District and School Enrollment Trends 3 2 60%
Program and Course Enrollment Trends 3 0 100%
Student Achievement 8 2 80%
Attendance, Discipline, Dropout, and 4 2 67%
Graduation Rates
Success and Risk Indicators, and 7 1 88%
Transitions
Program Outcomes 1 2 33%
Teacher Workforce and Student 2 4 33%
Achievement
Cost Effectiveness 0 4 0%
Cost Analyses 0 3 0%

The following table displays the detailed analysis of data required and gaps to answer each of the 48
high priority research and policy questions as derived from part one of this project.

Page 32 June 2010


Washington State K–12
K Education
Data Gap Analysis

Table 10. Research and Policy Questions Gaps


Question Entity Category Attribute Exists? Notes
District, State, and School Enrollment Trends
1.1 Compared to state trends, what are the variations in district/school No gap
enrollment trends at different grade levels by gender, ethnicity, eligibility
for free/reduced lunch, students in special education, students in ELL
programs, and combinations?
Student Enrollment Grade Level Yes
Student Demographic Race/Ethnicity Yes
Student Demographic Gender Yes
Student Demographic Economic Disadvantaged Yes
Status
Student SPED Primary Disability Type Yes
Student LEP LEP Status Yes Assumption: The Office for
Civil Rights uses the
acronyms ELL and LEP
interchangeably as they
have a similar meaning.
Student Enrollment School Code Yes
Student Enrollment County District Code Yes
Student Enrollment School Year Yes
1.2 What are the program and cost implications of demographic changes Data related to program
for specific subgroups, i.e., entry into special programs, need for cost information, staff
intervention/remedial support, and additional personnel? count by program, and
an
employee cost by
credential type are
required.
Student Demographic Race/Ethnicity Yes
Student Demographic Economic Disadvantaged Yes
Status
Student SPED Primary Disability Type Yes
Student LEP LEP Status Yes
Student Enrollment Program Code Yes Assumption: Program
information includes
intervention information.
Staff Assignment Program Assignment Yes
Staff Credentials Teaching Field or Area No
Authorized
Finance Staff Staff Cost No
Finance Program Program Costs No Have program cost, but not
linked to specific
subgroups and changes
within the program.
1.5/1.7 What are the characteristics and academic profile of students State entry date for non-
non
who are new to the state and to specific districts? LEP students is required.

Student Immigrant Number Months US Yes Only available for students


Attendance who are new to the

June 2010 Page 33


Washington State K–12
K Education
Data Gap Analysis

Table 10. Research and Policy Questions Gaps


Question Entity Category Attribute Exists? Notes
country.

Student Enrollment School Year Yes


Student Enrollment District Enrollment Date Yes
Section Grade Credits Earned Yes
Section Grade Credits Attempted Yes
Section Grade Letter Grade Yes
Section Grade GPA Yes
Student Demographic Race/Ethnicity Yes
Student Demographic Gender Yes
Student Demographic Economic Disadvantaged Yes
Status
Student SPED Primary Disability Type Yes
Student LEP LEP Status Yes
Student LEP Initial WA Placement Date Yes
Student Assessment Proficiency Level Yes
Student Enrollment Date entered WA No
1.6 What are the demographic characteristics of students in individual No gap
classrooms and how do classrooms vary?

Student Demographic Race/Ethnicity Yes


Student Demographic Gender Yes
Student Demographic Economic Disadvantaged Yes
Status
Student Enrollment Program Code Yes
Student Demographic Language Spoken at Home Yes
Section Course Course ID Yes
1.8 What percentage of our students transfer in or out at specific times No gap
of the school year by subgroup and where do they go?
Student Demographic Race/Ethnicity Yes
Student Demographic Gender Yes
Student Enrollment Program Code Yes
Student Enrollment School Enrollment Date Yes
Student Enrollment School Exit Date Yes
Student Enrollment District Enrollment Date Yes
Student Enrollment District Exit Date Yes
Student Enrollment School Withdrawal Code Yes Indicates reason exited,
but reason may not be
known and student's new
school may not be known.
Student Enrollment School Year Yes
Program and Course Enrollment Trends

Page 34 June 2010


Washington State K–12
K Education
Data Gap Analysis

Table 10. Research and Policy Questions Gaps


Question Entity Category Attribute Exists? Notes
2.2 How have individual district/school subgroup participation rates in No gap
advanced middle school courses changed and how do they compare to
similar districts/schools?
Student Demographic Race/Ethnicity Yes
Student Demographic Economic Disadvantaged Yes
Status
Student SPED Primary Disability Type Yes
Student LEP LEP Status Yes
Student Enrollment Program Code Yes
Student Enrollment County District Code Yes
Student Enrollment District Enrollment Date Yes
Student Enrollment School Enrollment Date Yes
Student Enrollment Serving County District Code Yes
Student Enrollment School Code Yes
Section Student Start Date Yes
Section Course Course Level No May be able to be derived
from Section and Course
ID.
Section Course Course ID Yes
Section Section Section ID Yes
2.3 How have individual district/school subgroup participation rates in No gap.
AP, IB, SAT, and ACT exams changed and how do they compare to similar
districts/schools?

Student Demographic Race/Ethnicity Yes


Student Demographic Economic Disadvantaged Yes
Status
Student SPED Primary Disability Type Yes
Student LEP LEP Status Yes
Student Enrollment Program Code Yes
School Indicator AP / IB Course Code Yes
Student Enrollment County District Code Yes
Student Enrollment Serving County District Code Yes
Student Enrollment School Code Yes
Section Course Course Designation Code Yes
Student Assessment Participation in AP, IB, SAT, Yes
ACT exams
School Assessment Assessment Administered No Derived from the file.
2.4/2.7 How have individual district/school subgroup participation rates No gap
in low level/remedial middle/high school courses and in elementary
reading and mathematics intervention programs changed and how do
they compare to similar districts/schools?
Student Demographic Race/Ethnicity Yes
Student Demographic Economic Disadvantaged Yes
Status

June 2010 Page 35


Washington State K–12
K Education
Data Gap Analysis

Table 10. Research and Policy Questions Gaps


Question Entity Category Attribute Exists? Notes
Student SPED Primary Disability Type Yes
Student LEP LEP Status Yes
Section Course Content Area Code Yes
Section Course Course ID Yes
Student Enrollment Program Code Yes
Student Achievement
3.1 What is the grade to grade progress of student subgroups on the No gap
state assessments in reading and mathematics, i.e., what percent of
students initially below proficient reach proficiency and what percent
either maintain or lose proficiency over time?
Student Demographic Race/Ethnicity Yes
Student Demographic Economic Disadvantaged Yes
Status
Student SPED Primary Disability Type Yes
Student LEP LEP Status Yes
Student Assessment GX Assessment Perf Level Yes
3.2 What grade to grade progress did individual students make on the No gap
state assessment? While the review of
proficiency levels can
provide a profile of
students, the State should
consider other growth
calculations.
Student Assessment GX Assessment Perf Level Yes
Student Identity SSID Yes
Student Enrollment Grade Level Yes
3.3 What is the grade to grade progress profile of students in specific No gap..
classrooms? While the review of
proficiency levels can
provide a profile of
students, the State should
consider other growth
calculations.
Student Assessment GX Assessment Perf Level Yes
Student Identity SSID Yes
Student Enrollment Grade Level Yes
3.4 What is the demographic, absence, mobility, program, class grade, No gap.
and course-taking
taking profile of students who do and do not achieve? For a richer analysis,
additional program and
growth data are required.
Student Demographic Race/Ethnicity Yes
Student Demographic Gender Yes
Student Attendance Number of Days in No Can be derived based on
Membership school calendar.

Page 36 June 2010


Washington State K–12
K Education
Data Gap Analysis

Table 10. Research and Policy Questions Gaps


Question Entity Category Attribute Exists? Notes
Student Attendance Cumulative Days Present Yes
Student Attendance Num Unexcused Absence Yes
Student Enrollment Program Code Yes
Section Course Course ID Yes Assumption: Course ID is
mapped to course name
and course level.
Section Grade Letter Grade Yes
Section Course Course Level No May be derived from
course ID.
Student Enrollment School Enrollment Date Yes
Student Enrollment District Enrollment Date Yes
Student Enrollment Exit Reason Code Yes
Student Enrollment School Exit Date Yes
Student Enrollment District Exit Date Yes
Student Enrollment School Entry Code Yes
Student Assessment GX Assessment Perf Level Yes
Student Assessment GX Math/LAL growth No Can be calculated.
3.7 How does the performance profile of high mobility students compare No gap.
to other students, i.e., attendance, proficiency, graduation? A policy decision is
required to define high
mobility.
Student Enrollment School Enrollment Date Yes
Student Enrollment District Enrollment Date Yes
Student Enrollment Exit Reason Code Yes
Student Enrollment School Exit Date Yes
Student Enrollment District Exit Date Yes
Student Enrollment School Entry Code Yes
Student Assessment GX Assessment Perf Level Yes
Student Attendance Number of Days in No Can be derived based on
Membership school calendar.
Student Attendance Cumulative Days Present Yes
Student Attendance Num Unexcused Absence Yes
Student Enrollment Expected Grad Year Yes
3.9 How do district/school changes in the percent of students who pass No gap.
AP courses and ACT, SAT, and IB exams compare to state trends?
Section Course Course Designation Code Yes
Section Grade Letter Grade Yes
Section Course Course ID Yes
Student Assessment SAT/ACT/IB exam results Yes
3.10 What is the high school preparation profile of students who Data related to post
successfully complete post secondary education? secondary education are
required.
Section Course Course Designation Code Yes

June 2010 Page 37


Washington State K–12
K Education
Data Gap Analysis

Table 10. Research and Policy Questions Gaps


Question Entity Category Attribute Exists? Notes
Section Grade Letter Grade Yes
Section Course Course ID Yes
Student Enrollment Enrolled in a Post Secondary No
Institution
Student Enrollment Post Secondary Exit Code No
3.11 What are the characteristics of districts/schools that meet or do not Additional funding data
meet accountability requirements, i.e., funding, programs and course may be required.
offerings, average class size, staff allocations, and teacher qualifications?

School AYP AYP Status Yes


School Type REAP Alternative Funding No
Indicator
School Directory School Code Yes
Section Course Course ID Yes
Section Section Section ID Yes
Student Enrollment School Code Yes
Student Enrollment Program Code Yes
Student Demographic Race/Ethnicity Yes
Student Demographic Gender Yes
Student SpEd Disability Code Yes
Student LEP Start Date Yes
Student LEP Exit Date Yes
Staff Demographic Race/Ethnicity Yes
Staff Assignment School Code Yes
Staff Assignment Course ID Yes
Staff Credentials Staff Type Code Yes
Staff Credentials Certification Status Yes
Staff Credentials HQT Certification Status Yes
3.12 What are the characteristics of districts/schools that show the No gap.
greatest success in helping low achieving students reach proficiency?
Student Assessment GX Assessment Perf Level Yes
Student Enrollment School Code Yes
Student Enrollment Program Code Yes
Student Demographic Race/Ethnicity Yes
Student Demographic Gender Yes
Student SpEd Disability Code Yes
Student LEP Start Date Yes
Student LEP Exit Date Yes
Student Attendance Cumulative Days Present Yes
Student Attendance Number of Days in No Can be derived based on
Membership school calendar.
School AYP AYP Status Yes
School Directory School Code Yes
Section Course Course ID Yes

Page 38 June 2010


Washington State K–12
K Education
Data Gap Analysis

Table 10. Research and Policy Questions Gaps


Question Entity Category Attribute Exists? Notes
Section Section Section ID Yes
Staff Demographic Race/Ethnicity Yes
Staff Assignment School Code Yes
Staff Assignment Course ID Yes
Staff Credentials Staff Type Code Yes
Staff Credentials Certification Status Yes
Staff Credentials HQT Certification Status Yes
3.13 What are the characteristics of districts/schools that show the No gap.
greatest success in improving the performance of students in special Recommend collecting
education and ELL programs? more detailed program
information.
Student Assessment GX Assessment Perf Level Yes
Student Enrollment School Code Yes
Student Enrollment Program Code Yes
Student Demographic Race/Ethnicity Yes
Student Demographic Gender Yes
Student SpEd LRE Code Yes
Student SpEd IDEA Disability Status No Can be derived from
Disability Code
Student LEP Start Date Yes
Student LEP Exit Date Yes
Student Assessment Assessment Achieved Yes
Standard (Alternative
Assessments)
Student Attendance Cumulative Days Present Yes
Student Attendance Number of Days in No Can be derived based on
Membership school calendar.
School AYP AYP Status Yes
School Directory School Code Yes
Section Course Course ID Yes
Section Section Section ID Yes
Staff Demographic Race/Ethnicity Yes
Staff Assignment School Code Yes
Staff Assignment Course ID Yes
Staff Credentials Staff Type Code Yes
Staff Credentials Certification Status Yes
Staff Credentials HQT Certification Status Yes
Attendance, Discipline, Dropout, and Graduation Rates
4.1 What are the characteristics of high attendance and low attendance Need data related to Title I
students by school, grade level, and subgroup? participation type to aid in
analysis.
Student Demographic Race/Ethnicity Yes
Student Demographic Economic Disadvantaged Yes
Status

June 2010 Page 39


Washington State K–12
K Education
Data Gap Analysis

Table 10. Research and Policy Questions Gaps


Question Entity Category Attribute Exists? Notes
Student Demographic Gender Yes
Student SPED Primary Disability Type Yes
Student LEP LEP Status Yes
Student Attendance Number of Days in No Can be derived based on
Membership school calendar.
Student Attendance Cumulative Days Present Yes
Student Attendance Num Unexcused Absence Yes
Student Enrollment School Code Yes
Student Enrollment Grade Level Yes
Student Enrollment School Enrollment Date Yes
Student Enrollment District Enrollment Date Yes
Student Enrollment Program Code Yes
Student Assessment GX Assessment Perf Level Yes
Student SpEd Disability Code Yes
Student LEP Start Date Yes
Student LEP Exit Date Yes
Student Title I Title I Participant Type No
4.2 How have district/school subgroup attendance patterns changed at No gap.
different grade levels?
Student Demographic Race/Ethnicity Yes
Student Demographic Economic Disadvantaged Yes
Status
Student SPED Primary Disability Type Yes
Student LEP LEP Status Yes
Student Attendance Number of Days in No Can be derived based on
Membership school calendar.
Student Attendance Cumulative Days Present Yes
Student Attendance Num Unexcused Absence Yes
Student Enrollment School Code Yes
Student Enrollment Grade Level Yes
4.4 What is the distribution of dropouts over the school year by subgroup No gap.
and which groups have the highest dropout rates?
Student Demographic Race/Ethnicity Yes
Student Demographic Economic Disadvantaged Yes
Status
Student SPED Primary Disability Type Yes
Student LEP LEP Status Yes
Student Enrollment School Exit Code Yes
Student Enrollment School Exit Date Yes
4.5 What are the characteristics of students in a school who have been Data
ata related to
involved in discipline incidents, suspended, expelled, or dropped out of incident/discipline data are
school? required.
Student Demographic Race/Ethnicity Yes
Student Demographic Economic Disadvantaged Yes

Page 40 June 2010


Washington State K–12
K Education
Data Gap Analysis

Table 10. Research and Policy Questions Gaps


Question Entity Category Attribute Exists? Notes
Status
Student SPED Primary Disability Type Yes
Student LEP LEP Status Yes
Student Enrollment School Code Yes
Student Enrollment School Exit Code Yes
Student Discipline Number of Days Suspended No
Student Discipline Number of Days Expelled No
Incident Instance Student Unique ID No
Incident Instance Incident Type No
Incident Instance Type of Discipline No
4.6 How do increases or decreases in district/school dropout rates by No gap.
subgroup compare to state dropout rates and dropout rates in similar
districts/schools?
Student Demographic Race/Ethnicity Yes
Student Demographic Economic Disadvantaged Yes
Status
Student SPED Primary Disability Type Yes
Student LEP LEP Status Yes
Student Enrollment School Code Yes
Student Enrollment School Exit Code Yes
4.7 How do district/school NCLB graduation rates for subgroups compare No gap.
to state graduation rates and graduation rates in similar
districts/schools?
Student Demographic Race/Ethnicity Yes
Student Demographic Economic Disadvantaged Yes
Status
Student SPED Primary Disability Type Yes
Student LEP LEP Status Yes
Student Enrollment School Code Yes
Student Enrollment School Exit Code Yes
Student Enrollment Grade Level Yes Used to determine if
student is retained.
Success/Risk Indicators, and K–12
12 Transitions
5.1 What is the relationship between absence and performance on state No gap.
assessments for different subgroups?
Student Demographic Race/Ethnicity Yes
Student Demographic Economic Disadvantaged Yes
Status
Student SPED Primary Disability Type Yes
Student LEP LEP Status Yes
Student Attendance Number of Days in No Can be derived based on
Membership school calendar.
Student Attendance Number of Days Truant Yes
Student Attendance Number of Days in Yes

June 2010 Page 41


Washington State K–12
K Education
Data Gap Analysis

Table 10. Research and Policy Questions Gaps


Question Entity Category Attribute Exists? Notes
Attendance
Student Attendance Attendance Rate No Can be derived.
derive
Student Assessment Proficiency Level Yes
5.2 What is the relationship between grades and performance on state No gap.
assessments?
Student Enrollment School Code Yes
Student Enrollment Grade Level Yes
Section Course Letter Grade Yes
Student Assessment GX Assessment Perf Level Yes
5.3 What are the attendance patterns and proficiency levels of students No gap.
who drop out by subgroup?
Student Demographic Race/Ethnicity Yes
Student Demographic Economic Disadvantaged Yes
Status
Student SPED Primary Disability Type Yes
Student LEP LEP Status Yes
Student Attendance Number of Days in No Can be derived based on
Membership school calendar.
Student Attendance Cumulative Days Present Yes
Student Attendance Num Unexcused Absence Yes
Student Enrollment School Code Yes
Student Assessment GX Assessment Perf Level Yes
Student Enrollment School Exit Code Yes
5.4 What were the early indicators of success or failure for students in an No gap.
elementary school, i.e., what is the K––3 profile of students who either A policy decision is
succeeded or failed? required to define
"success" or "failure".
Student Demographic Birth Date Yes
Student Demographic Years over age for grade Yes Can be derived based on
Date of Birth and Grade
Level.
Student Attendance Number of Days in No Can be derived based on
Membership school calendar.
Student Attendance Cumulative Days Present Yes
Student Attendance Num Unexcused Absence Yes
Student Enrollment School Code Yes
Student Enrollment Grade Level Yes
Student Enrollment Program Code Yes
Student Assessment GX Assessment Perf Level Yes
Section Grade Letter Grade Yes
5.5 What are the strongest elementary school indicators of success or No gap.
failure in the transition from elementary school to middle school, i.e., A policy decision is
what is the elementary school profile of students who succeed or fail in required to define
middle school? "success" or "failure".

Page 42 June 2010


Washington State K–12
K Education
Data Gap Analysis

Table 10. Research and Policy Questions Gaps


Question Entity Category Attribute Exists? Notes
Student Demographic Race/Ethnicity Yes
Student Demographic Economic Disadvantaged Yes
Status
Student SPED Primary Disability Type Yes
Student LEP LEP Status Yes
Student Enrollment School Code Yes
Student Assessment G3-8 Assessment Perf Level Yes
Student Enrollment Grade Level Yes
Student Enrollment Program Code Yes
Section Grade Letter Grade Yes
Section Course Course ID Yes
5.6 What are the strongest middle school indicators of success or failure No gap.
in the transition from middle school to high school, i.e., what is the A policy decision is
middle school profile of students who either succeeded or failed? required to define
"success" or "failure".
Student Demographic Race/Ethnicity Yes
Student Demographic Economic Disadvantaged Yes
Status
Student SPED Primary Disability Type Yes
Student LEP LEP Status Yes
Student Enrollment School Code Yes
Student Assessment GX Assessment Perf Level Yes
Student Enrollment Grade Level Yes
Student Enrollment Program Code Yes
Section Grade Letter Grade Yes
Section Course Course ID Yes
5.7 How are students from specific high schools performing at the post Need to collect
co data
secondary level, and what are the strongest predictors of post secondary related to post secondary
success, i.e., what is the high school profile of students who succeed at information. May be
the post secondary level? informed by National
Student Clearinghouse
data if available.
Student Demographic Race/Ethnicity Yes
Student Demographic Economic Disadvantaged Yes
Status
Student SPED Primary Disability Type Yes
Student LEP LEP Status Yes
Student Enrollment School Code Yes
Student Assessment GX Assessment Perf Level Yes
Student Enrollment Grade Level Yes
Student Enrollment Program Code Yes
Section Grade Letter Grade Yes
Section Course Course ID Yes
Student Enrollment Enrolled in a Post Secondary No
Institution

June 2010 Page 43


Washington State K–12
K Education
Data Gap Analysis

Table 10. Research and Policy Questions Gaps


Question Entity Category Attribute Exists? Notes
Student Enrollment Post Secondary Exit Code No
Student Assessment SAT/ACT/IB exam results Yes
Section Grade Post Secondary Grade No
School Type Post Secondary Institution No
Student Enrollment Post Secondary Entry Date No
Student Enrollment Post Secondary Exit Date No
Section Grade GPA Yes
5.8 What is the previous academic and attendance record of students in No gap.
this school who are new to the district?
Student Enrollment School Code Yes
Student Enrollment Grade Level Yes
Section Course Letter Grade Yes
Student Assessment GX Assessment Perf Level Yes
Student Attendance Number of Days in No Can be derived based on
Membership school calendar.
Student Attendance Cumulative Days Present Yes
Student Attendance Num Unexcused Absence Yes
Student Enrollment District Enrollment Date Yes
Student Enrollment School Enrollment Date Yes
Student Enrollment School Entry Code Yes
Program Outcomes
6.1 What reading and mathematics programs/interventions have shown No gap in elements. Need
the most success in increasing student proficiency at the elementary, a way to identify similar
middle, and high school levels in similar districts/schools? schools/districts.

Student Assessment GX Assessment Perf Level Yes


Student Enrollment Grade Level Yes
Student Enrollment Program Code Yes
Section Section Section ID Yes
Section Course Course ID Yes
Section Grade Letter Grade Yes
6.2 What dropout prevention programs have shown the most success in Data related to dropout
decreasing dropout rates in similar districts/schools? prevention are required.
Student Enrollment Exit Reason Code Yes
Student Enrollment Program Code No CEDARS collects Program
Code, but it does not
include dropout
prevention program
information.
6.3 What programs, services, and instructional models have shown the Data related to
most success in improving the performance of students in special instructional programs at
education and ELL programs in similar districts/schools? the school level are
required. Teacher
observation data would
provide a richer analysis.

Page 44 June 2010


Washington State K–12
K Education
Data Gap Analysis

Table 10. Research and Policy Questions Gaps


Question Entity Category Attribute Exists? Notes
Student Enrollment School Code Yes
Student SpEd Disability Code Yes
Student SpEd LRE Code Yes
Student SpEd Start Date Yes
Student SpEd Exit Reason Code Yes
Student SpEd Exit Date Yes
Student Enrollment Program Code Yes
Student LEP Start Date Yes
Student LEP Exit Date Yes
Student LEP Exit Reason Code Yes
Student LEP Placement Test Date Yes
Student LEP Assessed on English Language No Can be derived based on
Proficiency Placement Test Date.
Student LEP Placement Test Level Score Yes
Student LEP Progress/Attainment in No
Language
Student LEP Primary Language Code Yes
Student LEP Placement Test Scale Score Yes
Student LEP Initial WA Placement Date Yes
Student LEP Initial USA Placement Date Yes
Student Assessment Assessment Achieved No
Standard (Alternative
Assessments)
Section Section Section ID Yes
Section Course Course ID Yes
School Type Supplemental Services No
Provided
District Directory Instructional Model Code Yes Instructional model is only
collected at the district
level, school level will also
be necessary.
School Directory Other program, services, No
models
Staff Credential Staff Type Code Yes
Staff Credential Teaching Field Authorized Yes
Area
Staff Credential Paraprofessional Qualification No
Status
Staff Credential Certification Status Yes
Staff Credential Highest Level of Education Yes
Completed
Staff Credential HQT Certification Status Yes
Staff Credential Teaching Credential Type Yes
Staff Credential Technology Standards Met No
Staff Experience Years of Prior Teaching Yes

June 2010 Page 45


Washington State K–12
K Education
Data Gap Analysis

Table 10. Research and Policy Questions Gaps


Question Entity Category Attribute Exists? Notes
Experience
Staff Assignment School Code Yes
Staff Assignment Staff Category Yes
Staff Assignment Course ID Yes
Teacher Workforce and Student Achievement
7.2 What are the differences in qualifications and experiences of Need to collect additional
teachers across classrooms, i.e., is the quality of the teachers equitable data relating to staff.
across classrooms and different achievement levels?
Staff Experience Years of Prior Teaching Yes
Experience
Staff Assignment School Code Yes
Staff Assignment Staff Category Yes
Staff Assignment Course ID Yes
Staff Credential Staff Type Code Yes
Staff Credential Teaching Field Authorized Yes
Area
Staff Credential Paraprofessional Qualification No
Status
Staff Credential Certification Status Yes
Staff Credential Highest Level of Education Yes
Completed
Staff Credential HQT Certification Status Yes
Staff Credential Teaching Credential Type Yes
Staff Credential Technology Standards Met No
7.5 What are the characteristics of teachers who show the greatest No gap. For a richer
success in improving student achievement? analysis, additional growth
data are required.
Student Assessment Yes
GX
Assessment
Perf Level
Student Enrollment Grade Level Yes
Student Enrollment Program Code Yes
Section Section Section ID Yes
Section Course Course ID Yes
Section Grade Letter Grade Yes
Staff Demographic Race/Ethnicity Yes
Staff Assignment Course ID Yes
Staff Identity Certification Number Yes
Staff Assignment Staff Category Yes
Staff Credential Staff Type Code Yes
Staff Credential Teaching Credential Type Yes
Staff Experience Years of Prior Teaching Yes
Experience

Page 46 June 2010


Washington State K–12
K Education
Data Gap Analysis

Table 10. Research and Policy Questions Gaps


Question Entity Category Attribute Exists? Notes
Staff Assignment School Code Yes
7.6 What are the most common characteristics of the teacher workforce No gap. For a richer
in schools that show the greatest success with students? analysis, additional growth
data are required. A policy
decision is required to
define "greatest success".
Student Assessment Yes
GX
Assessment
Perf Level
Student Enrollment Grade Level Yes
Student Enrollment Program Code Yes
Section Section Section ID Yes
Section Course Course ID Yes
Section Grade Letter Grade Yes
Staff Demographic Race/Ethnicity Yes
Staff Assignment Course ID Yes
Staff Identity Certification Number Yes
Staff Assignment Staff Category Yes
Staff Credential Staff Type Code Yes
Staff Credential Teaching Credential Type Yes
Staff Experience Years of Prior Teaching Yes
Experience
Staff Assignment School Code Yes
Staff Certification HQT Certification Status Yes
7.7 What are the characteristics of elementary classrooms, e.g., class Additional staff data
size, student demographics, paraprofessional support, that show the needed. For a richer
greatest success in improving student proficiency? analysis, additional growth
data are required.
Student Assessment GX Assessment Perf Level Yes
Student Enrollment School Code Yes
Student Enrollment Grade Level Yes
Student Demographic Race/Ethnicity Yes
Student Demographic Gender Yes
Student Demographic Economic Disadvantaged Yes
Status
Section Section Section ID Yes
Section Course Course ID Yes
Staff Credential Paraprofessional Qualification No
Status

June 2010 Page 47


Washington State K–12
K Education
Data Gap Analysis

Table 10. Research and Policy Questions Gaps


Question Entity Category Attribute Exists? Notes
7.8 What were the pre-service
service programs of teachers who have high Data related to staff are
student success rates over time? required. For a richer
analysis, additional growth
data are required. A policy
decision is required to
define "high student
success".
Student Assessment GX Assessment Perf Level Yes
Student Enrollment Grade Level Yes
Student Enrollment Program Code Yes
Section Section Section ID Yes
Section Course Course ID Yes
Section Grade Letter Grade Yes
Staff Assignment Course ID Yes
Staff Identity Certification Number Yes
Staff Assignment Staff Category Yes
Staff Credential Staff Type Code Yes
Staff Credential Teaching Credential Type Yes
Staff Experience Pre-Service Program No
7.10 What is the relationship between the frequency and types of Data related to staff and
professional development provided in reading and mathematics, and professional development
improvements in state assessment results? are required.

Student Assessment GX Assessment Perf Level Yes


Student Enrollment Grade Level Yes
Section Section Section ID Yes
Section Course Course ID Yes
Staff Assignment Course ID Yes
Staff Experience Professional Development No
Course
Staff Experience Number of Professional No
Development Hours
Staff Experience Professional Development No
Course Start Date
Staff Experience Professional Development No
Course End Date
Cost Effectiveness/Benefits – Return on Investment (ROI)/Cost Analyses
8.1 What is the cost effectiveness of specific district/school programs, A policy decision is
i.e., what are the per pupil costs (personnel and program material costs) required to define "cost
of programs that have improved the performance of specific subgroups? effectiveness." Program
cost data are in iGrants,
but is not broken down to
the pupil level.
Section Course Course ID Yes
Section Assignment Section ID Yes
Student Enrollment School Code Yes

Page 48 June 2010


Washington State K–12
K Education
Data Gap Analysis

Table 10. Research and Policy Questions Gaps


Question Entity Category Attribute Exists? Notes
Student Demographic Race/Ethnicity Yes
Student Demographic Economic Disadvantaged Yes
Status
Student SPED Primary Disability Type Yes
Student LEP LEP Status Yes
Section Grade Letter Grade Yes
Student Assessment GX Assessment Perf Level Yes
Staff Assignment School Code Yes
Staff Assignment Course ID Yes
District Indicator Federal Programs Offered No
Staff Assignment Total Salary No
School Cost Program Yes
School Cost Classroom No
8.2 What are the cost benefits of federally funded supplemental Need additional funding
programs in meeting measurable student achievement targets, i.e., what data.
were the per pupil expenditures of these programs and what percent of
students met achievement targets?
School Type School Improvement Funds No
Allocation
School Type AMAO Progress Attainment No
Status for LEP Students
School Type AMAO Proficiency Attainment No
Status for LEP Students
School Type REAP Alternative Funding No
Indicator
School Type Supplemental Services No
Provided
District Indicator Federal Programs Offered No
District Indicator Funding Allocation Type No
Section Grade Letter Grade Yes
Student Assessment GX Assessment Perf Level Yes
8.3 What are the cost benefits of professional development expenditures Need professional
targeted to specific subject areas and programs, i.e., what percent of development data for staff
in-service
service teachers’ students show improvements over time in the areas and need to be able to
targeted by professional development? directly link that training to
a specific course.
Staff Experience Professional Development No
Course
Staff Experience Number of Professional No
Development Hours
Staff Experience Professional Development No
Course Start Date
Staff Experience Professional Development No
Course End Date
Staff Experience Cost of Professional No

June 2010 Page 49


Washington State K–12
K Education
Data Gap Analysis

Table 10. Research and Policy Questions Gaps


Question Entity Category Attribute Exists? Notes
Development program
Section Course Course ID Yes
Section Assignment Section ID Yes
Student Enrollment School Code Yes
Section Grade Letter Grade Yes
Student Assessment GX Assessment Perf Level Yes
8.4 What are the cost benefits of professional development expenditures Need data on professional
focused on teacher retention, i.e., comparison of costs of recruiting vs. development and internal
the costs of professional development? processes for recruiting
new staff.
Staff Experience Professional Development No
Course
Staff Experience Number of Professional No
Development Hours
Staff Experience Professional Development No
Course Start Date
Staff Experience Professional Development No
Course End Date
Staff Experience Cost of Professional No
Development program
Staff Assignment Contract Beginning Date No
Staff Assignment Term End Date Yes
School Staff Cost of Recruitment No
Cost Analyses
9.3 What is the instructional cost breakout by federal, state, and local Need the cost information
revenues at the district, school, program, and classroom levels? for each of the programs,
courses by class, and
schools. Cost per pupil
Section Section Section ID Yes
Section Course Course ID Yes
School Directory School Code Yes
Student Enrollment Program Code Yes
School Cost Program Yes
School Cost School Yes
School Cost Classroom No
9.5 What are the cost “savings” attributable to specific management Need to document cost
actions such as process improvements in the IT process to improve desk and processes in place at
response capabilities? the school and district level
to be able to review costs
over time.
School Internal Type No
Processes
District Internal Type No
Processes
School Internal Resources No

Page 50 June 2010


Washington State K–12
K Education
Data Gap Analysis

Table 10. Research and Policy Questions Gaps


Question Entity Category Attribute Exists? Notes
Processes
District Internal Resources No
Processes
School Cost Process No
District Cost Process No
9.7 At the aggregate level, what is the resource consumption (personnel Need additional staff data.
and non-personnel)
personnel) for the major expense categories defined by the
district, i.e., regular education, special education, vocational education,
administration, transportation, maintenance, etc.?

Staff Identity Certification Number Yes


Staff Assignment Staff Category Yes
Staff Assignment Instructional Grade Level Yes
Staff Assignment Age Group Taught (Special Yes Can be derived.
Education)
Staff Assignment Course ID Yes
Staff Assignment Migrant Education Program No
Staff Category
Staff Credential Staff Type Code Yes
Staff Credential Teaching Credential Type Yes
Staff Credential Special Education Program No
Contracted Services
Staff Credential Title III/LEP Instructor No
Credential Type
Staff Type Assignment Type Yes
District Cost Transportation Yes

June 2010 Page 51


Washington State K–12
K Education
Data Gap Analysis

SUMMARY RECOMMENDATIONS
The Office of Superintendent of Public Instruction has taken a number of steps towards improving and
tracking student achievement, including adoption of common standards, and the recent introduction of
CEDARS. With 295 school districts ranging in size from fewer than 100 students to more than 45,000
students, managing these efforts is a significant challenge.

To help manage the data requirements of the state and federal government and meet the Legislative
intent for a statewide longitudinal data system, OSPI intends to leverage the CEDARS data warehouse
once it is built as the primary vehicle for data collection and reporting. Although CEDARS collects a
significant number of data elements across important educational domains, it is in the early stages of
implementation
plementation with plans for further development as a full data warehouse.

Discussions with OSPI data managers and well as key state stakeholders interviewed through the
Research and Policy Questions portion of the project revealed a consistent focus on the need and desire
for the ability to collect, retrieve, and analyze quality data in order to guide instruction and improve
student achievement as well as meet the reporting requirements of the state legislature and federal
government. To do this will require re consolidation of many of the agency’s disparate data collections into
a comprehensive longitudinal data system. This comprehensive data system, along with a rigorous and
structured metadata documentation process, will allow for uniformity in definition, standards, and use.
As mentioned, Washington has a robust student data collection system in CEDARS but no data
warehouse or reporting solution. Washington is currently in the process of releasing an RFP to procure
and develop the data warehouse in accorda
accordance
nce with state requirements and the vision specified in their
successful 2009 SLDS grant award.

The following table displays recommendations gathered and synthesized through the interview process
and validated against the data dictionary. Please see the Washington Metadata Workbook for all
identified gaps. There are six major recommendations followed by supporting significant and minor
recommendations.

Table 11.. Summary Recommendations


ID Recommendation / Gap Discussion
1 Use the SharePoint workbook created OSPI and ERDC now have a significant resource available
through this project as the common data through the metadata mapping contained in the
dictionary to guide development of the OSPI Workbook. Both agencies would benefit from the
K-12 and ERDS P-2020 SLDS data warehouses continued development of the workbook and data
and data marts. roadmap.

2 Enable valid teacher effect calculations Although Washington is moving ahead with plans to
based on student growth percentiles
percentiles. implement a student growth model based on the Colorado
Student Growth Percentile approach,, include explicit plans
to link to teacher for the purpose of providing additional
insights and evaluation models supported in Race to the
Top.

Page 52 June 2010


Washington State K–12
K Education
Data Gap Analysis

2.1 Calculate and load student growth Include in data warehouse in order to expose
xpose to reporting
percentile into CEDARS data warehouse capabilities once built.
once built
2.2 Establish section entrance and exit for Currently course attendance is snapshot based.
class roster in CEDARS. Class schedule by
course by date.
2.3 Create Current, Prior Year 1 assessment Support longitudinal growth structure recommended by
score growth. NEDM.

3 Develop student drop-out


out / early warning Washington is examining this issue through the Building
prevention and reporting module using the Bridges Workgroup. Incorporation of at risk factors in a
ABC indicators recommended in the NGA state longitudinal data system offers distinct advantages
report (Absence, Behavior, Course Grade, over local systems for understanding risk at the state level.
and Over Age for Grade) Washington should examine drop-out out early warning
systems in the context of response
onse to intervention and
positive behavior solutions to provide the necessary
support for at risk students.
3.1 Collect student and incident level This was a theme echoed consistently throughout the
discipline data through CEDARS. project in order to establish critical cross linkage of data and
answer Research and Policy questions of interest.
3.2 Improve student attendance attributes OSPI has the foundation in place to collect count of days
to enable accurate accounting of student attended but lacks the ability to determine an excused
excused absences and school calendars. absence. Either define excused versus unexcused absence or
collect school calendar to determine attendance. Create
physical database structure to allow collection of daily
attendance in the future.
3.3 Extend course classification to all grades. OSPI has intentions to “turn on validation” thus improving
the use of the codes.

4 Replace teacher certification system with The certification system currently lacks many of the
one capable of collecting all required features requested via research and policy questions as
educator information including postpost- well as requires error-prone
prone manual intervention.
intervent
secondary performance and relevant major.
4.1 Develop plans to phase out paper
systems / collections: CTE, eCert, Special
Education discipline, e.g.
4.2 Data in eCertification is not connected to Data is manually entered twice.
Certificate DB; data not directly used.
4.3 Collect degree information and Significant interest was expressed in having more clear
institution related to certification. information on teacher education background
4.4 Extend system to maintain professional Vision for system extends to include tracking a teacher’s
growth plans connecting specific course entire history and their academic credentials including their
schedules
hedules and student outcomes with course, continuing education, degree, certificates,
teacher qualifications. endorsements, etc.

June 2010 Page 53


Washington State K–12
K Education
Data Gap Analysis

5 Commit to a feasibility study to use CEDARS Recommend detailed studies of variance of possible
data to drive apportionment. Run multiple funding using CEDARS as first step in determining district
models approximating Apportionment FTEs level differences between accounting methods.
with CEDARS head counts. Determine
variance. Design legislative action as
needed.
5.1 Washington should expand its chart of
accounts for all school financial
transactions and report the transaction
data to OSPI for analysis and
comparisons within the state data
warehouse once built.

6 OSPI should establish a database of record Although the CEDARS data ta warehouse does not yet exist,
for each data element in the EDFacts when established it should contain data snapshots for all
collections depending on the required official EDFacts reports.
reporting period. Those data can then be
published to the data warehouse as the
official record of the submission.
6.1 Build EDFacts data mart as part of data
warehouse.

Page 54 June 2010


Washington State K–12
K Education
Data Gap Analysis

APPENDIX

A. Excerpts from ESHB 2261


July 16, 2009

K-12
12 Education Data System: Legislative Expectations
Excerpt from ESSB 2261

NEW SECTION. Sec. 202. A new section is added to chapter 28A.300 RCW to read as follows:

Legislative Intent
(1) It is the legislature's intent to establish a comprehensive K K-12
12 education data
improvement system for financial, student, and educator data. The objective of the system is to
monitor student progress, have information on the quality of the educator workforce, wo
monitor and analyze the costs of programs, provide for financial integrity and
accountability, and have the capability to link across these various data components by
student, by class, by teacher, by school, by district, and statewide. Education data
systems must be flexible and able to adapt to evolving needs for information, but there must be
an objective and orderly data governance process for determining when changes are needed
and how to implement them. It is the further intent of the legisla
legislature
ture to provide independent
review and evaluation of a comprehensive K K-12
12 education data improvement system by
assigning the review and monitoring responsibilities to the education data center and the
legislative evaluation and accountability program commi committee.

Clients
(2) It is the intent that the data system specifically service reporting requirements for
teachers, parents, superintendents, school boards, the legislature, the office of the
superintendent of public instruction, and the public.

Data System
stem Features: Legislative Intent
(3) It is the legislature's intent that the K-12
12 education data improvement system used
by school districts and the state include but not be limited to the following information and
functionality:

(a) Comprehensive educ educator information,, including grade level and courses taught,
building or location, program, job assignment, years of experience, the institution of
higher education from which the educator obtained his or her degree, compensation,
class size, mobility of class
ass population, socioeconomic data of class, number of
languages and which languages are spoken by students, general resources available for
curriculum and other classroom needs, and number and type of instructional support
staff in the building;

(b) The capacity to link educator assignment information with educator certification
information such as certification number, type of certification, route to certification,
certification program, and certification assessment or evaluation scores;

June 2010 Page 55


Washington State K–12
K Education
Data Gap Analysis

(c) Common coding


ding of secondary courses and major areas of study at the
elementary level or standard coding of course content
content;

(d) Robust student information


information,, including but not limited to student characteristics,
course and program enrollment, performance on statewide and district summative
and formative assessments to the extent district assessments are used, and
performance on college readiness tests;

(e) A subset of student information elements to serve as a dropout early warning


system;

(f) The capacity to link educator information with student information;


information

(g) A common, standardized structure for reporting the costs of programs at the
school and district level with a focus on the cost of services delivered to students;

(h) Separate accounting of state, federal, and local revenues and costs;
costs

(i) Information linking state funding formulas to school district budgeting and
accounting,, including procedures:
(i) To support the accuracy and auditing of financial data;; and
(ii) Using the prototypical school model for school district financial accounting
reporting;

(j) The capacity to link program cost information with student performance
information to gauge the cost-effectiveness of programs;

(k) Information that is centrally accessible and updated regul


regularly;; and

(l) An anonymous, nonidentifiable replicated copy of data that is updated at least


quarterly, and made available to the public by the state.

District Data Systems Export Requirement


(4) It is the legislature's goal that all school districts have the capability to collect state-
state
identified common data and export it in a standard format to support a statewide K-12 K
education data improvement system under this section.

Reports
(5) It is the legislature's intent that the K
K-12
12 education data improvement system be
developed to provide the capability to make reports as required under section 203 of this act
available.

Legislative Funding for New Data Elements Required


(6) It is the legislature's
ature's intent that school districts collect and report new data elements
to satisfy the requirements of RCW 43.41.400, this section, and section 203 of this act, only to
the extent funds are available for this purpose.

Page 56 June 2010


Washington State K–12
K Education
Data Gap Analysis

July 16, 2009

K-12
12 Education Data System: Legislative Expectations
Excerpt from ESSB 2261

NEW SECTION. Sec. 203. A new section is added to chapter 28A.300 RCW to read as follows:

Purpose
(1) A K-12
12 data governance group shall be established within the office of the superintendent of
public instruction to assist in the design and implementation of a K K-1212 education data
improvement system for financial, student, and educator data data. It is the intent that the data
system reporting specifically serve requirements for teachers, parents, superintendents,
school boards, the office of the superintendent of public instruction, the legislature, and
the public. Membership

(2) The K-12 data governance group shall include representatives of the education data center,
the office of the superintendent of public instruction, the legislative evaluation and accountability
program committee, the professional educator standards board, the state board of education,
educati
and school district staff, including information technology staff. Additional entities with expertise
in education data may be included in the K K-12 data governance group.

Duties
12 data governance group shall:
(3) The K-12
(a) Identify the critical re
research and policy questions that need to be addressed by
the K-12
12 education data improvement system;

(b) Identify reports and other information that should be made available on the
internet in addition to the reports identified in subsection (5) of this section;

(c) Create a comprehensive needs requirement document detailing the specific


information and technical capacity needed by school districts and the state to meet the
legislature's expectations for a comprehensive K-12 12 education data improvement
system
ystem as described under section 202 of this act;

(d) Conduct a gap analysis of current and planned information compared to the
needs requirement document
document,, including an analysis of the strengths and limitations of
an education data system and programs cu currently
rrently used by school districts and the state,
and specifically the gap analysis must look at the extent to which the existing data can
be transformed into canonical form and where existing software can be used to meet the
needs requirement document;

(e) Focus on financial and cost data necessary to support the new K-12
12 financial
models and funding formulas, including any necessary changes to school district
budgeting and accounting, and on assuring the capacity to link data across financial,
student, and educator systems; and

(f) Define the operating rules and governance structure for K K-12
12 data collections,
collections
ensuring that data systems are flexible and able to adapt to evolving needs for
information, within an objective and orderly data governance proc
process
ess for determining

June 2010 Page 57


Washington State K–12
K Education
Data Gap Analysis

when changes are needed and how to implement the them.


m. Strong consideration must be
made to the current practice and cost of migration to new requirements. The operating
rules should delineate the coordination, delegation, and escalation au authority
thority for data
collection issues, business rules, and performance goals for each K K-12
12 data collection
system, including:
(i) Defining and maintaining standards for privacy and confidentiality;
confidentiality
(ii) Setting data collection priorities
priorities;
(iii) Defining and
d updating a standard data dictionary;
(iv) Ensuring data compliance with the data dictionary;
(v) Ensuring data accuracy
accuracy; and
(vi) Establishing minimum standards for school, student, financial, and
teacher data systems
systems.. Data elements may be specified "to the extent feasible"
or "to the extent available" to collect more and better data sets from districts with
more flexible software. Nothing in RCW 43.41.400, this section, or section 202 of
this act should be construed ttoo require that a data dictionary or reporting should
be hobbled to the lowest common set. The work of the K K-12
12 data governance
group must specify which data are desirable. Districts that can meet these
requirements shall report the desirable data. Funding from the legislature must
establish which subset data are absolutely required
required.

Updates and oversight


(4) (a) The K-12
12 data governance group shall provide updates on its work as requested
by the education data center and the legislative evaluation and acaccountability
countability
program committee.

(b) The work of the K-12


12 data governance group shall be periodically reviewed and
monitored by the educational data center and the legislative evaluation and
accountability program committee.

Reports
ata is available, the office of the superintendent of public instruction shall
(5) To the extent data
make the following minimum reports available on the internetinternet.. The reports must either be
run on demand against current data, or, if a static report, must have been run against the
t most
recent data:

(a) The percentage of data compliance and data accuracy by school district;

(b) The magnitude of spending per student


student,, by student estimated by the following
algorithm and reported as the detailed summation of the following components:
componen
(i) An approximate, prorated fraction of each teacher or human resource element
that directly serves the student. Each human resource element must be listed or
accessible through online tunneling in the report;
(ii) An approximate, prorated fraction of classroom or building costs used by the
student;
(iii) An approximate, prorated fraction of transportation costs used by the student;
and
(iv) An approximate, prorated fraction of all other resources within the district.
d
District-wide
wide components should be disaggregated to the extent that it is sensible
and economical;

Page 58 June 2010


Washington State K–12
K Education
Data Gap Analysis

(c) The cost of K-1212 basic education


education,, per student, by student, by school district,
estimated by the algorithm in (b) of this subsection, and repor
reported
ted in the same manner as
required in (b) of this subsection;

(d) The cost of K-12


12 special education services per student
student,, by student receiving
those services, by school district, estimated by the algorithm in (b) of this subsection,
and reported in the same manner as required in (b) of this subsection;

(e) Improvement on the statewide assessments computed as both a percentage


change and absolute change on a scale score metric by district, by school, and by
teacher that can also be filtered by a student
student's length of full-time
time enrollment within the
school district;

(f) Number of K-12


12 students per classroom teacher on a per teacher basis;

(g) Number of K-12


12 classroom teachers per student on a per student basis;

(h) Percentage of a classroom teacher p


per student on a per student basis; and

(i) The cost of K-12


12 education per student by school district sorted by federal, state,
and local dollars.

Reports
(6) The superintendent of public instruction shall submit a preliminary report to the legislature
by November 15, 2009,, including the analyses by the K K-12
12 data governance group under
subsection (3) of this section and preliminary options for addressing identified gaps. A final
report,, including a proposed phase
phase-in plan and preliminaryy cost estimates for implementation of
a comprehensive data improvement system for financial, student, and educator data shall be
submitted to the legislature by September 1, 2010
2010.

Technical requirements for submitting data


(7) All reports and data referenced in this section, RCW 43.41.400, and section 202 of this act
shall be made available in a manner consistent with the technical requirements of the legislative
evaluation and accountability program committee and the educat
education
ion data center so that selected
data can be provided to the legislature, governor, school districts, and the public.

Data Accuracy/Disclosure
(8) Reports shall contain data to the extent it is available. All reports must include
documentation of which data are not available or are estimated. Reports must not be
suppressed because of poor data accuracy or completeness
completeness.. Reports may be
accompanied with documentation to inform the reader of why some data are missing or
inaccurate or estimated.

June 2010 Page 59


Washington State K–12
K Education
Data Gap Analysis

B. List of Interviewees
Office Name Meeting (PST)

Digital Learning Karl Nelson 3/26/10 9:00 AM


Special Programs and Federal Accountability Mary Jo Johnson 3/30/10 9:00 AM
Child Nutrition George Sneller 3/30/10 1:00 PM
Highly Capable Programs and Advanced Placement Kristina Johnstone 3/31/10 10:00 AM
Title I Learning Assistance Programs, Consolidated Program Reviews Gayle Pauley 3/31/10 10:00 AM
Special Education Sandy Grummick 4/6/10 9:00 AM
Information Technology Services Terri Baker 4/6/10 1:00 PM
Information Technology Services Cynthia McCroy 4/19/10 10:00 AM
Career and Technical Education Phouang Hamilton 4/19/10 11:00 AM
Career and Technical Education Betty Klattenholff 4/19/10 11:00 AM
Learning and Teaching Support Jeff Soder 4/21/10 9:30 AM
Student Support Martin Mueller 4/28/10 10:00 AM
Professional Certification Laura Gooding 4/29/10 9:00 AM
Professional Certification Rebecca Jenkins 4/29/10 9:00 AM
Student Transportation Allan Jones 4/29/10 12:00 PM
Center for Improvement Student Learning (CISL) Rudi Bertschi 4/29/10 1:00 PM
Special Programs and Federal Accountability Bob Harmon 4/30/10 11:00 AM
Federal Programs and Accountability Anne Renschler 4/30/10 12:00 PM
School Facilities and Organization Gordon Beck 4/30/10 1:30 PM
School Facilities and Organization Angie Wirkkala 4/30/10 1:30 PM
School Facilities and Organization Brenda Hetland 4/30/10 1:30 PM
Professional Certification David Kinnunen 5/3/10 11:00 AM
Customer Support Geri Walker 5/5/10 1:00 PM
Customer Support Emily Brown 5/5/10 1:00 PM
Customer Support Micah Ellison 5/5/10 1:00 PM
Financial Services Cal Brodie 5/13/10 8:00 AM
Bilingual Migrant Education Paul McCold 5/13/10 9:00 AM
Bilingual Migrant Education Helen Malagon 5/13/10 9:00 AM
Teaching and Learning Jessica Vavrus 5/13/10 1:00 PM
Assessment and Student Information Robin Munson 5/17/10 8:00 AM
Assessment and Student Information Sheri Dunster 5/17/10 8:00 AM
OFM – Education Research and Data Center
enter Deb Came 5/25/10 1:00 PM
OFM – Education Research and Data Center
enter Michael Gass 5/25/10 1:00 PM
School and District Improvement Janell Newman 6/10/10 11:30 AM

Page 60 June 2010


Washington State K–12
K Education
Data Gap Analysis

C. Data System Gap Analysis Project Description

ABOUT THIS PROJECT


The
he Washington Legislature established the K-12 Data Governance Group within OSPI for the purpose of
assisting in the design and implementation of a K-12
12 education data improvement system for student,
financial, and educator data. The Data Governance Group’s tasks include:
• Identify critical research and policy questions;
• Identify reports and other information that should be made available on the internet;
• Create a comprehensive needs requirement document;
• Conduct a data system gap analysis;
• Focus on the financial and cost ddata that is necessary to support the new K-12
12 financial
financi models
and funding formulas; and
• Define the operating rules and governance structure for K K-12 data collections.

The K-12 Data Governance group has has, in turn, contracted with PCG Education to assist in performing a
data system gap analysis that analyzes the current status of OSPI data systems compared to the
Legislature’s intent. PCG Education will use this information in conjunction with a prioritized list of
research and policy questions that the stat
state
e data system should address to determine what data should
be included in the state data system.

Context for Interview


The identification of a data gap, between data desired and data collected, ultimately occurs at the
“element” level. While several syste
systems
ms may collect the same item, grade level for instance, a list of data
elements is the non-duplicated
duplicated list of all those collected items. The primary purpose of the interview is
to collect and validate the information necessary for identifying and documenti
documenting
ng the normative list of
data elements necessary for identifying data gaps. The types of questions you can expect include:

1) What system houses the data that your department collects?


2) What are the detail level elements that are collected in the system?
3) Are these
hese elements collected at a student level or aggregated by school or district?
4) How often is this data collected?
5) At what level is the data collected (e.g., district, school)?
6) What reports/outputs are generated from this system?
7) Are there any statistics that
at you currently pull and publish?
8) Is this system linked to any others?

June 2010 Page 61


Washington State K–12
K Education
Data Gap Analysis

D. Inventory of Existing Data Sources


Entity/Level Office / Business System Sub-System
System
Function
Student Accountability Alternative Learning Experience
School Enrollment P105 / October 1 Enrollment
Report
School Accountability P105B
School Accountability Private Ed Approval
School Accountability Private Participation in Federal
Programs
Staff Accountability Teacher Quality Data Collection
District Assessment AYP Preview
Student Assessment CAA/CIA Database (Exit / Exam
status)
Student Assessment Contrasting Groups Study
Student Assessment Promoting Academic Success
(PAS)
Student Assessment Washington Assessment
Management System
(WAMS)
Student Assessment Washington Query
School/District/State Assessment Washington State Report Card
Staff Assessment WASL Math Range Finding
Student Assessment Test Registration (OPT)
Staff Assessment Test Scoring Application
Student Bilingual LEP Migrant Student Data and
Recruitment (MSDR)
Staff Certification Electronic Certification
School/District Child Nutrition CNP2000
Student Child Nutrition Direct Certification Free Lunch
Student Child Nutrition Direct Verification
Location Child Nutrition Summer Food Site Listing
District Career and Technical Career and Technical Education
Education
School/District Career and Technical Grad and Teen Parent Spreadsheet
Education
Public School Career and Technical iGrants Annual Agricultural
Education Education Program Report

Page 62 June 2010


Washington State K–12
K Education
Data Gap Analysis

School District Career and Technical iGrants Perkins End of Year Report
Education
School Digital Learning Multi-district Online Provider
Department Application
Student Digital Learning Online Course Registration
Department System
School/District Digital Learning School / People Database
Department
School/District Digital Learning School sign-up system
Department
School/District Directory Education Data System
Staff District and School National Board for Professional
Improvement Teaching Standards (NBPTS)
Scholarship
School/District Ed Tech Tech Survey
District Financial Services Apportionment System
School? Financial Services Apportionment System School District Revenue
Projections (F-203
(F and F-
203X)
Staff Financial Services Apportionment System Personnel reporting (S-275)
(S
District Financial Services Apportionment System Student Enrollment (P-223)
(P

District Financial Services Apportionment System Budgeting (F-195)


(F
District Financial Services Apportionment System Budget Revisions (F-200)
(F
District Financial Services Apportionment System Year End Financial (F-196)
(F
District Financial Services Apportionment System County Treasurer’s Report
(F-197)
District Financial Services Grants Claim System
District Financial Services I728 Report
District Financial Services SAFS
Highly Qualified
Teachers
Academic Standards Learning And Teaching EALRS
Support
Academic Standards Learning And Teaching EALRS Management
Support
Staff Development Professional Events Manager
Meeting Development
Staff Professional Practices Statewide Fingerprint-based
Criminal Background Check
(FMS)

June 2010 Page 63


Washington State K–12
K Education
Data Gap Analysis

ESD Safe and Drug Free iGrants Title IV Safe Consort


Schools
School District Safe and Drug Free iGrants Title IV Safe District
Schools
District Safe and Drug Free Safe and Drug Free Schools and Principles of Effectiveness
Schools Communities
District Safe and Drug Free Safe and Drug Free Schools and
Schools Communities
Student Special Programs Honors Award Nomination
Student Student Information CEDARS CEDARS - Comprehensive
Education Data And
Research System
Student Student Information Core Student Record System
(CSRS)
Student Student Information Core Student Record System P210 – End of Year
(CSRS) Enrollment Status
Student Student Information Home Based Report
District Student Information Homeless Children and Youth
Data Collection Form
School Student Information, Attendance and Weapons
School Safety Centers
Student Student Services Student Learning Plan
Staff Student Bus Driver Authorization
Transportation
District Student Operations Allocation System
Transportation
District Student School Bus Information System School Bus Depreciation
Transportation
District Student School Bus Information System School Bus Inventory
Transportation
Staff / District / ESD Student Traffic Safety Education
Transportation Program Approval
School Tech Ed School Improvement Planning
Tool
District Healthy Youth Survey
Multiple Multiple iGrants 174 form packages

Page 64 June 2010

You might also like