|
|
SUMMARY
The Office of the Auditor General has conducted a baseline
study of the current state of English Language Learner (ELL) programs and
available data. The report describes the ELL programs and participants in a
sample of 18 school districts and charter schools, chosen to provide a
cross-section of districts and charter schools across the State. This report
also provides a point of comparison for future biennial audits, which are
mandated by A.R.S. §15-756.12 to report on the effectiveness of the State’s ELL
programs in achieving English proficiency. In addition, this report discusses
the reliability of data necessary to manage and audit the programs and
recommends steps to improve future data before the biennial audits begin. To
place Arizona’s ELL programs in a national context, recent ELL data and trends
from across the nation are presented.
ELL students and programs in Arizona
and in the U.S. (see pages 1 through 9)
Nation-wide, ELL students have composed 10 percent of the
total student enrollment for kindergarten through 12th grade. In Arizona—one of
five states with the highest concentration of ELL students—the 138,449 ELL
students composed approximately 14 percent of the State’s total enrollment in
fiscal year 2007. Although ELL students speak more than 400 different languages,
Spanish is spoken by 80 percent of all ELL students nationally and 81.2 percent
in Arizona. In Arizona, as in the nation as a whole, most of the ELL students
are in elementary grades. The percentage of Arizona’s ELL students diminishes
consistently from kindergarten through 12th grade. ELL students constitute over
15 percent of Arizona kindergartners and less than 2 percent of its 12th
graders.
Arizona’s approach to ELL standards and assessment differs
from many other states. Since 2004, Arizona’s districts and charter schools have
relied on state-defined proficiency standards. In contrast, many districts in
other states also use standards at the district, school, and even classroom
level. For program entry, exit, and monitoring, Arizona’s districts are required
to use the Arizona English Language Learner Assessment (AZELLA), a composite of
speaking, listening, reading, and writing scores. The assessments used in other
states are not as comprehensive. Ninety percent of districts surveyed in other
states based entry decisions solely on oral proficiency, and nearly 82 percent
used oral proficiency as the sole basis for exiting the program. Under the No
Child Left Behind Act of 2001, the federal government requires that all public
school districts annually assess all students with limited English proficiency,
but neither defines proficiency nor specifies which tests that states must use
to measure it.
Arizona’s most recent ELL legislation, HB2064, was an
outgrowth of more than a decade of litigation and legislation, and further
specifies the State’s ELL program structure. The bill requires the ELL Task
Force to develop state-wide program models, ADE to monitor and report on the
program’s success, and the Auditor General to audit the models’ effectiveness on
performance-based outcomes and English proficiency. This study focused on
current ELL programs operating at a sample of Arizona school districts and
charter schools and the data these schools reported to ADE. The sample of 13
districts and 5 charter schools was designed to reflect different types of
programs based on district or charter school size, location, number, and
proportion of ELL population and percentage of ELL students reclassified as
English proficient.
Characteristics of sample ELL programs
and participants (see pages 11 through 19)
ELL programs in the 18 sample districts and charter schools
differ greatly from each other and from the structure they will have to employ
in the future under the ELL Task Force’s new Structured English Immersion (SEI)
models. Adopted in September 2007, the new models specify state-wide ELL policy
regarding instructional approach and classroom practices. Implementation of
these SEI models is expected beginning in fiscal year 2009. In fiscal years 2007
and 2008, the sample districts and charter schools operated ELL programs that
varied by grade and proficiency levels and across individual schools. Although
programs at the sample districts and charter schools were aligned with the new
models’ requirements regarding assessment and English-only classroom materials,
their instructional approaches were significantly different from the models’
future SEI requirement regarding English language development (ELD).
The Task Force’s models require all new ELL students to
receive daily 4-hour ELD classes. In fiscal year 2008, only 3 of the 18 sample
districts and charter schools were operating classes that met this upcoming
requirement, but only for a portion of their ELL students. In 2007, the most
recent year for which comprehensive data was available, more than half of all
ELL students in the sample districts and charter schools attended programs that
mainstreamed all ELL students, providing no hours of ELD instruction in an SEI
setting. Forty-two percent were in programs that provided up to 2 hours of daily
ELD instruction. The remaining 6 percent provided more than 2 and up to 4 hours
of daily ELD. Across sample districts and charters, program officials identified
the schedule and resource challenges to meeting the 4-hour daily ELD
requirement. For example, program officials at three rural districts stated that
they currently have difficulty filling regular teaching slots, and they do not
know where they would find four to six more teachers with the qualifications to
teach SEI classes.
The new models are designed to bring ELL students to full
proficiency in 1 year. For the sampled districts and charter schools, about 7
percent of the approximately 8,700 ELL students became fully proficient in
fiscal year 2007, and most of them had been in the program for at least 2 years.
Between fiscal years 2006 and 2007, nearly two-thirds of the students remained
at the same proficiency level or regressed, while about one-third moved to a
higher proficiency level. ELL students who were at the lowest proficiency levels
tended to make the most progress.
ELL data and data systems need
attention (see pages 21 through 28)
A.R.S. §15-756.10 requires ADE to collect and maintain ELL
data related to program funding and monitoring. A.R.S. §15-756.12 requires the
Auditor General to review compliance with program requirements in district
performance audits and biennially report on the overall effectiveness of the
State’s ELL program. Based on reviews of the data submitted by sample districts
and charter schools to ADE and of source records, auditors found several errors
affecting both the accuracy of funding ADE provides to districts and charter
schools and the measurement of ELL students’ progress.
To adequately fund, manage, and audit the effectiveness of
Arizona’s ELL programs, three main types of information are needed: the number
of ELL students, achievement outcomes, and time in the program. First, since
state funding is awarded on a per-student basis, ADE needs an accurate
determination of the number of ELL students in each district or charter school.
However, auditors’ review of sample files found that basic student
identification information was not necessarily present, or if present, was not
necessarily correct. Further, 2 percent of the sample ELL students who were
funded had no assessment scores or had an assessment score indicating that,
prior to entering the program, the ELL student was already proficient. Second,
although accurate proficiency information is needed to monitor and evaluate
districts’ and charter schools’ success, for 6.5 percent of the students, the
classification of overall proficiency levels did not match their corresponding
AZELLA scores. Additionally, 4 percent who were reclassified as proficient
actually had scores that were below proficient or lacked valid assessments.
Third, since funding is limited to 2 years, ADE needs to know how long each ELL
student has been participating in the program. However, 6 of the 18 sample
districts had ELL students with questionable entry and exit dates. To prevent
such errors, ADE needs to work with district and charter schools to develop
better ELL data submission and review processes.
ADE does not have adequate procedures in place to identify
these errors and either correct them or minimize their impact. When data
submitted by districts and charter schools is inaccurate or incomplete, ADE’s
own internal processes should be able to identify the problems. Auditors found
that ADE’s integrity checks and controls for doing so are insufficient.
Consequently, the errors and limitations in the data are carried into the
system. For example, based on schools’ erroneous data submissions, ADE included
English-proficient students in program-funding counts. Further, an agency’s data
system should include process controls to ensure that no data is added, lost, or
altered during processing. In August 2006, the Auditor General’s performance
audit of ADE’s information management recommended that ADE add such controls.
The presence of such controls could help users and ADE prevent potential
problems with student achievement and funding data, such as the detection and
correction of a massive state-wide recalculation and override of ELL counts that
occurred in July 2007. In this case, because of a processing error, over 20,000
ELL students were excluded from end-of-year funding counts. ADE eventually
corrected the $8 million error, but analysts struggled to identify its cause.
Further, ADE did not investigate prior years’ funded counts for similar
processing exclusions.
Also, in order to better monitor implementation of the new
SEI models and assess program success, ADE could collect additional data that is
currently available from districts. Information about the number and
qualifications of ELL teachers is not currently collected and maintained by ADE,
although ADE requires districts to maintain ELL teacher certification and
endorsement documentation for review during monitoring visits. Information about
the number of hours of ELD instruction could also assist with monitoring
efforts. Sample districts and charter schools provided auditors with information
on students’ hours of ELD instruction based on their proficiency and grade
levels.
Read full report in Acrobat PDF format
|
|
