|
|
SUMMARY
The Office of the Auditor
General has conducted a performance audit of the
Technology Assisted Project-Based Instruction
(TAPBI) Program, pursuant to Laws 2005, Ch. 323,
§2. TAPBI is Arizona’s primary approach to
providing an Internet-based alternative to
learning in traditional brick-and-mortar
schools. This audit provides an overview of the
Program and examines three aspects of its
operations: whether TAPBI schools are
appropriately applying state enrollment and
funding requirements, how their costs compare to
traditional brick-and-mortar schools, and what
efforts they are taking to ensure student
achievement.
In 1998, the Arizona Legislature created the TAPBI Program to
“improve pupil achievement and extend academic options beyond the four walls of
the traditional classroom.” Still in a pilot stage, the TAPBI Program has grown
from 4 schools to 14 as of fiscal year 2007, including 7 charter schools and 7
school districts. Further, program enrollment grew from about 500 students to
more than 15,000 in fiscal year 2006. Most TAPBI schools serve many different
types of students including gifted, at-risk, and homebound students, but a few
target specific student populations, such as adult-aged high school or at-risk
students. Four TAPBI schools provide distance learning to elementary students in
kindergarten through grade 8, while the remaining 10 schools serve only high
school students. Distance-learning programs are operated in all 50 states, and
Arizona is 1 of 26 states that provide the program through individual schools
rather than a state-wide program through the Department of Education.
TAPBI schools typically use Internet-based applications,
known as learning management systems, to create and deliver learning content,
such as online reading materials, interactive exercises, discussion forums,
video clips, and quizzes. Students may also spend time on activities such as
reading textbooks, completing homework assignments, or working with hands-on
projects. Several TAPBI programs allow students to enroll in and start TAPBI
courses at any time of the year, and they are generally self-paced as students
are allowed to spend more or less time on individual lessons than may occur in a
traditional classroom.
The Superintendent of Public Instruction, the State Board of
Education, and the State Board for Charter Schools provide oversight for the
TAPBI Program. The schools are required to file annual reports including
descriptions of educational services provided, the Program’s operational and
administrative efficiency and cost-effectiveness, measures of academic
achievement, and results of student and parent satisfaction surveys. The Joint
Legislative Budget Committee, along with the two state boards, issues a
compilation of the TAPBI schools’ individual annual reports each fiscal year.
Further, the state boards are statutorily required to review the effectiveness
of each TAPBI school once every 5 years and determine whether to renew the
school’s participation in the Program.
TAPBI schools are provided state and local funding using the
same per-pupil method as for other public schools, which is based on Average
Daily Membership (ADM). In fiscal year 2006, the 4,475 ADM for the TAPBI Program
generated an estimated $23.8 million in funding.
Fiscal year 2006 TAPBI ADM
overfunded by about $6.4 million
(see pages 11 through 18)
TAPBI program errors and noncompliance increased the
cost of public education by an estimated $6.4 million. The Program is funded on
the same ADM method used for brick-and-mortar schools, which may not be the best
basis for funding online education. For online programs such as the TAPBI
schools, the ADM method is more susceptible to error. Specifically, while ADM is
determined by teacher attendance records in traditional schools, for the TAPBI
schools it is determined by students’ self-reported hours of time spent on
coursework. Current literature does not identify an ideal funding method for
online learning programs. Although Arizona is one of 21 states using a per-pupil
funding method for distance-learning programs, some of these states have
additional requirements, such as course completion, to obtain funding. Also, a
full-time equivalent method may address some of the ADM issues as it would be
based on the number of courses taken rather than instruction hours reported by
the students.
Further, trying to apply the ADM method to online programs
contributes to program overfunding. The Arizona Department of Education’s (ADE)
errors when applying statutory ADM limits were the primary reason for TAPBI
overfunding. First, statute does not provide for TAPBI students to be funded at
more than a total of 1.0 ADM. However, ADE does not limit students who are
concurrently enrolled in brick-and-mortar and TAPBI schools to 1.0 ADM. The
Student Accountability Information System (SAIS), ADE’s computer-based program
that calculates ADM, currently does not have the capability to adjust for
concurrent TAPBI enrollment. Therefore, the allocation process is done outside
the SAIS system using a variety of formulas in a spreadsheet program. However,
the spreadsheet process does not include adjusting the brick-and-mortar school’s
funding for concurrently enrolled TAPBI students. For example, a student who
attends a brick-and-mortar school full-time and a TAPBI school half-time could
be funded as 1.33 ADM rather than being split proportionately so that funding
remains at 1.0 ADM.
Another source of overfunding occurs when students enroll in
a TAPBI program after the cutoff date for traditional school funding or attend
“summer school.” Specifically, students enrolling in a brick-and-mortar school
after the 100th day of the school year would not generate any ADM funding.
However, because TAPBI schools operate year-round, students enrolling in TAPBI
schools generate funding regardless of when they enroll. Also, brick-and-mortar
schools do not receive state funding for instruction time provided outside the
normal school year, so they typically charge students tuition for attending
summer school. However, ADE does not consider TAPBI summer enrollments to be
limited to 1.0 ADM as with concurrent enrollments, and therefore, includes this
ADM for TAPBI funding purposes.
As a result of ADE not properly limiting TAPBI ADM, in fiscal
year 2006, about 6,800 of the 10,600 TAPBI students enrolled in multiple schools
were overfunded by approximately $6.4 million.
The TAPBI schools’ failure to adhere to statutory enrollment
restrictions further contributed to overfunding. Specifically, TAPBI schools
must ensure that at least 80 percent of their new enrollments are students who
attended public schools during the previous school year, that enrolling
kindergartners already have a sibling enrolled in a TAPBI school, and that their
enrollment growth does not exceed 100 percent in a fiscal year. However, in
fiscal year 2006, 6 TAPBI schools exceeded one of these statutory limits,
increasing the TAPBI program cost by about $88,000.
TAPBI schools’ operations cost less,
but further savings may exist
(see pages 19 through 26)
In fiscal year 2006, TAPBI schools spent an average of
approximately $5,500 per pupil, which is about $1,200 less than the $6,750
per-pupil state average. TAPBI schools achieve lower per-pupil costs because
most students take classes at home. As a result, the schools have little, if
any, transportation or food service costs and significantly lower plant
operation and maintenance costs. Further, TAPBI schools reported that fewer than
6 percent of their students had special needs, which significantly lowered their
special education costs.
However, costs varied significantly among the 14 TAPBI
schools because of differences in their operations. Specifically, TAPBI charter
schools’ per-pupil costs averaged $6,140, which is only about $600 lower than
the state per-pupil average and much higher than the $2,900 per-pupil average
for TAPBI school districts. These higher costs occurred primarily in
administration costs, which were influenced by higher administrative staff
salary and benefit costs and staff entertainment costs, such as restaurant
purchases and gift cards.
Software licensing agreements also tended to increase TAPBI
schools’ per-pupil costs. For example, in fiscal year 2006, one TAPBI charter
school paid nearly $2.5 million, or $3,535 per pupil, as an ongoing cost for its
learning management system and related consulting services. Further, a TAPBI
school district’s $8,138 per-pupil cost was also primarily influenced by its
purchase of a learning management system, which was a one-time cost of about
$1,040 per pupil, and the ongoing cost of system support at about $520 per
pupil. In contrast, another TAPBI school district developed its own learning
management system in 1998, and in fiscal year 2006, had the second lowest
per-pupil cost of all the TAPBI schools. Two other TAPBI school districts use
this same system through intergovernmental agreements, which lowered their
learning management system-related costs. One of these two districts had the
lowest overall per-pupil cost of all the TAPBI schools.
Although required by statute, the TAPBI schools have not
accurately identified their costs or properly reported their cost-effectiveness.
The TAPBI schools did not always properly allocate costs between their TAPBI and
other operations, making it difficult to evaluate program cost-effectiveness.
Further, the TAPBI schools did not consistently categorize their reported costs,
making it difficult to compare costs among the different schools.
Student achievement measures and
practices can be improved
(pages 27 through 37)
The TAPBI Program’s effect on student achievement is unclear.
Currently, the state boards measure student achievement by comparing TAPBI
students’ standardized test scores to state-wide students’ results. However,
many TAPBI students attend multiple schools or have short tenure in the Program.
Also, high school students do not take the AIMS test each year. As a result,
this method of determining the TAPBI schools’ effect on student achievement is
not reliable. National studies are also inconclusive on how online learning is
affecting student achievement.
TAPBI schools can take steps to improve instruction and
student support practices and potentially improve student achievement. First,
TAPBI schools should ensure that students are receiving the required minimum
number of instruction hours for their TAPBI courses. Auditors found that a
sample of TAPBI high school students received an average of 48 percent fewer
instruction hours than required, while kindergarten through 8th-grade students
received an average of 9 percent fewer hours than required. Further, finding
ways to ensure academic integrity is even more important in an online learning
environment. All but 2 of the TAPBI schools had formal student-teacher
communication policies or measured the timeliness or frequency of communication.
However, only 6 of the 14 schools required students to take exams in a proctored
environment and pass the exams to receive credit for their classes. Further,
some TAPBI schools did not provide teacher training programs that were specific
to working with students in an online environment.
Appendix (see pages a-1 through a-16)
The Appendix provides alphabetically organized one-page
information sheets on the individual TAPBI schools. Each page summarizes the
school’s enrollment and costs and provides other descriptive information.
Read full report in Acrobat PDF format
|
|
