|
SUMMARY
The Office of the Auditor General has completed an evaluation
of the Family Builders Pilot Program. This evaluation was conducted pursuant to
the provisions of Laws 1997, Ch. 223, §3, and provides information regarding
the Pilot Program’s implementation and effectiveness in achieving its goals.
The goal of the Family Builders Pilot Program is to enhance
parents’ ability to create safe, stable, and nurturing home environments that
promote safety of all family members and healthy child development. It is
accomplished by responding to potential- and low-risk reports of child abuse and
neglect using a network of community-based providers. Specifically, the Child
Protective Services (CPS) Division of the Department of Economic Security refers
potential- and low-risk reports of child abuse and neglect to community-based
Family Builders providers. Providers offer services aimed at reducing the
problems contributing to the potential for abuse or neglect, such as parenting
skills training, counseling, housing search and relocation, transportation,
childcare, and emergency services. Participation in the program is voluntary.
Most families participate for four to six months.
The Arizona Department of Economic Security is responsible
for administering the Family Builders Pilot Program. To do this, DES contracts
with eight community-based social service agencies in Maricopa and Pima Counties
to provide services to families. Providers are paid a capitated rate for
completion at each of three service delivery stages: referral, assessment, and
service plan.
A total of 8,335 families were referred to Family Builders
providers from January 1, 1998 through July 31, 1999. Of these referrals, 5,578
families declined services; 2,757 families received an assessment; and 2,326
families continued on to complete a service plan and receive services.
Family Builders Unable to Show
Impact on Child Safety
at This Time
(See pages 13 through 17)
At this time, the Family Builders Pilot Program does not show
an effect on increasing children’s safety in their homes. As shown in Table 1,
there was no meaningful difference between the proportion of Pilot Program
families who received CPS reports alleging child abuse or neglect subsequent to
their involvement in the program and the proportion of comparison group families
who later received CPS reports. One comparison group is comprised of families
who were referred to, but did not receive, Family Builders services, for reasons
such as they refused services, did not need services, could not be located, or
were referred back to CPS. The other group consists of families who received DES
investigations because the program was not available in the county where they
live. In addition,
there was no difference in the proportion of families who
received subsequent reports where CPS substantiated that instances of abuse or
neglect occurred. Furthermore, incomplete or inaccurate information about
families served by the program limits the ability to explain the underlying
reasons for these results. However, families who accepted services from the
Pilot Program experienced a slight but statistically significant decrease in the
risk for child abuse and neglect, as measured by one instrument, the Family Risk
Scale, completed by the Family Builders specialist. Another scale measuring
families’ perception of their risk for abuse or neglect could not be analyzed
because the providers did not collect enough self-assessments from Pilot Program
families.
DES Needs to Examine Costs
and Number of
Services Delivered
(See pages 19 through 27)
DES currently lacks the necessary information to determine if
providers are delivering adequate services and are being paid at appropriate
rates. There are indications that the rates paid to many providers may be higher
than the costs of the services provided. A review of 80 randomly selected cases
found that the capitated payment rate exceeds providers’ actual costs by an
average of $1,700 per family served. There are also indications that clients may
not be receiving many of the services identified as necessary to meet their
needs. In the same review of case files, evaluators found that providers did not
meet 42 percent of the families’ service needs. In almost half of those cases,
services were not provided because the family failed to follow through with the
referral.
Incomplete data limits DES’ ability to make informed
judgments about whether rates are appropriate or services are actually provided.
When checked against the records in the DES database, 93 percent of the cases
contained inaccurate information regarding the type, number, or cost of services
delivered. As a result, DES needs to improve the quality and completeness of
program information and use it to determine the appropriateness of funding and
service delivery rates.
DES Should Improve Program
Oversight to Ensure All
Requirements Are Met
(See pages 29 through 33)
DES is not performing adequate oversight to ensure that
providers meet several requirements of the Pilot Program. In many cases, basic
demographic information about participants is inaccurate or unknown. In 68 out
of 80 cases reviewed, providers failed to collect or accurately report
demographic information about families being served by the program. Omissions
and errors occur, in part, because until June 1999 providers were not required
to update demographic information after the completion of assessments, when much
of the demographic information becomes available. Further, until February 2000,
DES was not aware that it was not receiving updated information contained in the
provider databases. In other cases, providers have failed to collect or document
required demographic information in either the client file or the program
database. In those cases, the lack of information appears to be an oversight on
the part of the worker completing the assessment. DES has addressed these issues
through training and modification of required forms. These efforts should be
increased to ensure important program data is collected.
Providers have also not obtained the Brief Family Assessment
Scale from most of the families in the program. Specifically, providers obtained
only 49 intake and exit assessment forms from 692 families served by the
program. As the assessments are useful in both case planning and program
evaluation, DES needs to better monitor provider collection and use of the risk
assessment and provide additional training as needed.
Finally, Local Advisory Boards, which are required under the
statute, are not fulfilling all of their statutory responsibilities. The meeting
minutes from all Local Advisory Boards indicated that service continuity has not
been consistently addressed, that some boards need to improve their oversight of
program operations, and that Board representation from both families and public
agencies in the community is difficult to attract and retain.
Statutory Evaluation
Components
(See pages 35 through 46)
Pursuant to Laws 1997, Ch. 223, §3, the Office of the
Auditor General is required to measure the program’s effect in reducing the
number of reports to Child Protective Services, the number of investigations
conducted by Child Protective Services, and the number of children placed
outside of the home. Although the number of reports CPS received has declined
since the Pilot Program's implementation, the effect cannot be attributed to the
Pilot Program since the report rate has declined at the same or greater level in
counties that did not have the Family Builders Program.
The number of investigations performed by CPS has also
declined. There are at least two explanations for the decrease in CPS
investigations. First, all reports referred to the Family Builders Program no
longer require a CPS investigation. Second, the overall decline in reports CPS
received results in fewer investigations.
Further, the number of out-of-home placements statewide has
increased over the last three fiscal years with the major portion of the
increase occurring within the first two years. However, the percentage of Pilot
Program families who later had a child removed from the home did not differ from
that of families who were referred to, but did not receive, program services.
Read full report in Acrobat PDF format
|
