|
|
SUMMARY
The Office of the Auditor General has conducted a performance audit of the substance abuse treatment programs provided by the Department of Health Services (Department), Division of Behavioral Health Services (Division), pursuant to an October 5, 2006, resolution of the Joint Legislative Audit Committee. This is the second audit in a series of three reports on the Department and was conducted as part of the sunset review process prescribed in Arizona Revised Statutes (A.R.S.) §41-2951 et seq. This audit focuses on substance abuse treatment outcomes and system oversight. The first audit focused on the Division of Licensing Services, and the final report will be an analysis of the 12 statutory sunset factors.
According to its Annual Report on Substance Abuse Treatment Programs, the Division spent more than $121 million for substance abuse services in fiscal year 2008. Program participants, whom the Division refers to as “consumers,” numbered more than 63,000 adults and children. Most were enrolled in the Arizona Health Care Cost Containment System, or AHCCCS, the State’s Medicaid program. They received alcohol- and drug-related services ranging from counseling and skills training to crisis intervention and detoxification in hospitals or other inpatient facilities. The Division provides these services through contracts with four regional behavioral health authorities, or RBHAs, and three tribal regional behavioral health authorities, or TRBHAs, which contract with a network of more than 100 substance abuse treatment service providers throughout the State.
This performance audit focused on the program’s outcomes—that is, the extent to which services reduced dependency on alcohol and drugs—and on the Division’s oversight of behavioral health authorities and providers.
Division should focus on strategies that improve
outcomes (see pages 9 through 29)
Although substance abuse is difficult to treat, the Division can take steps to improve outcomes for individuals who participate in substance abuse treatment. Auditors analyzed 3 years of data related to four measures commonly used to evaluate program effectiveness—extent of continuing alcohol or drug use, employment, criminal activity, and homelessness. The analysis showed that outcomes related to continued alcohol and drug use were associated with the following two factors:
- Deciding to abstain from using alcohol or drugs before treatment started— More than half of all consumers reported that they were abstinent when their treatment started. Within this group, more than 93 percent reported that they were still abstinent when they left the program. By contrast, most of those who reported using alcohol or drugs when they began treatment were still using these substances at about the same level when they left. About one person in every four who began treatment while still using alcohol or drugs reported diminishing his/her use of alcohol or drugs or stopping it altogether by the time he/she left treatment.
- Completing treatment—Overall, 58 percent of consumers did not complete their treatment. Providers lost contact with many of them, while others refused treatment or left for other reasons, but rates of continued use of alcohol or drugs varied substantially between those who completed the treatment and those who did not. For example, among consumers who reported using alcohol or drugs when they began treatment, 27 percent of those who completed their treatment reported abstinence when they left. By contrast, among consumers who reported using alcohol or drugs when they began treatment, only 17.6 percent of those consumers who left before completing their treatment reported that their use had diminished.
The analysis showed little change across the three remaining performance measures—lack of recent arrests, employment, and stable housing. For example, 21 percent of consumers reported recent arrests upon entering treatment, and 18 percent reported new arrests at the time of their update or disenrollment. Similarly, 38 percent said they were employed when they entered treatment, while 41 percent reported being employed at their annual update or disenrollment. Finally, slightly more than 7 percent were homeless upon entering treatment, and slightly less than 7 percent were homeless at their annual update or disenrollment.
Substance abuse is difficult to treat, and auditors’ more detailed case studies of a limited number of consumers showed that the reasons for success or failure are complicated and varied. Nonetheless, research and best practices indicate the best opportunities for increasing success rest in three main strategies:
- Focusing on treatment retention—Research corroborates what auditors’ analysis showed: consumers who remain longer in treatment experience better outcomes. The Division can take several steps to increase consumer retention, including establishing performance goals, monitoring completion rates, and using incentives and other case management techniques. Other states that auditors reviewed have taken such steps, and the Division may be able to adopt some of these approaches.
- Ensuring that consumers have access to a full range of services that can potentially be used to address their particular needs—This strategy, called continuum of care, involves incorporating appropriate types of treatment over time and placing the consumer in more or less intensive treatment as needed. Auditors’ case studies showed that while some consumers showed good outcomes and received appropriate services, others did not necessarily receive the services and therapies that might improve the chances of good treatment outcomes. The Division can take several steps to ensure continuum of care, including collecting and monitoring data relevant to assessment, better defining case management, and working with RBHAs to improve the continuum of care when weaknesses are identified.
- Following practices that have been shown to carry the greatest chance of success—These evidence-based practices have been validated by observation or experience as improving treatment success. Examples include motivational interviews, which is a counseling style designed to help consumers recognize and accept the need for continued care. Although the Division requires the RBHAs to use evidence-based practices, RBHAs are not necessarily doing so, and the Division is not ensuring compliance. According to a 2008 federal grant review that focused on programs for children and adolescents, the Division had done a good job of establishing evidence-based practices in some areas but had not identified such practices across the continuum of care and could do more to ensure sustainability of the emphasis on evidence-based practices. The Division reached similar conclusions in another study of intensive outpatient programs for youth. Steps the Division can take to place greater emphasis on evidence-based practices include encouraging RBHAs to offer a wider variety of programs, monitoring compliance with its contractual requirements to use evidence-based practices, and expanding its work with the RBHAs to ensure that providers have the guidance needed to implement specific evidence-based practices.
Division should improve oversight of substance abuse
programs (see pages 31 through 42)
The Division should take steps to improve its oversight of the substance abuse programs administered by RBHAs. These steps take two main forms:
- Increasing the use of information about treatment outcomes—Although the Division collects outcome information to complete certain reports, auditors found that oversight efforts focused almost entirely on process-related information, such as the timeliness of services or coordination with a consumer’s primary care physician as required by the Division’s contract with AHCCCS. As a result, the Division is largely unable to determine if its substance abuse treatment programs are achieving positive results or if its resources are being used effectively. Additionally, because the Division does not compare substance abuse outcome measures across RBHAs or providers, it cannot assess which providers’ treatment services are resulting in improved client outcomes or identify underperforming providers. Needed actions include continuing its efforts to streamline uniform outcome data collection, establishing relevant performance goals in contracts with the RBHAs, and encouraging the RBHAs to consider ways to reward providers who meet standards and penalize those who do not.
- Expanding utilization reviews to focus more on service costs, consumer assessments, and case management—Although its oversight efforts contain many elements that could potentially help manage costs, the Division could implement several actions that could improve its ability to do so. Greater emphasis on cost appears warranted. Auditors’ review of division data from fiscal years 2006 to 2008, for example, identified 14 substance abuse consumers with service costs over $100,000. One incurred $82,000 in medical detoxification costs, during which time he continued to drink and require detoxification three or four times a month. The Division was not aware of these cases until auditors brought them to officials’ attention. Actions needed include (1) regularly reviewing high- and low-cost substance abuse treatment cases, (2) collecting data to identify consumers who may be overutilizing or underutilizing certain types of services, which could indicate a lack of alternative forms of treatment or a need for other changes to improve treatment, (3) comparing variations in the use of types of treatment at each RBHA to see if the use of such services positively affects consumer treatment outcomes and adjusting treatment accordingly, and (4) determining how to best use assessment and case management to contain costs while maintaining quality of care. Further, to improve oversight, the Division should continue its efforts to fill vacant positions in its data systems and analysis and quality management functions, and should perform follow-up work to ensure that the restructuring it initiated in April 2009 has provided management with the information to do so.
Read full report in Acrobat PDF format
|
|
