Agreement between self-report and observational measurement of the use of evidence-based practice in community early intervention programs for children with ASD

Aims: The goals of this study involved (1) comparing provider report of strategy use to observed strategy use and (2) examining the relationship between community intervention and evidence-based practices (EBP) for ASD. Methods: Eighteen providers serving preschoolers with ASD completed a telephone survey of techniques used in their programs and how they adapt these techniques in practice. Test/retest reliability was 90%. Providers were then videotaped in their classrooms for two hours per day across two days. Trained observers scored videos for activity type and intervention techniques used. Descriptive analyses examined provider use of EBPs and were used to determine agreement between self-report and observational data. Results: Ninety-one percent of providers reported using at least one EBP in their program; however providers typically combined strategies (up to seven methods). Only 9% of providers chose techniques based on research evidence. All providers had concerns about limited training opportunities Agreement between self-report and observational data was poor. For example, 95% of providers reported using sensory integration strategies; however these methods only appeared in 5% of observations. Ninety-one percent of providers indicated using child choice as a teaching strategy, however this was observed in only 2% of observations. Discussion: These data highlight the importance of observational data in determining EBP use in community programs. The extensive use of eclectic models may be impacting the effectiveness of intervention by diluting the depth and structure of specific strategy use. Innovative methods of translation of EBP to community