Development of the Trauma Informed System Change Instrument: Evaluation of factorial validity and implications for use
This paper outlines the process of developing and evaluating an instrument measuring the extent to which a complex community system has changed as a result of a community initiative, and for purposes of this research, doing this within the content area of developing local trauma informed child welfare systems in specific communities in Michigan. The instrument was designed for the Southwest Michigan Children's Trauma Assessment Center's (CTAC) SAMHSA-funded initiative to bring a trauma informed perspective to professionals working with children in child welfare. Because there is not a standard set to define what constitutes trauma informed treatment of children in the child welfare system, this instrument serves a dual purpose - to address a gap in evaluation methodology and practice as well as to guide interventions seeking to change child welfare systems in becoming trauma informed. An instrument was developed with the input of experts in trauma informed system change. Two distinct parts of the study were carried out: evaluation of the instrument, and an impact study for the first year of the initiative, utilizing the pilot version of the instrument. The sample used for the study was obtained from the pool of professionals attending community-based trainings provided by CTAC on trauma informed system change. Following the two tracts of the study, two distinct tracts of analyses were conducted. For the validity study, the two parts of the instrument — Community Characteristics and Individual Characteristics — were analyzed separately using the Maximum Likelihood method and confirming results with Weighted Least Squares Means and Variances. A three factor model was fit for Community Characteristics and a two factor model for Individual Characteristics, with goodness of fit and parsimony indices within the acceptable range for both sections of the instrument and reliability using Cohen's alpha adequate to good. The impact study was conducted by comparing means utilizing the valid factor structure and triangulating results with qualitative data from the project evaluation. Although adequate factorial validity was obtained for the instrument, specific items on the instrument that were problematic in fitting the model were identified, and suggestions for revising the instrument for improved functionality are offered, as are other potential uses of the instrument.
0443: Educational evaluation
0452: Social work