The evaluation of new criteria for polytomous DIF in the DFIT framework
Using a Monte Carlo research design, this study examined the efficacy of the item parameter replication (IPR) method (Oshima, Raju, & Nanda, 2006) for determining cutoff values for polytomous items within the differential functioning of items and tests (DFIT) framework (Raju, van der Linden, & Fleer, 1995). It was hypothesized IPR-based cutoffs would be more likely to detect differential item functioning (DIF) than previously recommended fixed cutoffs and would have false positive rates close to the nominal significance level. Results supported the efficacy of the IPR method. Further, the accuracy of DIF detection was compared between the IPR method and likelihood ratio (LR) test (Thissen, Steinberg, & Wainer, 1988). Across a number of conditions and items studied these methods demonstrated comparable power and Type 1 error rates. Differences between methods were observed in conditions with non-uniform DIF, where the LR test demonstrated more power to detect non-uniform DIF of small magnitudes. Sample size, focal group ability distribution, proportion of test-wide DIF, and direction of DIF had minimal effect on DIF detection across methods.
0632: Psychological tests