Development of sequential procedures for comparing a simulation output to a limit standard
The analysis of model output is arguably the most important and technically demanding aspect of discrete event simulation. Usually, this analysis requires comparing the output to a requirement or benchmark, otherwise known as a standard. One type of standard is a maximum and/or minimum value for an output, which prescribes a range of desired or acceptable system performance. This type of standard is known as a limit standard. Limit standards utilize proportions. Current output analysis methods defined for standard comparison do not distinguish between limit standards and other types of standards (typically standards using the mean of the observations). The current methods utilize confidence interval methodology. Confidence intervals tend to decrease as the sample size increases and require normality. Further, empirical and statistical analyses show the distribution of sample proportions is decidedly skewed as the distribution approaches 0 or 1. As a result, the application of current analysis methods to limit standards can result in incorrect conclusions. This research looked at the general characteristics of a limit standard, and developed Frequentist and Bayesian methodologies for use in the comparison analysis of these types of standards to model output. The approaches used in this research are derived from acceptance sampling, sequential analysis and Bayesian statistics. This research pioneers the adaptation of acceptance sampling methodology to verification of probabilistic limit standards using stochastic (Monte Carlo, DES) simulation and introduces a new elicitation method for determining the parameters of a Beta-distributed initial prior probability density function for use in the Bayesian analysis of limit standards. These methodologies were applied to the output of an M/M/1 model, and results documented. The results from this research will provide simulation analysts several methodologies for accurately comparing a simulation output to a limit standard. Also, future areas of research in limit standard output analysis were identified.
0790: Systems design
0796: Operations research