IBM Journal of Research and Development
IBM Skip to main content
  Home     Products & services     Support & downloads     My account  

  Select a country  
Journals Home  
  Systems Journal  
Journal of Research
and Development
    Current Issue  
    Recent Issues  
    Papers in Progress  
    Recent publications  
    Author's Guide  
  Contact Us  
  Related links:  
     IBM Research  

IBM Journal of Research and Development  
Volume 19, Number 3, Page 252 (1975)
System Performance Evaluation
  Full article: arrowPDF   arrowCopyright info


Design of Experiments in Simulator Validation

by M. Schatzoff, C. C. Tillman
A common problem encountered in computer system simulation is that of validating that the simulator can produce, with a reasonable degree of accuracy, the same information that can be obtained from the modeled system. This is basically a statistical problem because there are usually limitations with respect to the number of controlled tests that can be carried out, and assessment of the fidelity of the model is a function of the signal to noise ratio. That is, the magnitude of error which can be tolerated depends upon the size of the effect to be predicted. In this paper we describe, by example, how techniques of statistical design and analysis of experiments have been used to validate the modeling of the dispatching algorithm of a time sharing system. The examples are based on a detailed, trace-driven simulator of CP-67. They show that identical factorial experiments involving parameters of this algorithm, when carried out on both the simulator and on the actual system, produced statistically comparable effects.
Related Subjects: Analytical models; Operating systems; Simulation; Storage hierarchies