Characterization of Intrusion Detection System Performance using Design of Experiments

Sandia National Laboratories is investigating the application of a rigorous statistical, model-based test and evaluation methodology for the evaluation of intrusion detection systems. This approach, termed Design of Experiments (DOEx), enables the experimenter to maximize the information generated per experiment while minimizing the total number of experiments required for generating viable performance data. The methodology varies multiple factors at the same time, identifies the key factors influencing a measured response (e.g., distance at first detection or alarm display time), and generates a predictive model as a function of factors that are determined to be significant. This paper defines and explains the DOEx methodology, and illustrates how it has been applied, or is being applied, to intrusion detection system evaluations at Sandia National Laboratories.