Abstract
Intrusion Detection Systems have been observed to trigger an abundance of false positives, that is alerts not reporting security problems. Assuming that in real installations most of the alerts are reviewed by human security analysts in a timely manner, it is possible to use supervised machine learning techniques for automated alert classification to classify alerts into true and false positives. This paper explores the requirements for such an alert classification system and shows that, being a difficult and challenging machine learning problem, it is particularly suited for the application of abstaining classifiers, i.e., classifiers that can refrain from classification in some cases.
We show that by applying our method for finding optimal, abstaining classifiers based on the ROC analysis, one can significantly reduce the rates of false positives and the false negatives as well as overall misclassification cost, making this method particularly viable for this application domain. Finally, we validate our method on one real-world proprietary dataset and one synthetic, publicly available dataset.