Military and defense organizations rely upon the security of data stored in, and communicated through, their cyber infrastructure to fulfill their mission objectives. It is essential to identify threats to the cyber infrastructure in a timely manner, so that mission risks can be recognized and mitigated. Centralized event logging and correlation is a proven method for identifying threats to cyber resources. However, centralized event logging is inflexible and does not scale well, because it consumes excessive network bandwidth and imposes significant storage and processing requirements on the central event log server. In this paper, we present a flexible, distributed event correlation system designed to overcome these limitations by distributing the event correlation workload across the network of event-producing systems. To demonstrate the utility of the methodology, we model and simulate centralized, decentralized, and hybrid log analysis environments over three accountability levels and compare their performance in terms of detection capability, network bandwidth utilization, database query efficiency, and configurability. The results show that when compared to centralized event correlation, dynamically configured distributed event correlation provides increased flexibility, a significant reduction in network traffic in low and medium accountability environments, and a decrease in database query execution time in the high-accountability case.
Joint Chiefs of Staff. Joint publication 3-13: information operations. United States Department of Defense, 13February2006.
2.
LibickiMCJohnsonSE (eds). Dominant battlespace knowledge. National Defense University Press, October1995Washington, DC., USA.
3.
GrimailaMRFortsonLWSuttonJL. Design considerations for a cyber incident mission impact assessment (CIMIA) process. In: Proceedings of the 2009 International Conference on Security and Management (SAM09), Las Vegas, NV, 2009.
WilshusenGPownerD. ‘CYBERSECURITY: Continued Efforts Are Needed to Protect Information Systems from Evolving Threats’, http://www.gao.gov/new.items/d10230t.pdf (2009, accessed 10 December 2009).
10.
SahA. A new architecture for managing enterprise log data. In: Proceedings of LISA, 2002, Pp.121–132.
11.
MyersJGrimailaMRMillsRF. Towards insider threat detection using web server logs. In: Proceedings of the Cyber Security and Information Intelligence Research Workshop (CSIIRW 2009), Oak Ridge National Laboratory, Oak Ridge, TN, 13–15 April 2009.
12.
MyersJGrimailaMRMillsRF. Insider threat detection using distributed event correlation of web server logs. In: Proceedings of the 2010 International Conference on Information Warfare and Security (ICIW 2010), WPAFB, OH, 8–9 April 2010.
13.
ShenkJ. Demanding More from Log Management Systems, Technical report, SANS, 2008.
14.
AndersonJP. Computer security threat monitoring and surveillance. Fort Washington, PA: James P. Anderson Co, 1980.
15.
LonvickC. RFC 5424: the syslog protocol. Internet Engineering Task Force, Network Working Group, 2009.
16.
GerhardsR. RFC 3164: the BSD syslog protocol. Internet Engineering Task Force, Network Working Group, 2001.
17.
BingMEricksonC. Extending UNIX system logging with SHARP. In: Proceedings of the 14th USENIX conference on System administration (LISA ’00), 2000, pp.101–108.
18.
JakobsonG. The technology and practice of integrated multi-agent event correlation systems. In: Proceedings of KIMAS, 2003, pp.568–573.
19.
BouloutasAHartGWSchwartzM. Simple finite-state fault detectors for communication networks. IEEE Trans Commun1992; 40: 477–479.
20.
CronkRNCallahanPHBernsteinL. Rule-based expert systems for network management and operations: an introduction. IEEE Network1988; 2: 7–21.
21.
DoorenbosRB. Production Matching for Large Learning Systems, PhD Dissertation, CMU-CS-95-113, Department of Computer Science, Carnegie Mellon University, 1995.
22.
PougetFDacierM. Alert correlation: review of the state of the art, Technical report, EURECOM Research Report RR-03-093, Institut Eurecom, France, 2003.
23.
SadoddinRGhorbaniA. Alert correlation survey: framework and techniques. In: Proceedings of the 2006 International Conference on Privacy, Security and Trust (PST ‘06), 2006, 37.
YeminiSAKligerSMozesEYeminiYOhsieD. High speed and robust event correlation. IEEE Commun Mag1996; 34: 82–90.
26.
AamodtAPlazaE. Case-based reasoning; foundational issues, methodological variations, and system approaches. AI Commun1994; 7: 39–59.
27.
LewisL. A case-based reasoning approach to the management of faults in communication networks. In: Proceedings of the 12th Annual Joint Conference of the IEEE Computer and Communications Societies (INFOCOM ‘93), April 1993, pp.1422–1429.
Ben-GalI. Bayesian networks. In: RuggeriFFaltinFKenettR (eds) Encyclopedia of statistics in quality & reliability. Wiley & Sons, 2007.
30.
LazarAAWangWDengRH. Models and algorithms for network fault detection and identification: a review. In: Proceedings of Singapore ICCS/ISITA ‘92, November1992, pp.999–1003.
31.
GruschkeB. Integrated event management: event correlation using dependency graphs. In: Proceedings of the 9th Annual IFIP/IEEE International Workshop on Distributed Systems, Operations, and Management, October 1998.
32.
DavisRShrobeHHamscherWWieckertKShirleyMPolitS. Diagnosis based on description of structure and function. In: Proceedings of the 2nd National Conference on Artificial Intelligence, American Association for Artificial Intelligence, 1982, pp.137–142.
33.
HanemannASailerM. A framework for service quality assurance using event correlation techniques. In: Proceedings of Telecommunications, Advanced Industrial Conference on Telecommunications/Service Assurance with Partial and Intermittent Resources Conference/e-Learning on Telecommunications Workshop, July 2005, pp.428–433.
34.
WietgrefeHTuchsKJobmannKCarlsGFröhlichPNejdlW. Using neural networks for alarm correlation in cellular phone networks. In: Proceedings of the International Workshop on Applications of Neural Networks in Telecommunications, 1997, pp.248–255.
35.
SteinderMSethibAS. A survey of fault localization techniques in computer networks. Sci Comput Program2004; 53: 165–194.
36.
HasanMViswanathanRSuglaR. A conceptual framework for network management event correlation and filtering systems. In: Proceedings of Integrated Network Management, 1999, pp.233–246.
HansenSEAtkinsET. Automated system monitoring and notification with swatch. In: Proceedings of the 7th USENIX conference on System administration, November 1993.
JakobsonGLewisLBufordCJShermanCE. Battlespace situation analysis: the dynamic CBR approach. In: Proceedings of the Military Communications Conference (MILCOM 2004), 2004, pp.941–947.
48.
JakobsonGBufordJLewisL. Towards an architecture for reasoning about complex event-based dynamic situations. In: Proceedings of the International Workshop on Distributed Event-based Systems (DEBS 2004), 2004.
49.
TaiWO’SullivanDKeeneyJ. Distributed fault correlation scheme using a semantic publish/subscribe system. In: Proceedings of the 11th IEEE/IFIP Network Operations and Management Symposium (NOMS 2008), April 2008, pp.835–838.
50.
JakobsonGWeissmanMBrennerLLafondCMatheusC. GRACE: building next generation event correlation services. In: Proceedings of the IEEE/IFIP Network Operations and Management Symposium, 2000, pp.701–714.
SailhanFBourgeoisJ. Log-based distributed intrusion detection for hybrid networks. In: Proceedings of the Cyber Security and Information Intelligence Research Workshop (CSIIRW 2008), Oak Ridge National Laboratory, Oak Ridge, TN, 12–14 May 2008.
53.
MITRE. ‘Common Event Expression: A Standard Log Language for Event Interoperability in Electronic Systems’, The MITRE Corporation, http://cee.mitre.org/ (accessed 13 November 2009).
54.
Symantec. State of Enterprise Security, Technical report, Symantec Corporation, 2010.
Federal Trade Commission. ‘In the Matter of Genica Corporation, a corporation, and Compgeeks.com, also doing business as Computer Geeks Discount Outlet and Geeks.com, a corporation’, FTC File No. 082 3113, http://www.ftc.gov/os/caselist/0823113/index.shtm (2009, accessed 2 June 2010).
58.
LyonG. ‘NMAP: Network Mapper’, http://nmap.org/ (accessed 15 October 2010).
59.
MyersJGrimailaMRMillsRF. Adding value to log event correlation using distributed techniques. In: Proceedings of the Cyber Security and Information Intelligence Research Workshop (CSIIRW 2010), Oak Ridge National Laboratory, Oak Ridge, TN, 21–23 April 2010.
