Analysis of System Logs for Pattern Detection and Anomaly Prediction

Abstract

Log data is an important and valuable resource for understanding system status and performance issues. Machine logs record system states and significant events at various critical points to help debug performance issues and failures and perform root cause analysis. The log format is the standard log format which contains time stamp, process name, message, log type, id, etc. These logs are analyzed to detect any sequence of events which provide us with the patterns necessary for further implementation. From these patterns, future critical situations like memory issues, network down, machine shutdown are found. After detecting these critical situations, auto-remediation is done by sending alert messages or notifications which state the solutions like system restart, code re-execution which will help in avoiding these future critical situations and help to protect the system.