July 30, 2013
Part III: Insights of Log Content Analytics and Its Future
By: Colin Puri

The future of log content analytics relies heavily on how extensible the solution platforms and frameworks are that exist. Naturally, this raises question of what insights many may find useful and what can be garnered from log files. The near future for log content analytics lies in creating algorithms, platforms, and frameworks that support the following:

  • Vendor agnostic views of the underlying data

  • Parallel execution of tasks that scale elastically and horizontally

  • Discovery of correlations within and across log files

  • Semantic understanding and linking of events and concepts

  • Machine learning for anomaly detection and error tracking in both real time and offline modalities

  • Discovery of relationships and trends without knowledge of the underlying data

  • Advanced recommendation systems for cleaning and presentation of log file contents

Whether or not vendors see a need for certain machine learning or mining approaches and what value is there in the insights beyond parsing, storage, indexing, searching, alerting, and dash boarding of log file contents only time will tell.

What log content analytics is today will change and go beyond the current offerings that only provide parsing, storage, indexing, searching, alerting, and dash boarding. Deploying a solution platform that allows for the customization and implementation of machine learning algorithms will ultimately prevail. What can machine learning be used for? It can be used to discover patterns, detect anomalies, find interesting facts about customer data, pull out trace correlations both within a log file and across log files, and discovery of data trends. To accomplish this, platforms and solutions need to provide: technical and economic scalability, advanced ingestion and parsing, analysis and exploration capabilities, and enhanced visual expressiveness. Beyond these attributes, the solutions of today must allow for the implementation of custom algorithms and in the future must start looking at trace mining, correlations, advanced pattern and anomaly detection, trend detection, and recommendation systems. Log content analytics can be as broad or as scoped as needed, but for growth an enterprise needs to look at the future and choose the solutions with the greatest amount of extensibility.

Popular Tags

    More blogs on this topic