A Method for Scalable Real-Time Network Performance Baselining, Anomaly Detection, and Forecasting

Introduction

The scale of today’s enterprise and service provider data network infrastructure presents unprecedented management challenges. This is particularly true in the area of network performance monitoring, which involves the real-time analysis of numerous variables collected continually from the potentially vast number of instrumented elements that make up the computing network infrastructure.

At the same time, businesses depend more than ever on the sustained reliability and performance of the computing network infrastructures, which host virtually every business function. Effective management of these assets is a business imperative.

Functionality versus cost is, of course, a key dynamic of the challenge. Poorly managed networks will prove unreliable, costing the business dearly. Well managed networks will reliably support the needs of the business, but can represent significant expense. The goal for network management application providers is to convince prospective customers that their application suite (or service) represents the best in class balance – delivering features that provide the most effective performance management capabilities, at the lowest total cost of ownership.

The network performance management solution must process potentially vast quantities of incoming data from the instrumented elements of the network, and from that data identify accurate, relevant, actionable information that allows network support staff and/or automation applications to address problems quickly and to preemptively address looming problems before they negatively affect service. This is a Business Intelligence (BI) challenge – to derive insights that inform business decisions, from the oceans of available data.

Three highly desirable features for an effective Performance Management application include:

• •

  Baselining - establishing performance patterns that should be considered normal, or expected, at any point in time

• •

  Anomaly detection - identifying traffic patterns (in real time) that are abnormal or unexpected.

• •

  Forecasting / Trend projection – extrapolating observed past performance into the future in order to make predictions for capacity planning and problem prevention.

These three areas are especially challenging to implement. They can be complex to design and describe, difficult and/or expensive to achieve good performance and high scale, and can produce confusing or inaccurate results.

Nevertheless, when done well, the appeal is undeniable; to effectively address these three areas is to produce concise, relevant and powerful insights that enable network operations to monitor and predict network performance and to maintain desired service levels.