Introducing a Framework to Monitor NetApp E-Series Solutions
Joe McCormick
Good software development considers functionality from the perspective of the user. For example, as a systems administrator, I want to easily monitor the systems in my environment to ensure that they perform and function as expected. But software is rarely described by a single user story, so let’s add another. As a systems administrator, I want to easily monitor the systems in my environment from a single monitoring solution.
New Feature of E-Series Performance Analyzer
That user story forms the foundation of our vision for the newest feature of the NetApp® E-Series Performance Analyzer: plug-ins. If you have ever played PC games, you might be familiar with the concept of a mod. Mods are community-developed content that extends the original functionality of a game. Mods can be as simple as new character skins, or they can create entirely new game modes. A key takeaway is that players are able to build upon core functionality to add features that the original developers might not have envisioned.Before we go on, if you’re unfamiliar with the E-Series Performance Analyzer, let me briefly introduce you to it. There are a few key components to monitoring any environment:
- A way to collect desired metrics
- A database to store those metrics
- A method of graphically displaying the stored metrics
Why Add Plug-ins?
Plug-ins came about after we tried to implement BeeGFS support into this monitoring stack for ourselves. Originally, that process was entirely too convoluted, and thus the new plug-in framework was born. We wanted a simpler mechanism to extend the core functionality of the E-Series Performance Analyzer beyond simply monitoring E-Series storage systems.Starting with the 2.1 release of the E-Series Performance Analyzer, you can create your own plug-ins by defining new collection mechanisms to store metrics in InfluxDB and add dashboards to Grafana. As of version 2.1, InfluxDB and Grafana are considered core components, and all collection methods (including collecting metrics from E-Series) are considered plug-ins.
One of the reasons we created this new plug-in framework is to be able to add support for monitoring new NetApp solutions. For example, NetApp offers support for BeeGFS, and we wanted you to have a simple way to monitor BeeGFS alongside your E-Series storage systems. There is a BeeGFS monitoring service (beegfs-mon) that can feed metrics into InfluxDB. It provides several predefined Grafana dashboards but does not include InfluxDB or Grafana. The E-Series Performance Analyzer provides a convenient “batteries included” approach to monitoring BeeGFS, and the plug-in for BeeGFS is available on GitHub.
At this point, you might be wondering about the limits of this plug-in approach. A key requirement is to get metrics from the monitored endpoint into the containerized InfluxDB instance. If what you’re monitoring can push metrics into a database, you simply need to point it at the InfluxDB instance, and your plug-in needs only the Grafana data source and dashboards that you want to include. For other devices, you might need a “middleman” that can pull metrics from the endpoint and route them into InfluxDB. For E-Series systems, this was accomplished with a Python script. Some software might, like BeeGFS, provide its own method of retrieving and storing data (for example, beegfs-mon). If this piece is external to the monitored endpoint, it might make sense to containerize it as part of your plug-in for the E-Series Performance Analyzer, but this is not a requirement.
We’d love to get your feedback on the E-Series Performance Analyzer and hear how you’re using it in your environment. In particular, if you want to report a bug or make an enhancement request, please submit them on GitHub:
- E-Series Performance Analyzer bugs and enhancement requests
- BeeGFS plug-in bugs and enhancement requests
Learn More
Read the NetApp E-Series + Grafana: Performance Monitoring solution brief to learn more about this powerful, easy-to-use tool for monitoring the performance of your E-Series storage systems.
Joe McCormick
Joe McCormick is a software engineer at NetApp with over ten years of experience in the IT industry. With nearly seven years at NetApp, Joe's current focus is developing high-performance computing solutions around E-Series. Joe is also a big proponent of automation, believing if you've done it once, why are you doing it again.
