Deploying BeeGFS with E-Series Storage
Joe McCormick
Deploying a complex parallel file system across multiple nodes can be tediously repetitive. As you thumb through (okay, scroll through) the deployment guide for the umpteenth time, have you ever found yourself wishing it would just deploy itself? If so, this post is for you.
Imagine This
Let’s consider a scenario. It’s 6:02 p.m. on a Friday, and that software you said would be deployed by the weekend still doesn’t work quite right. As you yet again retrace your way through the small novel of a deployment guide, you realize you missed step 42 substep J on node 13 in which this thing needs to be set if you want that thing to work (which most people would).Does this sound familiar? We’re all human; we all make mistakes sometimes.
Because thinking up fictional IT horror stories during working hours is fun, let’s consider another scenario. It’s 3:00 a.m. and you’re on pager duty. The phone rings to inform you that a critical piece of software has gone down and you’re responsible for fixing it. Looking through change requests, you see a new node that was added earlier in the day and that seems to be the crux of the problem. Sure enough, the new node was deployed with an outdated configuration file that somehow broke everything under the load.
This one really isn’t anyone’s fault. The new guy followed the steps in your internal wiki, but nobody thought to update the notes after the last upgrade. Tribal knowledge can be a real pain, especially when it wakes you up at 3:00 a.m. What if there were a way for your infrastructure to be self-documenting? What if there were a way to describe your infrastructure as code? (We’ll call that “IaC” from here on for the sake of brevity, and because engineers love acronyms.)
The Benefits of Infrastructure as Code (IaC) Tools
I love IaC tools because they allow you to describe how you want your infrastructure deployed, and then to click “run,” sit back, and watch. I’m resisting the urge to say that they encourage more coffee breaks, but they certainly make them an option. As a bonus, you can often knock out a lot of configuration work before new equipment even shows up on the dock.Another key benefit of IaC is version control. Describing your infrastructure in a series of human-readable files allows them to be stored in source control tools like Git. You can track who made changes when, and a well-written commit message can capture why. You can even tie in continuous integration and continuous deployment (CI/CD) tools that push configuration changes to a staging or lab environment for testing before deploying to production. IaC allows you to manage your infrastructure configuration like a software pipeline, which goes a long way toward ensuring consistent and reproducible deployments.
In our first scenario, the desired configuration would have been locked down before node 13, and you would have headed home long before 6:02. For our second scenario, all the new guy would have needed to know is how to use your IaC tool to deploy new nodes, with no ambiguity about how they must be configured. I’m just scratching the surface here; you can find dozens of lengthier posts online with greater depth on the benefits of taking this approach to managing infrastructure.
Deploying BeeGFS
When I first got started with BeeGFS, I captured the deployment (and removal) tasks in Ansible playbooks that evolved into an Ansible role. As I got more experience with BeeGFS, I found new ways to optimize the deployment and to build on the role. This approach allowed me to document the recommended steps while making them happen automatically.We understand that getting started with tools like Ansible requires time, so we used this as the foundation to develop an Ansible role that you can use to deploy BeeGFS with NetApp® E-Series storage. Although you might need to tailor the role to fit your specific needs, we hope this role eliminates most of the upfront effort necessary to get started using Ansible with BeeGFS and E-Series storage.
This role is being released as part of a new NetApp E-Series BeeGFS Collection for Ansible, available on both Ansible Galaxy and GitHub. You will also need to install the NetApp E-Series SANtricity® collection, which is also available on both Ansible Galaxy and GitHub.
Although I’m hesitant to touch the debate about which IaC tool is best—there really isn’t one good answer—we settled on Ansible for its simplicity and ease of use. Perhaps more important, we’ve already built strong support for E-Series into Ansible, enabling you to complete the end-to-end deployment of a storage solution built on E-Series at any scale.
Learn More
Read: Automating BeeGFS Deployment with Ansible Join Webinar: Accelerate and Scale Your HPC Workflow with BeeGFS and NetApp
Joe McCormick
Joe McCormick is a software engineer at NetApp with over ten years of experience in the IT industry. With nearly seven years at NetApp, Joe's current focus is developing high-performance computing solutions around E-Series. Joe is also a big proponent of automation, believing if you've done it once, why are you doing it again.
