Leading the way for stateful Kubernetes with Astra

At NetApp, we saw the promise that Kubernetes held for stateful enterprise workloads early on, leading us to invest in, develop, and release NetApp® Trident (now called NetApp Astra™ Trident), an open-source, dynamic, external storage orchestrator. When released in April 2017, Trident was the world’s first external storage provisioner for Kubernetes. Trident enabled our customers to consume persistent storage quickly and easily from NetApp’s broad and proven storage portfolio, both on premises and in the public clouds. Today, hundreds of customers use Astra Trident as a persistent storage provider to back thousands of Kubernetes applications. Astra Trident implements the Container Storage Interface (CSI) standard for accessing persistent storage from Kubernetes pods and much more. 

Trident solved the first-order problem of backing stateful applications with reliable and high-performing NetApp storage. But Trident, by itself, did not satisfy our customers’ appetite for a richer set of data services, which was essential for the broad adoption of Kubernetes and containers for their business-critical applications.

We built NetApp Astra (announced as Project Astra in April 2020) to provide our customers with a richer set of data services essential for business-critical Kubernetes workloads with Trident as the foundation. With the general availability (GA) release of Astra in 2021, we offered the industry’s first fully managed (SaaS), multihybrid cloud data-management service built for Kubernetes with support for Google Kubernetes Engine (GKE) workloads on Google Cloud. Astra supports a rich set of application-aware data services, including backup and recovery, disaster recovery, application portability, and migration. We added support to Astra for Kubernetes applications on Amazon Web Services, Azure, and on premises. We even built a self-managed variant of Astra for customers who want more control over their environments and/or have specific security and data residency needs.

Here's what one customer has to say.

“We were really satisfied with Astra Trident, but it required technical knowhow. Astra Control gave us one-click deployments of volumes and backups. The dashboard allows us to see what is going on in the system without digging deep into the command line. Working with Astra has been a breeze.”

Nikola Milojević, CTO, The Remote Company

With Astra, we have driven Kubernetes-centric innovations across the broader NetApp portfolio to optimize NetApp’s storage for Kubernetes workloads.

NetApp Cloud Volumes Service, a fully managed storage offering in Google Cloud, introduced the concept of storage pools to address unique Kubernetes storage requirements. Persistent Volumes (PVs) created within storage pools offer a large number of small volumes with a minimum volume size of 1GiB, performance decoupled from provisioned storage capacity, high-speed volume creation and deletion, and data services like NetApp Snapshot™ copies. These enhancements make Cloud Volumes Service a great persistent storage provider for Kubernetes workloads in the public cloud. Astra integration offers additional application-aware data services, making it an even more attractive destination for Kubernetes apps.

Kafka is an important building block for many containerized cloud-native applications, and we made sure that Kafka runs well on NetApp ONTAP^® and ONTAP based products. Earlier this year, we contributed to the upstream Linux kernel for stream data-processing applications like Kafka to leverage network-attached storage by resolving the “silly rename” issue that caused Kafka to crash when resizing or repartitioning the Kafka cluster running on NFS. Our engineers implemented the changes on ONTAP (NFS Server) and the Linux NFS client. We contributed back the NFS client changes to the upstream kernel community. The client-side changes will be generally available in Red Hat Enterprise Linux 8.7 and Red Hat Enterprise Linus 9.1.

The upstream Kubernetes community has done a great job defining the CSI standard, which provides a standard interface for accessing file and block storage from Kubernetes clusters. At NetApp, we have always pushed the boundaries of CSI and are always working to solve our customers’ Kubernetes persistence challenges.

When our customers wanted to bring their business-critical applications to Kubernetes, which have stringent RPO and RTO requirements, we integrated Astra with NetApp’s fast and efficient SnapMirror® replication technology, enabling our customers to build business continuity for their applications with a low recovery point objective (RPO) and low recovery time objective (RTO). To match what our customers are using, we support disaster recovery of Kubernetes workloads for the following scenarios:

• On-premises data center A to on-premises data center B
• On-premises data center to a public cloud region
• A public cloud region to on premises
• Within and across public cloud regions

Here's what our customers have to say.

“We have several Kubernetes workloads that require DR failover. Today this is a very manual error-prone process to ensure data consistency during the recovery. Our process takes about 90 minutes. With Astra Control we can cut it down to a couple of clicks and a few minutes. That is a huge difference for us.”

Casey Shenberger, Cloud platform architect at Hyland Software

“For our DR solutions, the major concern we have is RPO and RTO. Without Astra, our recovery process is manual. This means restores take a long time. With Astra, it’s just one click to recover and that saves us a lot of time.” 

Jacob Jiang, Storage engineer, SuccessFactors at SAP

Cross-namespace PV sharing (coming soon) provides the ability to share persistent volumes across namespaces. Today, due to the design of the PVC API in Kubernetes, it's not possible to share across namespaces, because every PVC exists in exactly one namespace. The obvious workaround is to redesign a workload to run in a single namespace, but that's not always possible. When our customers asked us to address this issue, we built this feature even though the standard API didn't allow it.

Our customers are increasingly running stateful Windows applications on Kubernetes, or they want to do so in the future. We noted this trend and will soon add support for SMB to Astra Trident drivers for the NetApp storage portfolio in the public cloud (Azure NetApp Files, FSx for NetApp ONTAP, Cloud Volumes ONTAP, Cloud Volumes Service), and on premises. Once we add SMB support to the Trident drivers, customers can consume NetApp’s enterprise-class persistent storage from their Windows K8s workloads just like they do for their Linux workloads. In the near term, we will provide a complete set of advanced data services for Windows containers to address our customers’ business continuity and disaster recovery needs using Astra.

Astra is a NetApp offering, but we have responded to our customers’ desire to use data services with non persistent storage providers. We have added support for popular persistent storage providers like Amazon EBS, Google Persistent Disk, and Azure Disk in addition to NetApp’s storage portfolio. We also recognize that our customers are persisting the state of their K8s applications in popular DBaaS offerings in various public clouds and that they want Astra to incorporate such means of persistence.

Astra supports a broad set of popular Kubernetes platforms, including Amazon EKS, Google Kubernetes Engine, Azure Kubernetes Service, OpenShift Container Platform, Rancher Kubernetes Engine, VMware Tanzu, and upstream K8s. We added support for these platforms, which are the most popular among our customers, making Astra’s data services available on the world’s most popular K8s platforms.

We work with independent software vendors of popular cloud-native software like CloudBees to further the cause of protecting business-critical applications built using such building blocks.

With CloudBees and Astra, we have created an enterprise disaster recovery solution for continuous integration (CI) systems so that our customers can quickly and easily recover from an outage of their CI system, saving them hundreds of thousands of dollars lost in developer productivity. 

We haven't kept all our innovation to ourselves. We have launched open-source projects to share our learnings about Kubernetes data management. Earlier this year, we launched an open-source project called Verda, under the Apache 2.0 license, to provide “protection recipes” for popular containerized applications like Cassandra, Elasticsearch, MariaDB, MySQL, MongoDB, PostgreSQL, Redis, and Kafka in conjunction with Astra. These recipes include custom actions that are application-specific, including but not limited to hooks/scripts for quiescing/unquiescing databases before and after taking a Snapshot copy. Over time the project will be expanded and maintained with new content for protecting popular cloud-native applications with contributions from NetApp and the broader community. We also launched an Astra Toolkit project, which published a Python SDK to guide developers working with the NetApp Astra Control API.

At NetApp, we regard Kubernetes as the next platform on which most modern business-critical applications will be built. Therefore we're paving the way for data-rich business-critical workloads by innovating when necessary and integrating with NetApp’s existing enterprise-class storage and data services that have enabled enterprises to run their most important workloads on NetApp for 30 years.

Get started with Astra today!