[ Subtitles have been automatically generated ] Hello everyone. Welcome to the recorded version of session 1351 how to design a NetApp StorageGRID deployment best practices and advice. Take a minute to read this confidentiality notice, please. My name is Ben Hauser. I'm from Pittsburgh, Pennsylvania. I graduated from the University of Alabama in Huntsville in 2022, and I was fortunate enough to land here at NetApp not too long after. I'm a technical marketing engineer for StorageGRID, focused on performance and sizing. So just to set the stage, StorageGRID is an extremely flexible product with many configuration options and features. By understanding StorageGRID capabilities and best practices, you can deploy a grid that meets and exceeds the needs of your business. Here's a quick agenda for today's session. we'll start with some key StorageGRID concepts. Then we'll dive into designing a StorageGRID deployment including grid topology, data protection, future growth, and performance. And finally we'll go over a simple sizing example. So the first thing I want to cover before even diving into StorageGRID is the object market as a whole, how it's changed and where it's going. These bubbles on the right show the progression of object storage in our customers data centers. On the left is object 5 or 10 years ago, primarily deep in deep archive. On the right is object today and into the future. And I want to note that the archive workloads haven't shrunk , but rather everything else has grown around them. We see growth in app storage, data streaming, and HPC, and massive adoption of analytic workloads on object. We have some stats from IDC here, but the key point is that the demand for object storage continues to grow larger and larger. The benefits of object have led to S3 becoming the standard for cloud native apps as object has become essential to all sorts of businesses we've seen on prem and hybrid Cloud Object continue to grow. So I'm assuming most of us are familiar with StorageGRID. But just in case, I wanted to give a quick high level overview. StorageGRID is Netapp's software defined object storage suite with native support for the S3 API. It's been evolving for over 20 years and continues to lead innovation for on prem object storage. StorageGRID is extremely flexible in deployment with several deployment options available. It's also extremely scalable, growing up to 16 sites, 300 billion objects, and 800PB of capacity under a single namespace. You can see in this example we have four sites from the West coast to the East Coast, all in a single namespace. StorageGRID also integrates seamlessly with AWS, S3, Azure, and Google Cloud for a powerful and flexible hybrid cloud deployment. Another important slide here. This covers our deployment options and appliance portfolio. StorageGRID can be deployed as software only on VMs or bare metal. But today I want to focus on our NetApp engineered appliances. Our hard drive based appliances are the SG 5812, the SG 5860, and the SG 6160. The SG 5800 are great entry level platforms, while the SG 6060 is our workhorse appliance for primary storage and data lakes. We also have the All Flash SGF 6012, our next generation high performance platform now offering 30 terabyte QLC SSDs for deep and performance storage. I'm also very excited to tell you about the metadata only SGF 6012, which is coming soon in StorageGRID 11.9. For some background, StorageGRID stores object data and metadata separately. For example, an object may be an MP4 video and its metadata will contain its Uuid location, Timestamp, etc. . The new SGF 6112 configuration will store only metadata, so no object data. This will help to expand metadata capacity and metadata performance when object capacity is not needed. It will be the same hardware, but a new deployment option. When you install a 6112 with 11.9. Conversely, 11.9 will also introduce a data only option for all our appliances, opening the door for even greater flexibility in deployment. We also have our service appliances, the SG1 ten and 1100. These can be deployed as load balancers and as admin nodes. The SG1 ten is best for smaller deployments or for an appliance admin node, and the SG 1100 provides high performance load balancing for large grids. The service appliances also offer flexible QoS roles with traffic classifiers. One of StorageGRID most fundamental features is Information Lifecycle Management, or ILM. An ILM policy is a ranked list of rules, and each of these rules defines how data is stored for a certain set of filters. You can filter based on tenant bucket, object size, user defined metadata, and more. Every object in the grid is continuously evaluated against these rules, and ILM will ensure that everything is where it needs to be. StorageGRID 11.8 introduced a feature called ILM Policy Tags, which allows the grid admin to create multiple ILM policies in addition to the default. Tenants can then choose between these policies on a per bucket basis. This is great for service providers who offer different levels of service. Here I have an example ILM policy from a three site grid. The first rule evaluated is to copy at dc1 for tenant VH, so all objects for this tenant will be stored just on data center one with two copies. The second is erasure coding seven plus five, which will store all objects greater than one megabyte with geo dispersed erasure coding across my three sites. Finally, the default rule at the bottom is one copy per site, which will replicate one copy of each object to each of my data centers if none of the other filters apply to that object. Now that we have a basic understanding of StorageGRID and the object market, we can dive into the main content of this session. Designing a StorageGRID deployment. So when we start to prepare for our new StorageGRID deployment, there are four main areas we need to consider and understand that I'll be covering today. The first is capacity. This is pretty straightforward. How many terabytes or petabytes of usable space do I need both for object and metadata? The second is data protection. StorageGRID is very flexible with how you can store and protect data, so there are some key decisions to be made here. The topology of the grid, whether it's a single or multi-site grid, will impact this. The third is future growth. Even though you might not think to consider expansions when initially deploying the grid, there are some steps we can take now to ensure that future growth is as simple as possible Understanding how you can grow in the future can also be helpful when deciding what you need now. It's good to have some idea of the expected growth and how large future expansions may be. And finally, the fourth item is performance. Some grades will have stricter performance requirements than others, so it's important to understand your requirements in the planning phase. So before diving into data protection with ILM, I want to cover something fundamental here. StorageGRID can be deployed in a single site or in up to 16 sites, and the topology will have a major impact on the four elements I just mentioned. I understand there isn't always a choice to be made around site counts, but rather it depends on existing infrastructure, budget requirements, etc. . Either way, it's important to understand the benefits and drawbacks of both single and multi site grids. Keep in mind that StorageGRID is extremely scalable. You can always add or decommission sites in the future. So I'll start by discussing single site deployments. One of the main benefits here is that metadata capacity is not limited by other sites, and I'll discuss this more in a second. You also don't need to worry about networking requirements between sites in a single site grid, which can make for a simpler deployment. Some drawbacks to single site deployments. The obvious one when compared with multi-site, is less durability and availability. If anything happens to my data center and I don't have extra backups anywhere else, then the data stored there is lost. Single sites are best when one site protection is okay, and for starting small and scaling in the future. Moving on to multi-site grids. The biggest benefit here is the increased durability and availability. In a multi-site grid with a write in config, you can lose a data center without losing any data or impacting the grid's availability. This is true for object data and metadata, as each site holds a complete copy of the grid's entire metadata database. You can also spread requests across sites for greater performance in a multi-site grid. And ILM makes it simple to ensure the right data remains in the right data centers to keep you compliant with any regional restrictions. Some drawbacks of a multi-site grid I mentioned that each site holds the entire metadata database for the grid. This means that the grid wide metadata capacity is actually limited by the smallest site. For example, if I have one site with eight storage nodes and another with four storage nodes, the entire grid can only hold four storage nodes worth of metadata. So half of the metadata capacity at the larger site will not be available, because when the smaller site fills up with metadata, the grid is effectively full. Metadata only appliances will help a lot here, but if you're considering asymmetrical sites, this is a good thing to keep in mind. Another key detail there are some latency and bandwidth requirements between sites for erasure coding, and I'll cover this more soon. Multi site grids are generally recommended over single site grids as they provide site loss protection. They're also crucial when there's a need to keep specific data in specific regions. And if you want to provide endpoints in different locations for super low latency. So now we'll cover one of StorageGRID most powerful features , which is data protection with ILM. There are several challenges businesses face around data protection. The primary being the need for durable data.loss is not acceptable and we need to ensure that everything is protected. At the same time, we need to ensure that we're getting the most out of our capacity. We need to protect data while storing data efficiently. We also need to ensure that data is stored in the correct regions for availability and compliance. And finally, we need to be flexible and meet different requirements for different types of data. Luckily, StorageGRID ILM makes it super easy to solve these problems with rule and policy based data protection. Some key things to remember about ILM.can protect data with replication or erasure coding, which I'll deep dive on here in a second. Storage appliances are already protected with Raid or DDP, which is basically erasure coding on the appliance. A level below ILM, we call this dual layer erasure coding. ILM only impacts the locations of object data, as metadata is always stored with three copies per site. Future ILM changes can be resource intensive if they need to move or reprocess data. For example, converting replicated data to erasure coded data will utilize compute and networking on the grid and could impact client performance. Finally, keep ILM policies as simple as possible to avoid unintended outcomes, especially when you're dealing with ILM rules that delete data. So erasure coding is one of two methods that ILM can use to protect data. Erasure coding will break each object up into individual fragments on ingest, which can later be reconstructed to retrieve the object. What's key here is the concept of data and parity fragments for k plus one erasure coding scheme, where k is the number of data fragments in M is the number of parity fragments. You need k fragments to retrieve the object and can sustain the loss of up to M fragments. So just to make this easy with an example, with four plus two erasure coding, each object will be broken up into four data fragments and two parity fragments. The six total fragments are distributed across six of our nodes, and if any four fragments are available, whether they are data or parity fragments, we can return the object. The key benefit of erasure coding is the ability to achieve similar or greater protection than replication, with more storage efficiency. StorageGRID supports ECC with individual sites and geo dispersed DC, where data fragments are split between three or more sites. This table covers StorageGRID supported geo dispersed erasure coding schemes. All of these schemes are supported on single site grids as well. Some best practices with DC. One key detail is that geo dispersed EC is not recommended for sites with Wan latency over 100 milliseconds. This is due to the need to send lots of data between sites. If each get needs to pull data fragments from sites with high latencies, you can imagine how this would impact client performance. Erasure coding is not recommended for objects smaller than one megabyte, and is not supported for objects smaller than 200kB. This is due to the extra work that Ek takes to split up and reconstruct objects. For small objects with high transaction rates, that extra work is much more impactful. As a general rule of thumb, start with a minimum of K plus M plus one nodes for erasure coding. For example, with four plus two ek, you would want to start with seven nodes at a minimum. This ensures that if a node is down, you can still fulfill the requirements with six nodes. Replication is the second data protection method offered by ILM. It's very simple. You decide how many copies you need for each object and where the copy should be stored. Replication is generally less resource intensive on the nodes without the need to break up and rebuild each object. Some replication best practices. You can use replication with any object size. You should never create ILM rules with one copy to avoid risking data loss, and you should use replication for multi-site grids with high latency between sites when EC cannot be used. This screenshot is the default one copy per site ILM rule on multi-site grids. This rule will make one copy of all our objects on each of our data centers. So how can you choose an ILM configuration? The key questions to ask are one. What is my average object size? Two what is my grid topology? How wide of an EC scheme can I support? Three if this is a multi-site grid, what is the latency between my sites? And four if it's a multi-site grid, do I need one complete copy on every site for availability? Based on these answers, the choice should be pretty simple. We recommend erasure coding for most situations. Replication is best for when EC cannot be used due to high latency between sites or for small objects, or when you want a copy on every site for high availability. Now that we understand the details around data protection, I want to cover grid expansions. Even though expansions occur after the initial deployment, we can ensure that future growth is easy and issue free by keeping a few things in mind. Now. It's also easier to understand what you need now when you understand how you can grow in the future. We find that StorageGRID expansions are pretty common. A lot of customers buy StorageGRID for something like archive or FabricPool, but find that one's object is available. The use cases start to come out of the woodwork. StorageGRID expansions are flexible. You can add nodes to existing sites and add entirely new sites. As you can see in this example, the grid starts with three storage nodes and one admin node. And then we can expand to add two new storage nodes. And finally we can expand to add a whole new site. Expansions and StorageGRID are non-disruptive, and we have a few avenues for growth that I'll cover in the next slide. There are a few things to consider during the planning stage that will make life easier in the future when it comes time to expand. The first is the performance impact of mixing SG 5800 nodes with SG 6100 nodes. Mixing these node types in a single site can lead to performance issues, as the fast nodes end up waiting on the slow nodes, and a bottleneck develops around metadata performance. This is soon to be improved with data only appliances and 11.9 as data only. SG 5800 nodes will not face the same metadata bottlenecks. We generally recommend starting with SG 6100 nodes if you expect growth, especially around performance, but 11.9 opens the door for some more flexibility. Here, the active EQ scheme and current capacity utilization will also dictate future expansions. For example, if you're using EQ two plus one and the current nodes are all full, you'll need to expand with three or more nodes to ensure that each EC fragment can be stored on its own node. With wider EC schemes, the issue becomes more prominent in the same situation with EC eight plus two. For example, you would need to add at least ten nodes to fulfill the requirements. Another option is EC rebalance, which will let you move erasure coded data around to balance the capacities on the nodes. Though it's always better to expand early since rebalance can be a lengthy and resource intensive operation. Finally, we always recommend to not wait too long to expand the grid. Once full, a grid becomes read only and can no longer ingest data. Like I mentioned, a full grid will offer less flexibility in expansion when using erasure coding. Be sure to understand your procurement cycle and plan ahead to avoid risking a read only grid or an emergency expansion. So when it comes time to expand the grid, you have three potential paths. Adding new nodes. Replacing existing nodes, and growing existing nodes. Adding new nodes is pretty straightforward. You can expand a current site or add a new site, and most expansions will follow this path. The next option is replacing an existing node, which is best for refreshing older hardware. StorageGRID offers a capability called Node Clone, which enables duplicating a current storage node to a new one.thing to note here for node clone, the destination node must have a higher capacity than the source node. So generally you will need to node clone to an appliance with larger drives than the original. An exception is cloning from the SG 6060 to the SG 6160. You can node clone with the same drive capacities here, since the SG 6060 actually has two more hard drives than the SG 6060. It's also important to note that you cannot clone to SGF 6012 if node clone is not available. The other option is to add new nodes and then decommission the old ones. Finally, the last option is to grow existing nodes. This is only available for the SG 6060 or SG 6160, which support adding 1 or 2 expansion shelves to increase the object data capacity. All right. Now we can move into one of my favorite topics, which is StorageGRID performance. I wanted to start the discussion with a deeper look at our StorageGRID SG 6100 appliance family. These appliances are engineered for high performance and are what we recommend for workloads with high performance requirements. The SG 6001 60 is a CVO box with 60 hard drives. We offer a few capacity options for the drives, ranging up to 22TB on the SG 6160. It also uses two SSDs for caching, which are dedicated to metadata I o to avoid any disk bottlenecks there. The SG 6001 60 excels when deep storage and stable performance are key. The SF 6001 12 is our next generation all flash appliance, so one new box with 12 NVMe SSDs, and we offer up to 30 terabyte QLC here. The SFF 6001 12 provides blazing fast performance for primary storage data lakes, AI, ML and analytics, and more. The SGF 6112 has only seen one StorageGRID software update since its debut, but in that update, we saw a 40% performance improvement for some workloads. So this thing really is the future of high performance object storage. Finally, I mentioned the metadata only option for SGF 6012. This is the same hardware as the standard SGF 6012, and metadata only is an option when you install the node. The use cases here are to provide additional metadata, performance and capacity when needed. Now I want to cover some best practices around performance sizing StorageGRID performance scales linearly with node count, so as you add new nodes, your performance will increase as well. Stored object compression and encryption will impact performance, and you can expect a 5 to 10% reduction in throughput when these. Options are enabled. Like I mentioned, do not mix SG 5800 and SG 6100 on the same site. With a data only appliance option coming soon. There will be more flexibility here, but SG 5000 806,100 nodes should not be mixed if they're both handling metadata. The SG 6001 60 and the SGF 6001 12 can be mixed in the same site with some considerations. Check out INSIGHT session 1032 for more info. In a customer case study on mixing flash and hard drive nodes. This is a super interesting case study on a unique grid topology, and I would urge everyone to check it out. You can also mix our refreshed appliances with their predecessors in the same site with no issues. So for example, you can mix a SG 6060 and an SG 61 60in the same site with no problems. Erasure coding and replication will perform differently on different appliances. On some appliance models for certain operations and object sizes, replication will be faster, but this isn't always the case. Even though there's extra work with EQ, with large objects and high concurrency. EQ can provide faster performance in some situations. For performance guidance or sizing help, be sure to contact your NetApp partner. This is also a good time to cover StorageGRID load balancer best practices. Load balancers are key to StorageGRID performance, and you want to ensure they're configured properly to get the most out of your grid. It's important to consider the max bandwidth of SG 110 or SG 1100 load balancers. When sizing, each load balancer can handle a certain amount of bandwidth, and for high performance, you may need several load balancers. Global load balancing can improve performance by spreading requests across sites and service appliances. This can be accomplished through third party load balancers or with our StorageGRID service appliances through the load balancing link cost setting. It's always recommended to configure two or more load balancers in a high availability or Ha group to ensure that the endpoint remains available if one of your load balancers goes down. The last thing I want to cover here is the importance of client concurrency with StorageGRID performance. StorageGRID performs differently at different concurrencies. This graph shows put performance on the SGF 6012 across different thread counts for different object sizes. The x axis is the thread count per node and the y axis is throughput. Each line on the graph is a different object size. As you can see, each object size behaves differently and more thread counts are not always faster. Different appliances and workloads will see different behavior. So this graph might look a lot different on an SG 6160 or for even two copy on STS 6112. It's important to understand concurrency requirements and characteristics of your applications and workloads as best you can. We might not always know before the grid is deployed, but it's good to get an idea of the client concurrency if possible. So now I want to go through a really basic StorageGRID sizing. In real life, this process will be a collaboration between NetApp and the customer, and it relies on our internal sizing tools. My hope here is to kind of pull back the curtain and show how this process works, with a simple example. So here are the requirements that the grid should meet. First of all, we need five petabytes of total usable capacity. We're starting with three sites in the same region. Since the sites are geographically close, we don't need to worry about the latency impacting erasure coding performance. All data must remain available with the total loss of one site or the loss of one node per site. So with one site down, all of our data needs to be still available. And if all sites are up, we need to be able to lose one node per site. For performance, each site needs to provide four gigabytes per second. Put performance with four megabyte objects so 12GB per second total across the grid. The client concurrency is configurable, which makes it easy on us here to optimize the performance. And finally, the future growth is expected. So we want to keep that in mind as we're sizing now. So we can start by considering the data protection requirements. Since the average object size is four megabytes and we don't have an issue with latency between sites, I immediately chose to use erasure coding here. Since we have three sites. I picked out the ECC schemes that support a minimum of three sites, though all of these options provide slight loss protection out of four plus two, six plus three, two plus one and seven plus five, only six plus three and seven plus five can keep all of our objects available with one node down per site. Let's choose six plus three since it requires less storage overhead and fewer nodes than seven plus five with one node down per site, so three nodes down total, we will still have at least six fragments available on the remaining nodes. So next I used our sizing tool to generate some options for the capacity requirement using six plus three erasure coding. Since I only care about capacity now, I use the largest drive options for each appliance. Here you can see three options with SG 6160, SG 5860, and SGF 6112 with QLC for both SG 6160 and SG 5860. We need 12 nodes total and four per site, and we actually have a lot of capacity headroom here with 7.5PB available for SGF 6112. We need 27 nodes total and nine per site to meet the capacity requirement. You can also see here that the certain amount of objects are supported per grid. For the SG 6160 grid, we support 6 billion for the 5860 grid, we support 2 billion. And with all the nodes in the SGF 6112 grid, we support 13.5 billion. Next, we need to consider the performance requirement, which was four gigabytes put per second with four megabyte objects per site. Once again, I used our sizing tool to generate some options for SG 6160. SG 5860 and SGF 6112 that meet the performance requirements. As you can see, the SG 6160 and SG 5860 require more nodes to meet the performance requirement than the capacity requirement for SG 6160. We need 24 nodes total . For SG 5860, we need 39 nodes total. Since we need more nodes to meet the performance requirement than the capacity requirement, I switched to a lower capacity point with eight terabyte drives on the 6160 and 5860 for SGF, 6112. We only need 12 nodes total to meet the performance requirement, but the sizing does not meet the capacity requirements, with 2.4PB available. So after the sizing exercise, we end up with two options. I disqualified the SG 5860 since it required almost 40 nodes to meet the performance requirement. And since future growth is expected, I'd rather start with SG 6100 appliances. Here we have a spinning disk option with SG 6160 with eight terabyte drives. We need 24 storage nodes total and eight per site to meet all the requirements. The sizing here was driven by performance. After consulting our performance data, I saw that 128 client threads per site will provide the maximum performance, and one active SG 1100 can handle the performance requirements for each site. For high availability, we'll go with two SG 1100 and an Ha pair for each site. For an all flash option, we have the SGF 6012 with QLC. The sizing here was driven by capacity and we have a lot of headroom for performance. In this case, 1000 threads per site will provide the best performance, and once again, one SG 1100 can handle the performance of each site. The choice between spinning disk and all flash really comes down to your individual requirements and preferences. So some key takeaways from this session one we should understand StorageGRID key capabilities. Two we should understand best practices around grid topology ILM expansions and performance. And finally, after today's session, I hope you have the knowledge to deploy a grid that meets and exceeds the needs of your organization. Here are some additional StorageGRID links. These are all great resources, but I want to highlight the StorageGRID enabled site which contains technical reports, guides, APIs, and more published by the TM team. I want to invite you to check it out if you haven't, there's a lot of good information there. Here are some additional insight sessions covering StorageGRID. And finally, feel free to reach out with any questions or comments or feedback at Ben hauser@netapp.com. Thank you for watching.