BlueXP is now NetApp Console
Monitor and run hybrid cloud data services
Hi. Welcome. Thank you for joining us. My name is Dean Stedman. I am a product manager here at NetApp. And joining me today will be Diane Patton. And Diane is going to walk us through a webinar on optimizing cloud utilization. So with that I will hand things over to Diane to get started. Thank you. Dean. So today we're going to talk about two ways to rightsize your cloud utilization in order to reduce your costs while still delivering that needed performance that your applications desire. So I'm Diane Patton, I'm a tme with NetApp, and I support Google Cloud NetApp volumes. And today we're going to discuss two things. The first is flex independent scaling. And then we'll go through auto tiering, which are two ways that you can help reduce your cloud storage costs with Google Cloud, NetApp volumes. We also have a demo at the end. But before that, let's go over what is Google Cloud NetApp volumes? Cloud NetApp volumes is a first party, fully managed Google service that provides. NFS and SMB storage services in Google Cloud as a first party service. It also integrates with other Google services such as metrics, and we actually have that in the demo later. Key management, billing, IAM, etc. in addition, it uses NetApp ONTAP technology underneath leveraging over 30 years of storage experience, including features such as backups, snapshots, replication, customer managed encryption keys, among many others. It supports a vast array of workloads from windows with SMB, and in fact, NetApp volumes is the only storage service in Google Cloud that supports SMB. It also supports, of course, Linux NFS apps. It also supports apps such as SAP, VMware Virtual Desktop, just to name a few. And even has integration with vertex AI. But based on your workload demands, there can be a need for different service levels. So NetApp volumes offers four different service levels. There's flex, there's standard, there's premium, and there is extreme. So today we're going to be covering the flex service level. The flex service level also comes with two different performance options. One is called custom performance and the other is called default performance. Today we're going to be covering these features that come with the custom performance. And remember custom performance is available with zonal flex pools. One of the huge advantage of that custom performance is it offers the ability to decouple the storage pool size from the performance that you get to that pool. And that's what we also call flex independent scaling. So if you hear me talk about custom performance or flex independent scaling, is pretty much the same thing. So what is flex independent scaling then? So flex independent scaling as I just mentioned earlier, is the ability to decouple the performance you get on the storage pool from the capacity or the size of the storage pool. So in the past, if you needed let's say a small storage pool with large performance, you had to overprovision that capacity, right? You had to make a pretty big storage pool in order to just achieve the performance that your application needs. But no more. Now, what we can do is we can create a small capacity or storage pool, right, with a small capacity, and increase the amount of throughput and IOPs on that small pool and vice versa. So in the case of flex independent scaling, as I was mentioned, as I was mentioning right there, you can create a storage pool of any size and then you can assign the throughput and the IOPs that you want to that specific storage pool. So now when you create that storage pool within a flex custom performance zonal pool, you get 64 Meg of throughput with it by default. It doesn't matter what size storage pool you use. And every additional Meg of throughput that you configure includes 16 IOPs. So again, you get 64 Meg of throughput and then you get 1024 IOPs with that by default on any capacity on any storage pool that you configure. And then with every additional Meg of throughput that you configure to that, you will also get 16 IOPs. So for example let's say you have you need a very small storage pool. So you can figure, I don't know a ten tebibyte storage pool. With that you will get 64 Meg of throughput and 1024 IOPs, but your application actually needs 128 Meg of throughput, right? So you can configure that additional 64. But with that additional 64 you'll also get an additional 1024 IOPs. So now you've got 128 Meg of throughput with 2048 IOPs. Now if you need additional IOPs, you can add on to that IOPs. Now in addition to that, this is all very dynamic. So if you're the capacity of your storage pool is too small, you can increase it. If you discover that hey, I need more throughput because now maybe I'm adding another volume or my application starts beefing up and needing more throughput. You could do that on the fly too. In addition to the IOPs. So what are some of the benefits of doing it this way versus the default performance way? Well, the first thing obviously is you can optimize cost efficiency, right. So you can optimize your cloud spend and be able to provision only what you need and not any more in terms of either performance or the capacity of your storage pool. And this allows you to be able to adapt your workloads or adapt your performance and your capacity to the workloads directly. It also helps eliminate thoseperformance bottlenecks. So let's say there's 1 or 2 times a day or a week when you know that the performance goes way up, but you can't really accommodate it. And that starts a bottleneck, you know, but you don't really want to have a huge or a very big, uh, storage pool to be able to accommodate that performance. Well, now you can keep that storage pool very small and still be able to accommodate that performance. And in addition to that, of course, there's no complex capacity planning. You just add up what you need and you configure what you need and you're good to go. So how do you set up the flex independent scaling? It's very simple. So the first thing you do is under your creating your normal storage pool. Right. You need to select the supported region and a zone. And today we support about 17 different regions with the custom performance that's changing all the time. So if you're watching this on a recording, please make sure to look at the Google documentation because that 17 is going to be increased. Um, and you'll be able to see exactly which regions and aresupported with the custom, uh, flex independent scaling. We also call it custom performance. And then you would choose the capacity that you want and the zone that you want. Next thing is that you would go ahead and set the throughput again. It comes with 6464. Every single storage pool comes with 64. So if you need more than that you would set that. You would only pay for the difference between what you set and that 64. And then the IOPs also comes with 1024 unless you configure more throughput. And then it would come with more based upon that additional throughput that you configure. And then you would configure that, and then you would just go ahead and create that storage pool. So the second feature we were talking about was auto tiering. And this also is currently in preview. Um, custom performance is not in preview, but auto tiering is in preview currently. And this also helps you save costs by giving you the ability to move data that you do not access, often into a cooler tier with less cost. So generally, what happens in this case is you've got data in a hot tier. This hot tier is frequently accessed if there's certain data that is not accessed often, and you can figure how long you would wait for that data to be accessed anywhere between two days and 183 days, which is about half a year. If it's not accessed, it will be automatically moved to a cold tier. And that cold tier ischeaper right than the hot tier is. So you can just leave it there for basically until you access it again. And you have to be able you have to access it via a random read in order to get it, because we don't want to move backups, for example, all the way back into the hot tier, which is used sequential. So if you do a random read to that cold tier, then the cold, the data in the cold tier will be considered hot again and will move back to that hot tier. So this is a way to really be able to save money for data that perhaps you don't use very often, you just need to store it somewhere, maybe for compliance reasons. Maybe it's old data that an old team had. You don't really want to delete it because you might need it someday. Um, but you're not really sure maybe what's in it, but you really don't want to delete it because it could be valuable. This would be a good option for that is to just move that into the cold tier. So that way you have it, you have access to it. You can always pull it back if you need it, but you're not spending a whole lot of money in order to maintain that. And again, tiering provides for uniform file access for both SMB as well as SME, um, NFS. So in order to enable auto tiering, right, it's basically done on two on the storage pool level and on the volume level. So on the storage pool level is where you would go ahead and allow auto tiering for volumes. So it's just a check mark in the UI. Of course, like anything else, you can either use the API or the CLI as well. Um, or you know, if you use automation, Terraform or something like that, you can use that as well. So you would go ahead and turn on the auto tiering on the storage pool level. At this point, all the volumes in that storage pool will be enabled for auto tiering. And then on the volume level, when you go ahead and turn the volume up, that's where you denote the cooling threshold. So again that could be anywhere between 2 and 183 days. This generally says that let the data remain unaccessed between this number of days before moving it to the cold tier. Now, the other option you could have in a volume is if you have a volume. In a storage pool that has auto tiering on, but you do not want it on that volume. You can pause it. You cannot turn it off, but you can pause it per volume. So how do you size the hot tier configuration? Well, in general you want to take your data set and then add a little bit of um, additional data or additional space to it. And then that's where you would configure the size of the hot tier. So this generally is the amount of volume that you think that you will need for all your hot data. There's a possibility you could do an auto increase. So in this case if this gets very close to being at capacity in that case, then you can go ahead and you can enable the hot tier auto increase which will automatically increase thehot tier size. Another way you could do that is to keep an eye on the Google metrics, because as I mentioned, we integrate directly with the Google metrics. So if you want to, you know, manually keep your eye on Google metrics and see, hey, I'm getting very close to the top of the hot tier. I want to add more. Then you can go in manually and do that. So again we allow an auto increase if you'd like to do that. Or you could do it manually by keeping track of how much data is in there via the metrics or, you know, just looking at it in general. Now another feature that we offer with the flex auto tier is called hot tier bypass. And again, this is also just a check mark in the UI again available via the API and the CLI. But in this case what would happen is we generally think of it for use for migration. So if you're migrating data from, let's say, an archive into Google Cloud NetApp volumes, and you don't really want it to go into the hot tier because you know, it's archived data, you can just send it directly into the cold tier. So then when you turn on the hot tier bypass, the migration data will bypass the hot tier. Go directly into the cold tier, and then maybe there is some data in there that you access regularly. Well, if that's the case, you would just turn off the hot tier bypass after the migration is done. As you access that specific data, which is probably a minority of the data that will get brought back to the hot tier as it's accessed. So this would be a good feature to use for data migration, like maybe from on prem into Google Cloud. NetApp volumes for example. And again it can be disabled after that initial migration. So let's do a demo. Quick demo. We've got a demo here on independent scaling. Um, for reasons of time, we've only done one demo on independent scaling. If you're interested in seeing a demo on the auto tiering, please feel free to reach out to us and we can make that happen too. So what this demo consists of is I've got one Linux VM called tme fio vm zero that's mounted to a volume in a flex storage pool with zonal availability with custom performance. So 310 gig volume. What we'll do at that point, we've also got Metrics Explorer taking a look at all of the data on that. So what we'll do is we'll start Fio and we're going to just start sending data throughit to the flex storage pool. And again that's got custom performance set at a one gigabit per second throughput. After that's going we can see it coming up on Metrics Explorer. Then we're going to go ahead and change that storage pool throughput on the fly. So we'll go back to the storage pool. We'll edit it and we'll go from one gig to two gig. We'll leave the capacity exactly the same at 310 gigs. And then we'll see Metrics Explorer go up to two gig. So let's get started. So we're going to start by creating the storage pool. We'll give it a name. Call the tme perf pool. We're going to choose a region. Service level is flex and availability is zonal. And then we're going to choose zone from that. And we chose zone A. Now you can see the capacity is asking for the capacity. So we'll enter the capacity. And now you can see the custom performance is enabled in that location. So we're going to choose a throughput. We'll do 1000 which is um one gig as we had just mentioned earlier, and then a number of IOPs. We'll set up our connections and then we're going to go ahead and label it, and we'll go ahead and create that storage pool. So after some time, that flex storage pool with the custom performance will come up. And remember we did a throughput of one gig. Now we're going to create a volume. So we'll do we'll call it tme Fio. We'll select the storage pool that we just created. The capacity is 310. As I mentioned earlier don't forget the export rules. So that way we're able to mount and then we'll go ahead and label that and wait for that volume to come up. So that volumes up. And now we've already mounted that Linux VM to that volume as you can see here. So then at this point we'll go ahead and we'll start running Fio to create some traffic. And we should be limited to one gig based upon what we configured on our, um, storage pool. So we can see now that Fio is already reporting that. So now let's go to the Metrics Explorer. Now I sped this up in the interest of time, as you can see here. So we can see that starting to increase and increase over time. And now it's reporting that it also is getting that one gig that we configured on that custom storage custom performance storage pool. So now that we're pretty stable there, we'll go back to that same volume and we'll edit it. And now we're going to change that throughput to two gig and we'll leave everything else the same. But you could change the IOPs also if you wanted to. And now we can see over time that the Metrics Explorer is also reporting um sending through it two gig. So some key takeaways here is with the flex zonal availability custom performance which is also called flex independent scaling. We can decouple the capacity from the throughput and the IOPs. And make them independent of one another which allows you more granularity for your applications. So now if your applications need higher performance, but perhaps a smaller storage pool size or smaller volume, it's able to do that and you only pay for what you use. So you want to be sure to use the right performance numbers that your workload needs. And as your workload evolves, your performance can be changed on the fly. So if you decide to increase your storage pool size, you can also increase your performance or decrease your performance. Um, and you can also increase your IOPs as well if you want to add more volumes, for example, you can do all of that, and that way you only are paying for what you're actually using. You can also take auto tier and auto tier your cold data, which will reduce your costs even more. So if you're interested in learning a little bit more about this, we can get started today by enabling NetApp volumes in your environment. We also have a lab in Google Cloud skill boosts for NetApp volumes. If you're interested in more information on the flex service level as well as Flex Custom Performance, you can go to this URL. Um, this QR code here, as well as to learn more about auto tiering. There's a blog that was just published recently that goes through the flex auto tiering and how that works. So thank you very much for watching. Great. Thank you Diane. We did have a couple of questions. Come in via the chat functionality during the presentation here. Okay. So the first one of those is um what Google Cloud regions are these capable. Are these capabilities available in. So theflex custom performance specifically is currently as of today available in about 17 different regions. This is regularly updated in the Google Cloud documentation. So if you're watching this via a video I would highly suggest going to that documentation and taking a look, as there's constantly additions to those regions where it's available and the default performance is available on all regions. Great. Thank you. And then the second question is, uh, what is the pricing of cold tier data?on the cold tier is priced lower than data on the hot tier, and you can find that on the website. Um, and then for there's also an additional small fee for the data transfer. So you need to keep that in mind. Also, if you're thinking about, um, doing a lot of transfers back and forth. So you would want to be sure that the date, the,amount of time, you know, between the two days and the 183 days is appropriately sized to try to minimize the amount of data transversing back and forth. So, yeah, you need to do a little bit of math there in order to be able to determine exactly, um, the right number between that two days and that 183 days, based upon your application and your workload. Excellent. Thanks, Diane. All right, folks, that wraps today's webinar. I want to thank everyone for joining. And, uh, we will catch you on our next podcast. Thank you very much. Thank you.
Explore the importance of right-sizing your storage provisioning to optimize cloud performance and efficiency using Google Cloud NetApp Volumes.