BlueXP is now NetApp Console
Monitor and run hybrid cloud data services
Hey, thank you. Welcome everyone. Thanks for joining us today. My name is Dean Steedman. I'm a product manager here at NetApp. And today we'll be discussing how Google Cloud NetApp volumes helps customers to crush latency and control CL costs within the cloud. Uh so um a few bits of housekeeping before we begin our session. Uh if you do have any questions uh throughout the presentation, go ahead and enter those into the Q&A feature of Zoom here uh we will address a handful of those at the end of the session uh as well as we might address a few of those via text uh throughout thesession as well. So um with that today's session is being recorded. You will receive a link uh after the session with a copy of the materials and thedownload. Uh so if you want the slides or to watch the presentation again or share it with a colleague, you'll have access to that. Okay, with all of the housekeeping out of the way, let's go ahead and get started. Um so here at NetApp, a lot of the customers that we talk to are, you know, looking at ways to succeed uh in cloud deployments um without sacrificing some of the aspects of their business that they've had on premises uh for a very long time. And so of course, you know, cost, performance, being able to have an optimized deployment uh to make sure that you can scale at your cloud deployments as your business grows. Um, folks also want to maintain simplicity, making sure that they have all of their services online and available to their end users. Um, but want to be able to do that without adding a lot of complexity into their management and into their administration of their systems. And then finally, you know, data management, security are always first and foremost within our thoughts. uh making sure that we're helping people to keep their data safe, make sure that we're thechampion and the shepherd uh of information and that our customers can depend on us to uh help them to be able to maintain that posture um with their data. Um so those are some of the things thatwe think about um here at NetApp and that we partner with Google on.Now to this end we've introduced a product we've been inout now for um coming up on a year and a half uh in production a new product called Google Cloud NetApp volumes um which is essentially a firstparty Google service that is a storage solution uh that runs inside Google cloud it is a Google product so it is maintained supported and uh sold by Google it's just using NetApp's ontap storage operating system on the back end. And with this, we really focus on helping customers to uh manage and deploy data for enterprise applications. So it's everything from your oracles, your SAP, your virtualization instances um and theoperating systems that support those applications. And then of course looking to the future into you know AI and modern applications. So we've got deep integrations now with Google Enterprise or Gemini Enterprise as well as strong solutions for running Kubernetes uh in the cloud including Red Hat Open Shift. And so this really allows us to meet a very wide type of use case for the customers that we work with and really helps us to help uh deliver the right amount ofstorage performance and customization and data services that customers need to succeed with these deployments.Now, as a Google service, this is of course fully uh you know baked into all of Google's management. Uh so all of the administration, configuration, administration, configuration, administration, configuration, monitoring for Google Cloud NetUP volumes, all of that is done through Google's consoles. So that's theircloud console as well as all of their APIs. Um really making this a seamless experience from a customer perspective. uh so that folks can focus on managing their data in the cloud rather than having to manage the infrastructure within the cloud. For Google Cloud Netup volumes, we have four service levels. Um our flex service level really has the uh the greatest regional coverage. It's available in all 42 Google Cloud regions today. uh and is really designed to give customers a uh scalable, flexible solution that'll work for many different workload types with volumes starting as small as one gigabyte and scaling all the way up to 300 terabytes uh of storage capacity. Our premium and extreme service levels offer the highest amount of throughput performance, so up to 30 gigabytes per second. Uh they also offer up our largest volume sizes scaling all the way to three pabytes for an individual volume. So massive amounts of storage, massive amounts of throughput so that we can hit every workload type that a customer may have. And of course, you know, the advantages of using a firstparty service inside of Google Cloud as opposed to uh administering a you know, a third party or your self-managed storage system. Uh the first isthat it gives our customers a lot of speed and agility allows them like I said to focus on managing their data in the cloud and not managing infrastructure. Second, it's also much less complexIt allows folks to focus again like I said onmanaging that data but then also being able to skip all of those validation steps uh that are needed if you were to go with a thirdparty service. And then of course with it being fully integrated into Google cloud it does come through with a much lower cost. Um you know some of that being from you know your ability to focus just on the data rather than the infrastructure. The rest of it just being able to leverage um you know the you the different service levels we have to meet different requirements. So with Google Cloud Netup volumes, one of the recent features that we announced here uh last month was our ability to add in block storage solutions into the cloud as well as file. Uh this really again allows us to service up a much wider range of use cases for customers. Also gives folks africtionless migration experience. So if you think about the way that you've got data on premises today, whether that's file data or block data, being able to migrate all of that data, all of those applications to the cloud through the same process regardless of the data type simplifies that experience, makes it a little bit more streamlined, helps folks to get into the cloud faster, so they can begin uh changing their business and being able to innovate uh in the cloud rather than dealing with infrastructure. And of course, you know, some of the workloads that wefocus on here, the one that I'll be talking about the most today uh is our ability to service up uh high performance storage for databases. So, in particular here, I'll be talking a lot about SQL Server and Postgress uh just because those are two kind of bite-size databases um in a lot of instances. Um for these databases, our new ice scuzzy block service allows us to deliver submillisecond latency on a consistent basis, making this the ideal platform to support those workloads. Now, one of the features that we customers really enjoy is the ability to dynamically increase and decrease both capacity and performance in the way of throughput or IOPS as workloads change. And so you can make those type of real-time adjustments. If you've got an application that starts, you know, scaling and you need to increase the database performance to be able to handle more transactions per second, we can provision additional throughput or IOPS to meet those requirements. Uh depending on the workload type thatdatabase is really allowing customers to dial in and make sure that they're provisioning the right amount of storage resources for theirend users. Um, our products of course also in always include immutable backups and snapshots giving customers a nice integrated data protection. And then finally, we have instant cloning. And I'll talk about all of these features in a little bit more detail here. So when we talk about performance, of course, you know, being able to, you know, see performance validation and be able to do performance validation yourself is always critical. Um, I always recommend that when you're using a new solution. uh our team has put together a really thorough blog post that talks about the way thatwe do, you know, what we call kind of four corner performance testing and this is really designed to hit kind of four different types ofIO that we see from applications. So the first two of those are large block sequential reads and writes. Uh and so those are kind of you know the most amount of throughput that we would expect to see from a system. So you can see in our performance here, our uh ice sky service levels are designed to hit 5,000 uh or 5,000 megabytes per second, 5 gigs per second. Uh in our test here, we were limited uh from thesize of the virtual machine that we were using to only be able to go up to 4700 megabytes. Uh and then on the sequential write, uh we were limited by the journal replication of the database itself. Um, I really appreciated whenour performance engineers did this testing to identify what the bottleneck that they saw in the testing was and then share how we would identify that so that customers can do that in their own testing is a way to go through your own logs and to say, "Okay, I'm only going this fast. Why?" And then be able to come up with an answer to that and make sure thatanswer meets your environment that you're testing in. Um the bottom two corners there are that we hit in this performance tests are the small block uh read and write limits. So random uh 8k block sizes. Um so for our ice sky volumes that 160,000 IOPS is our max u for the service. Uh and then on the right IOPS again getting up to just over 113,000 over 113,000 over 113,000 um again limited there by the journal replication of the database itself. So again, you know, you can go through uh all of the scripts to do these performance tests and the details of how we did thedesign on this are all available in that blog post for you there for the QR code. Again, like I said, this gives you the ability to go through, test the service, make sure that it's going to meet your requirements for your applications, and be able to dial in the performance results so that you know that your applications are going to succeed uh in a cloud deployment. So for databases inside of uh from a storage perspective um it is you know one of the nice things you can always do um with databases is customize uh your database storage layout uh depending on the workload of the database. Now you know if you tear databases apart there's kind of three different categories that we can put file types into for the databases. That's the bin, that's their storage and their executables and configurations. Uh the binaries uh the actual data stores um for the uh databases themselves and then any of the redo and archive logs that databases require. Now from a simplicity perspective as a storage administrator I can create one volume and store all of the data for a single database into a single volume. And that is extremely easy to administer. It's one database, one volume. Uh it lets you know where all of yourbits are uh for an application. Um and Irecommend that for a lot of use cases, especially for smaller, you know, independent um you know, rapidly deployed databases. Um however, when you're getting into, you know, some larger databases or workloads that have a bit more complexity to them, there are some unique storage capabilities that we can unlock by splitting those database file types up into their own dedicated volumes. So actually doing a volume for bin data and logs as I've got here in my example. Um and so I'm going to go through some of these tricks that you can do as a storage administrator to just unlock some additional uh capabilities that um that might improve a lot ofuh workflows.So the first one of these of course is to optimize the storage for each one of those file types. So you know in my example here I've got my bin, my data and logs. again split that out into three volumes. And I've sized my volumes in order to meet the IO and capacity requirements of each one of those. So for my binaries, I can go with a, you know, relatively small volume capacity wise um and not give it a lot of performance. Thoseuh those binaries and executables sometimes don't do a lot of IO to them. A lot of them run inmemory. Um whereas my data file itself I want to be able to do if I'm doing an OLTP transaction I might want to do a lot of uh have a high number of IOPS so 10,000 IOPS on that volume here in my example. Um or if you've got something that's doing a lot of large block IO going with something with a higher throughput uh might meet those requirements better. And so we can dial that data volume in to match the application type thatthis database is servicing. Um, and then finally, the log files. Usually you want to have afair fairly large amount of throughput on those. There's not a lot of random IO, but there's a lot of sequential writes and then a lot of sequential reads to it. So again, knowing how those file types are going to be used and being able to match the storage for that use case free up things here. And of course, anytime in uh storage configurations where you can fine-tune the way that you're executing and using storage is going to drive down your overall storage cost. So, I'm going to be able to save money on my configuration by dialing in the resources that I need, give my databases exactly what they need to be successful, and don't overspend. So, um so that's our first kind of, you know, trick and optimization from this. The second isthat we can use uh snapshots for quick versioning. So I'm going to go through here and talk a little bit about um how snapshots work uh in Google Cloud NetApp volumes and ways that they can be implemented. Um so snapshots are a uh immutable point in time version of a volume uh inside of our system. All snapshots are always uh thinly provisioned, meaning that they only ch contain the changed data blocks that exist prior to that snapshot. So in my example here, I've got avolume. I'm going to go ahead and write three blocks worth of data to it. Blocks one, two, and three. If I were to take a snapshot on this, I would my snapshot would be thinly provisioned, meaning it would be a pointer back to all three of those data blocks. and that this would consume you know essentially consume no capacity uh inside my system. Um and so you can have of course schedules to create snapshots. We can also create snapshots manually. So if I write a fourth data block and take another manual snapshot that second snapshot will just be a pointer to the data that has changed since the previous snapshot was created. So in this case that would be pointing to just a single data block. And then if we wrote more data and then had our regularly scheduled snapshot come up, um that last snapshot would just contain the data that has changed since the previous one. Now volumes can have up to 255 snapshots uh per volume. So that gives you the ability to create a lot of uh recovery points. Um now theexistence of snapshots because they're part of the volume don't have any real impact to the volume's performance. Um and you know so that gives this ability to create these things extremely quickly uh and be able to create them with very little cost from a capacity perspective uh and to have no performance impacts just means they're a great tool to have in your toolbox. Um they're you know relatively inexpensive and you know quick and easy to do and who doesn't like the to have aroll back point uh for their data. So once we get these snapshots now we can start doing some cool things with them. Um so the first is of course you know we can use them as recovery points and I always you know stress to folks snapshots are crash consistent. Um meaning that they're not always synchronized with the application. And from a DBA perspective I always say that crash consistent is actually a little bit better than it sounds. Um and it's just because we can coincide the creation of snapshots with database operations. So you know one of the things we can do is that I to put my uh SQL server database here into a hot backup mode. Um basically freezing its IO for a moment. Um Microsoft uh SQL Server does this uh through their shadow copy service that allows it to freeze the application. um flush all of the data down to disk, pause sending any additional I/IO down to disk um and still remain inoperational throughout that operation. Um and so this gives us a really quick way to get a consistent data copy. So we can you know freeze thedatabase from an transaction perspective, create that snapshot super quickly, and then as soon as that is done, we thaw or restart the database uh getting it back into production. Now, because this is done in coordination with the database itself, we can leverage that shadow copy service and really allow us to create a nice clean consistent image of the database. Um, if I needed to do any additional um, you know, transaction replays, you could always replay those log files to get back up to the exact transaction that you want to recover to. So this is kind of a combination of the instantaneous nature of the uh snapshots within the system as well as the combination of SQL Server's ability to replay transactions and logs. So really gives you the ability to dial in do some creative things from a recovery or from a deployment perspective if you need to. um my um now of course you know ifwe ever needed to use a snapshot for recovery um again you know having multiple recovery points gives me a lot of options. Uh so you know in this case if I wanted to um you know mount the volume back to my Tuesday snapshot um we could simply you know roll back to that recovery point. Um, this is now going to make the volume appear as it looked on Tuesday, meaning thatfifth data block doesn't exist. Um, now of course that fifth data block still matters to us from a storage perspective. And so any of that data that is unique to an individual snapshot layer is preserved and kept inside of that snapshot. So again, this gives me the ability to do a lot of um, you know, different recovery options. I could recover to thatmanual snapshot, do some testing, decide, hey, you know what? I need that fifth data block. Let's recover to the full Tuesday snapshot instead. So, you've got some options there on recoveries. Um, again, this is all about, you know, being able to u manage data within the application with a database um with as much flexibility and ease of use as we need to. Um, now these recoveries of from snapshots are nearly instantaneous. Um, so there, you know, we're not moving data around. We're not having torewrite anything. Um, we're simply changing the pointers for when theaccess point of the volume is. So again, gives me the ability to do these very quickly um and without a lot of interruption to your end users. And then of course the you know the second little trick here that snapshots unlock uh snapshots are a key portion of our cloning uh volume cloning technology. Uh clones are essentially a writable version of a snapshot. Uh and so um in my example here, you know, I've got that Tuesday snapshot. I want to use that for some testing or I want to deploy that into a new instance. Um we can simply take that snapshot, create a clone for it. Um it will do that as a brand new volume. Um each has their own um write space and is readwritable going forward independently of each other. Um, and if you do this as one of our iceguzzy sand volumes, then those clones are thinly provisioned, meaning that I create that new volume without consuming any additional data. So, you know, especially in a cloud environment where you're paying 10 cents per gig uh per month of storage, being able to clone a volume and use that for a second use case and not have to pay for that initial clone can really help drive down your costs and optimize your cloud usage. Um, so snapshots, I'm a huge fan. Obviously, um, like I said, some of the advantages of them, they've got quickcreation. Each one is a full consistent image of the application volume. I can do up to 255 different recovery points. So, a lot of options uh for me from a restore perspective. Those restore operations are nearly instantaneous, so super fast to do this. And I can use cloning to save time and money. Especially if you think about, you know, databases that are very large. Um, being able to instantly spin up a, you know, a 100 terabyte database. Um, and be able to have that online in seconds. Um, really frees us up to do a lot of testing, a lot of cool DevOps ca use cases. Um, so huge advantages here. However, I always stress to folks, snapshots are not backups. Um so they are stored as part of the active volume. Um so if anything happens to that volume if I delete it uh then all the snapshots go away. Uh and so you know I always like to stress this snapshots are not backups. Uh and so um to overcome this you know kind of gotcha to snapshots um we do you know always recommend uh you know mature data protection. And here at NetApp for a very long time, we've kind of simplified data protection down to a 321 mentality. Uh meaning I want to have three copies of my data. Uh and so, you know, within our products, we allow you to do hourly, daily, weekly backups and be able to select, you know, and mix and match each of those different times so that I do have multiple backup images to choose from should I ever need to do a recovery operation.Second, we always want to store snapshots or backup data on two different media types. Um, now traditionally, you know, with on premises deployments, we're usually talking about disk and tape or disk and SSDs. Um, two physical media types. Um, of course, in the cloud, we don't get to choose the media type that ourdata is stored on. We'repaying Google to make that decision for us. Um, so I always recommend for customers to combine snapshots and backup vaults. Um, so that you've got two different images that are stored in two different ways um, for your applications. And then that one of course is we want to have one remote copy and we can achieve that with uh, cross regional backup uh, images. So the way that backups work uh in Google Cloud NetUP volumes um very similar to our snapshot capabilities uh the you know we take an initial full backup image. Um the big difference between snapshots and backups is that backups are a separate copy of your data. So we are copying the physical blocks uh off to a backup vault um that's on a lower cost storage tier but it is still a full copy. Um once we get that full initial image then we just do incremental backups from there forward. Each backup again only includes the data that has changed since the previous backup image. So they're still kind of you know efficiently provisioned. Um but they are full copies. Um now with backups we do allow up to 1,000 backup images per volume. So many more options there. Uh and like I said, backups can either be inside the same region where the volume was originally deployed or can be to a separate region. And from a recovery operations, you can either restore uh data to the original volume or off to a new volume uh just depending on your use case and what type of recovery you're doing. And then the final bit there, I always recommend that folks set up dedicated uh IM permissions uh for their backup administrators just so that you've got that uh that split between folks that are you know focused on keeping applications live in productions and folks that are keep focused on uh deploying backup and security. So two different personas there um if you need that foryour security concerns. Now, of course, backups are fully integrated part of theservice that Google offers up uh with NetApp volumes.So, um with that, I've kind of talked a little bit here, you know, about how um uh NetApp volumes, you know, helps customers to, you know, really be able to dial in to meet those database workloads. Um the first part of thatwe spoke about was our ability to u dynamically scale capacity andperformance to meet database requirements. Second part of that were the tips and tricks I talked about there from a uh snapshotting anddata recovery perspective. Um and you know of course combining all of these different strategies together can really allow you to you know dial in make sure that you'redriving costs out of your cloud solution. um while in leveraging mature rich data services um to help your teams succeed. So um with that I do have acouple of questions um that has come in here um and so the first is um are backups using the Google cloud backup service? um they are not they are using um our kind of ouruh our integrated backup system. We are working from aroadmap perspective to integrate in with that. Uh and so it'll be kind of a mix and match uh on that going forward for customers depending on your use case. Um second uh question is can snapshots and backups have uh retention policies? Um, and so yeah, um, we do allow for retention policies to be created on backup images. Uh, a retention policy would make a backup unddeletable for a certain period of time. Um, so say you've got like a HIPPA requirement or something that you've got to have that backup for seven years. Um, you can deploy that um, with us. Um, I think our maximum is 15,000 days uh, that we will retain a backup image. So yes, wecan do that on backups. Snapshots do not have cannot have retention policies. So um just for backups and it's just because the two are different. Theyservice a different use case. So um but yeah, depending on what your compliance requirements are for backups, we can meet those. Um next question is um does NetUP Volumes provide any disaster recovery services? Um yes. uh is thequick answer to that. Um I always separate out backups and disaster recovery from each other. They'reboth different use cases. Um they have a lot of similarities uh but theyare two different things. So from a disaster recovery perspective with NetApp volumes uh you know ourfirst option of course is we can use backups as disaster recovery images. If you do a cross regional backup and a region goes away for some reason, um you'd have that backup image that you could use to do that recovery, um that would be a you know a rather you know there'd be a long you know long outage inmost of those cases. Um we also allow for cross regional replication uh which is essentially a uh asynchronous replication as frequently is every 10 minutes u being able to replicate data from one volume off to another volume uh usually in a different cloud region. So very similar to backups but it creates a uh kind of arecovery image that is hot and ready to be used. So, um, you know, again, it depends on youruh, RPO and RTO, uh, requirements, um, for which one of those we'd recommend and help you configure. And then last question we've got in here is on the um, ability to change volume sizes and uh, volume performance. What's the frequency and is there atimeout required with that? Um, so yeah, you can change uh volume capacity and volume performances on the fly. You can make those changes any time in the interface. Um, actually I believe you can only make you can only change them once every two minutes. So I think there is a limit on that. Um, and we do that just so that there's time for it to synchronize. Um, but yeah, that gives you the ability to make thosecapacity and performance changes, you know, as you need to. Um, so if you've got a volume and you start running out of space on it, you can increase the capacity. You've got an application that you know you'vegot a Super Bowl ad going on your OLTP database uh that's going to be taking a lot of transactions when that commercial hits, you can increase the IO uh requirements of that volume uh on the fly and then you know after the words dial it back down if you need to. So yeah, we can do those real time changes for you um as your workloads change. All right. So, as far as, you know, kind of next steps for folks, um, so we do have a ton of documentation, uh, that's available at Google Cloud doc Google.cloud.cometappvolumes. Um, so you can get started immediately with the service in there. Um, we also have an extensive library of Google Cloud Skills Boost Labs built out. Um, each one of these is a hands-on lab designed to teach you,know, a handful of subjects at a time. Um, they're built within a Google Cloud uhlab environment. So, it's not using your own resources. Um, I always say I love being able to be in a sandbox environment where you can explore things uh test them yourself, break them yourself, uh, as that's how I usually tend to learn. Um, so great resource for folks that are looking to uplevel uh theiruh Google Cloud skills. Um, and then of course we, you know, we'd love to spend deeper time with you, understand your requirements, what business challenges you're working on, and see how NetApp and Google together can help you meet your business requirements. Uh, you can go to netapp.com/googlecloudcont and schedule a time with our team. We'd love to sit down with you, uh, learn about your business and see how we can help you succeed. And with that, I want to say thank you. I appreciate everyone's attention today. U, thank you for the questions. Um, always great to have that conversation and look forward
Learn about Enterprise SAN with Google Cloud NetApp Volumes and hear how to crush latency, control costs, and seamlessly extend from on-prem with no compromises.