Finally. Sorry to uh keep you waiting for about 10 minutes. Uh we'll make it fast. Uh we want to show you some pretty cool demos. So you may have to apologize for it. So we're going to um present about Google Cloud Net App Volumes. And I also have Sean Dington from Google. He's going to share his um expertise and some of the pretty cool technical details. We actually added more slides so you're going to miss it. Uh where? >> Yeah, thanks. And so um we're going to I wanted to talk about today was a little bit about how you can operationalize NetApp volumes in Google Cloud. For those that um may not have heard, we announced on August 24th uh Google Cloud NetApp volumes. Uh this is actually something well um that we've extended our partnership for the last 5 years. Uh so Netup has had a managed service available in Google cloud uh cloud volume service for the last four years as well as cloud volumes ontap. Uh cloud volumes ontap is a customer-managed service uh that is available in every Google cloud region. The announcement we made on the 24th was actually extending this relationship to a firstparty offering. So now this is actually a Google managed service that you can procure directly from Google uh just as you would any other storage service whether that be cloud storage or persistent disc it doesn't matter it's all now an offering directly available through you instead of going through the marketplace. So what GA was mentioning we had a couple slides to set up uh tomorrow afternoon at 4:30 there is a session where we do have demos that are actually recorded as well as live. So if this happens again we know that we'll be able to play the recorded demos uh for you. Um but it is something that actually goes into a variety of capabilities that many of you are familiar withNetApp on tap on premises for the last few decades right but this is now something that our partnership has extended to be not only more detailed in terms of being in a Google first party service but also there are more capabilities that you can now take advantage of uh compared to NetApp cloud volume service. So many of the things that you're used to if you're using CVS today the cloud volume service uh it's a netapp managed service. So from that perspective, it feels very much the same. In the Google cloud console, you can select net volumes, but now the difference is it's going to be a Google managed service, which gives you additional capabilities with data protection. As an example, we're rolling out the capabilities for backup of your particular volumes. Uh this month, which was not previously available, other things like regional replication is also available for all the different service levels. And the multi-protocol file system support is one of the main drivers for this relationship from the beginning in that Google we don't have a first-party SMB managed storage offering. Customers have been rolling their own installing Windows storage server using NetApp cloud volume service or CVO as well. Um but this actually now further integrates that capability of SMB NFS or even a dual protocol to the same volume in a managed service. A couple other things that GAN is going to talk in more detail about so I'm just going to touch on briefly but the cloud experience this is something where it's a different mentality than everybody that's managing filers on premises in that yes you worry about capacity performance protection levels but you don't have to worry about cache hits you don't have to worry about how many spindles to SSD and MVME ratios you have in a particular filer so the sizing of what you actually go through to deploy storage for a given service is now in a consumption model where you say I need a couple terabytes or I need a 100 terabytes. I need this performance level and this protection level and you're done. We do all the hard work on the back end. Uh as well the other thing from the um the horizontals on the bottom that you see there, these are things that we integrate with other storage services like cloud storage or file store persistent disk in terms of being able to see the audit logs for capabilities that admins take having that be centralized into all the Google cloud products that you're using. This is now something that with net volumes being a Google offering, this is something that we now integrate directly into there. So if you're using uh VMs or if you're running Google Cloud VMware engine and you're logging VMware alerts and storage alerts and everything through the console, net volume shows up there as well. Same thing with the CMC. This is something that now you can use a Google key management. You can rotate the keys yourself. uh but also from a billing perspective this is now just build based upon the consumption that you provision for storage pools and you get build on a monthly basis. So these are the types of things that really help you operationalize net app volumes and Google cloud. So from a Google cloud perspective we do a number of things to really help you accelerate and transform your business as quickly as possible. From the operational experience this is one of the things that we focus on to make it easy uh whether it be uh identity and access management services. So different people in your organization can have different roles responsibilities for uh managing the services that you're running in Google Cloud. And this can be anything from SAP to different applications that you're going to run on GKE our Kubernetes engine or just our compute engine uh or even Google Cloud VMware engine which is a managed Google Cloud service for VMware and which we have a lot of joint success um uh to date uh with that. From a security privacy perspective, this is one of the things that we take obviously very seriously. Um your data is your data. You manage projects. You manage how your applications are deployed, what the network topology is, which projects have access to different projects in your organization as well as from a perspective of like Mandant. So if you want to look at higher level security services from um uh threat monitoring, we have services around Mandant. We also have like cloud armor for firewalls. There's a lot of different ways you can protect the services that are running in Google Cloud. Some of which are can be the same way like Paloto network firewalls. They can be the same way as what you're doing on premises, but a lot of times it can actually be a managed service. So again, trying to make it easy to operate and make your business run more quickly. From a reliability perspective, this is one of the things uh for GKE pods, right? We can spin up 40,000 pods in a matter of minutes, which is actually uh you know, it points to the cost optimization and performance optimization side of things where you can actually spin up and spin down resources really easily, whether it be Kubernetes or compute. Uh but this is also one of the things that from a cost optimization standpoint like with NetApp volumes we give you different price points for capacity performance availability and you get to choose what's the right service level. Similarly on the compute side you can choose the right uh CPU be number of BCPUs memory will it be uh Nshaped or C-shape uh etc. From a design perspective this is one of the things that you have the flexibility to do a number of things based upon what you're trying to do. So we have templates, we have recommendations for different architectures in our cloud architecture center that help you get started if this is your first time or first application moving to the cloud or whether you're a veteran uh deploying many applications. We have a lot of design principles and best practices uh around that. And so this is just a little bit more on the operational side and I'm going to turn over Guna in just a minute here. Um, but this is one of the things that from a managing capacity of quota, I kind of I wanted to call this one out because this is something that's different from the on- premises storage deployment that we're all used to over the last few decades and that you don't have to have a three-year horizon about planning and guessing and looking at that crystal ball and hoping to get it right. Uh, this is something that you can plan and consume on a monthly basis or even on a daily or weekly basis that you can spin up and spin down resources. There is a little bit of planning that's needed just from a capacity and quota in terms of what region you're looking at and what services you're looking to consume whether it be compute or storage that you do need to work with the account team or go through like a CCA capacity planning um quota exception and quota request for that but it's very easy to spin up and spin down netup capacity in Google cloud. This is available by the way in 14 regions today. It's a generally available service that you can go into the cloud console and you can enable the net volumes API and you can begin to provision storage immediately. If you start to need hundreds of terabytes or pabytes of capacity, then obviously that you support you submit quota requests and we actually can plan for that to make sure that we can support your uh changing business need. So with that let me turn it over to Guna to talk a little about on the operational side that this is uh with Net volumes are some new capabilities that we have. >> Thanks Sean. So just to get a feel for the room how many of you kind of um are cloud architects the storage architects? Perfect. Thank you. And then how many of you are using GCP as your cloud provider today?Good. Thank you. So um we appreciate your help. We're going to talk in detail. It's kind of a technical session. Um, so we'll get you in some details. Hopefully, uh, that satisfies your curiosity and requirements, but we have a small room. So, stop anytime. Uh, both Sean and I, we also have colleagues who worked with us on this project. You know, we can get into any level of detail you may choose. So, there are two big uh things we want to get to you. If you're using a GCP cloud or you're using a NetApp, you're an existing NetApp customer or you're existing GCP customer with Google Cloud NetApp volumes, you can leverage your expertise, the operational expertise of cloud GCP and also the familiarity with the NetApp commands and the tool chain. So I'll tell you about uh you know if some of you guys have been using NetApp for a while, all of you are familiar with the ONAP API. Most of you may use it in your scripts. You may actually use your own playbooks these days. So, Google Cloud Net App Volumes is actually offered volume as a service. When thecritical aspect one what Sean kind of alluded to, we are simplifying your life is you don't manage onap anymore. You actually still get the flex wall, the onap volume. The flex wall you'll actually get as a volume. And I'm going to kind of click through a couple of slides to show you the flexibility that you have with the flex walls. You don't need to worry about aggregates. You don't need to worry about snap relationships to another onap cluster. You don't need to worry about lifts like pretty much like most of these things are kind of become transparent to you. So this is like a big migration of SS from your mind in your mind like when you're using Google Cloud Net App volumes your operations are like massively simplified if nothing else you don't need to worry about patch management you know our SR deals with it right and obviously theupgrades from one version to another so now you know especially when you're operating anything at scale uh even on your on-prem um uh deployments You know you may be using unified manager or system manager you know similarly we do have if you go to console.cloud.google.comgoogle.com you actually get the cloud console netapp volumes shows up as a GCP resource right again some of you are familiar with GCP that's at another GCP resource at another Google cloud resource nothing special about it nothing proprietary about on tap you can use everything that you treat a GCP resource and apply the same thing to Google cloudnet app volumes so for example like you know let's assume it works G-Cloud CLI if you're like me like you know yeah you say G-Cloud say G-Cloud say G-Cloud that's what I wanted to show some demos but I'm going to say it so if you say G-Cloud Net app now you get like all the commands that you can use on G Google Cloud Net app volumes it's pretty straightforward andespecially if you're actually using Swagger API and running everything through a REST API um you know I think in your slides you'll also get an updated URI file where everything you get you can uh we also provide multiple SDKs again those of you if you're using already GCP you have like a Python SDK so you get the client libraries where you can manage the volumes you can create pools you can create volumes and you can set the performance levels and you can obviously change the capacity and then in addition again please do come to our session tomorrow at 4:30 30 you can basically get the snapshots cross region replication backups etc. Right? All the capabilities that you know how to use a flex wall, you will be able to consume it and like you know I want to almost say like zero operational overhead to you but minimal operational overhead and everything can be automated and especially uh you know this is a repeat user request to us. Uh you know those of you have been using playbooks especially the terapform playbooks uh that's something that we're going to introduce uhlater this quarter. So again just to give you an idea about uh the G-Cloud net app what I was telling you earlier these are all like you know bunch of subcategories. So the first thing is if you're using uh NetApp today for SMB to serve your Windows applications, you can actually use your onprem active directory. If you actually have like you know you still want to continue on your data center and like you know use uh GCP for your cloud and if you have your uh ADLB groups coming from your active directory you can extend the same AD to the cloud but you can also use Google managed AD right so you can also say you know what I'm not only the storage you're also u want to use a fully managed uh active directory service we support that too you can basically configure everything through G-Cloud Net App and then um from a key management perspective you know all the data uh that's stored in Google Cloud Net app volumes just like you know the way you would do with onrem on tap is all data at rest encrypted so you don't need to maintain any of your key management solution it comes with it you don't need to worry about it but should you have some constraints or compliance requirements where you want to integrate with your own external K management solution, right? So, we do integrate NetApp volumes with uh Google cloud uh again fully managed KMS, cloud KMS and we you can actually maintain your own uh customer managed encryption keys. So, that gives you a lot of flexibility. you can leverage with your current practices whatever you have but if you have more exquisite resources or uh constraints you can use it without any constraints and you can all you can do all these things again through API through CLI and you know very soon through terapform right now that's the operational uh simplicity we want to kind of help you on um thenthe locations uh so one of the things that you see you know kind of Sean alluded to is um uh compared to on-prem you know where you may maintain couple of data centers right with cloud you know I think GCP has like 37 now or expanding>> 38 >> 38 see like day by day we're expanding so you're like 38 regions like pretty much anywhere in the world we're already there in 14 of them so now when you think about your snap mirror relationships between your uh your on-prem data centers uh you really have like 14 of these things you can based on your where your customers are your developers are or maybe in your case where your data is right you have a lot of flexibility on how you maintain it uh withuh the CR relationships. So what it really means is you know when you create a resource you can actually choose a location in the cloud context. You know we always call it region right like you know so you can actually have a lot of flexibility where you can create those volumes and uhand also build this DR relationship with apier volume on the other region and then the there are the typical CR operations like you know so youcan actually say um uh you can create volumes you can delete volumes and kind of Sean alluded to this I just want to double click One more time when you adopt cloud you know it kind of reduces theprobabilities or the forecast you need to do because resources are elastic you have a lot of flexibility. So for example if your customer wants or your application developer wants a terabyte volume you don't need to create a terabyte volume right today you can actually grow it. So,thereason why this kind of flexibility is very critical is as your application needs change, you can dynamically changethe infrastructure and you know you're not really calling anybody right you know between the behind the doors you know we take care of theoperationsum I talked about storage pools and volumesone other big thing is like you know this is another common thing right when I migrate to cloud am I losing control like you know Do you get to see who's doing what, who is exercising what and you know especially some applications if you're colloccating multiple applications do you have control over the performance you're delivering to your applications right so uh we integrate netup volumes is fully integrated with the cloud monitoring that GCP offers one advantage you know our cloud solution architect head was telling me as was preparing for it is it's a pretty very good thing when you think about a when you're troubleshooting an issue and application performance issue I mean your application res storage is the bottleneck you know most of the time the issue resides somewhere else because we are integrated into cloud monitoring you can see the whole stack you can actually bring in a VM and you can look at the network resource you can now look at the storage resource everything together and you get to see what's happening for the application stack and just look at the power of what we can do when you see everything through the single pane of management ment right so net app volumes get to see it you know I mean here I'm giving you some examples of uh real life uh real time scaling the per scaling as it's happening and also show you like you know this kind of a spike you know maybe there is a little spike here how throughput and IOPS are consistently deliveredso because most of the time you know you may have your own internal customers especially the application teams uh you know they would have committed or you would have committed some performance levels you can actually show them you know and every 5 minutes we take uh u the data is streamed you can think about forecast performance forecasting if you choose you can also build lot of automation in terms of alerts the best part is everything is through cloud monitoring so you're again once you adopt Google cloud you're not really learning anything new about net app volumes you pretty much leverage you know you learn one and once and use it multiple times and same thing with the loggingSo especially when you create this resources and expose to your customers, you really want to see who's doing it and it's too small a font for you to see. You get to see like, you know, pretty good detail including the user who is performed an operation, right? Think of this way. You're the cloud architect or the storage admin. You created a storage pool and gave it to your application owners. Now people are remember I told you like they can resize the volumes as they choose and every one of those actions will be logged. So you get to know exactly who changed what to when you know but it could be used for auditing or it could be used for chargeback based on your cost modeling you know with your in your teams. The other big challenge is the quotas u especially I hear u a lot from typical on-prem storage uh architects adopting cloud is in the on-prem world you know you have absolute control you know you have an on top cluster you know what how much capacity you provisioned you know what storage efficiency you're getting you have a good feel for it but now when you go to the cloud you know you really cannot put controls remember like I told you like you're giving the flexibility back to the user because you want it self-service right you don't want anybody to slow down because of the infrastructureyou don't lose control because you can actually enforce quotas so there are two sets of quotas that we use the quotas as a resource for example uh you know you put a cap that uh you cannot create more than 25 terabytes uh in a region at that time even if your application developers or you know your downstream teams want to increase they won't pass it you have control on submitting the request back to the cloud provider. I mean again this will happen very quickly not like a 3-month period but you have like you know absolute control over how to increase the quotas and then you know in certain cases there are limits so that will help you when you design your cloud solution you know like what are the limits you're dealing with so you can size number of pools or x number of regions uh that you can use it to uh the other big challenge is like know that the traffic right like you know the first of all you know IO traffic is always bursty in nature so most of the time you may have peak periods I'm not talking about like know the dailysequence you know 8 to 10 uh working hours there may be a quarter end you know your finance team wakes up and like know they're super busy creating the reports for your xx staff so there are two things like net app volumes today supports three service levels you know everything we pended you know for a given amount of capacity what is the performance you get right we support three service levels one advantage we give especially for uh two of those service levels is you can go back and forth right so when you know uh those short bursts of time in calendar time where you really need highest performance you can actually pretty much double the performance without moving the data without reconfiguring your applications and more importantly you don't need to even remount count the volumes the same volume again if you're again familiar with the onap QoS levels we basically inherit some of those capabilities and expose through the service levels and give you the flexibility to move the volume back and forth between two QoS policies >> and thiswas actually one of the live demos that we were going to show it was a puppy video running on premium performance level and then dynamically reassigning that to a extreme storage pool the video would still be playing and then move it back. So not only is it non-disruptive from an operational perspective from a not you know he said you kind of mentioned you don't have to remount the volume you don't have to interrupt playing the video when you make that transition between two different service levels between the premium and extreme. Yeah, the key thing no service disruption basically the key thing is completely transparent applications in the case of puppy video like you know you enjoy the same experienceum and then the security um one of the critical things about security is uhthis is where now I'm going to play to the persona you as a cloud architect storage admin there are two three different personas you want to deal with you know the persona that you are providing a resource to your application developers or teams and what you allow the teams to do right I talked about you know the resizing volumes etc what the teams can do but you um you know again you do every one of these things through Google I am you don't need to learn anything Google IM that identity and access management you create roles we actually right off the bat we get you two default roles somebody who has the owner and who has the admin and you can create custom roles. So you can tell oh that particular team you want to give them the flexibility to configure snapshots right but in cases you know you may want you want to make sure that like you know people because that's actually helps you with your backup policies for example right so you don't want somebody to misconfigure it and then you know blame you later or you know in fact seek your help later so you can choose what operations you want to allow uh a particular Google AM uh user to And then the security privacy I talked about you know theCMC u so you get to see like all the flexibility there from a reliability perspective you know I'm going to go light on this slide because I expect all of you as onap users to know like you know pretty much anything and everything that on does um what I want to say is we support everything on does in terms of reliability youknow, you get the same snapshots, you get instantaneous clones, you get um a snap mirror, snap mirror to cloud because we're offering you a cloud resource at the volume level. You don't need to worry about how many peers are peering you need to do. We will transparently take care of you. And then uhlike in a week or so, we'll be rolling out integrated backup. So the way to think about this is you know you have a flex wall you take snapshots say x percentage of the volume for local snapshots but you know forDR purposes or for uh longevity purposes youwant thesnapshots to be retained for a longer time right because you don't want to spend you know x amount of dollars for or high cost dollars for storing local snapshots I mean that has a purpose But some cases you want to do it. So we going to we are integrate we are uh in public preview. Um if you're interested you know please see me later. We would like you to sign you up as a public preview customer. Um what you will do with this is you take the same snapshots and now store it onto the Google cloud storage object storage. So you're now freeing up the volume for storing snapshots. So you can now use the volume for primary data. And more importantly if you really have snapshots for a longer time because again could be for a myriad of reasons you can you want to retain the snapshots for a long time right so now you can actually put all the snapshots into an object storage you can keep it for three years you know you have a lot of flexibility on how you want to do it you don't need to go for another third party backup solution the backup comes integrated into it >> and there'sone difference so for those of you that are using net filers on premises there's absolutely the opportunity to do a cloudbackup service from onrem to Google cloud right where you're managing the cloud storage bucket the storage class the location the region multi-reion dual region options but with this and this is actually a managed service so now you're not managing another bucket as a target for the backups was describing you just select the source and destination location and we do everything on the back end so it makes it much easier to consume and use on a perv volume basis so if you have a thousand volumes you want to protect 50 you can back up 50 and you can protect the rest of them however you want to whether it be locally and or remotely with like replication. So all these options for managing the storage services is on a perv volume basis. So it's a little bit different way about thinking about that granularity. Thank you Sean. That'sa very critical point like you're not configuring backup for the whole cluster. You're actually configuring backup at the most least I mean the finest granularity which is a volume and you can create multiple policies. So u all in all I think thekey takeaway for this is if you know GCP cloud you don't need to learn about NetApp volumes it is at another resource uh since you most of you come from NetApp background or on top background we may call the same thing different name but it's the same onap capabilities that's powering NetApp volume loops so next few minutes Sean like know if you can go through the slides So we just had a few other options here. You know, I talked a little about kind of the hybrid deployment models. So we did want to kind of put into perspective about where people can be on their journey today and where they can go in the future because not everybody's going to move 100% to the cloud from day one. Um and so if you look at this, you know, there are different options for um thehybrid model. This is something you can do across obviously multiple data centers that you have and then obvious and the leverage our regions as a third option depending upon how you want to protect your data where you want to locate it. Um we were going to show you a few again demos but uh we're not uh and so this is actually something that uh again coming back to the two options that you have for cloud volumes ontap which is where you have all the onap APIs that you're used to for your on-rem filers and that's definitely an option that you can deploy today and we have lots of customers doing this. Um, I'll give you an example for I don't want to say simple because this is not a simple file sharing solution. This is a um a healthcare organization that had 83 different locations throughout the world and they had hundreds of filers. They're all running seven mode reaching their end of life. So what they did was they actually looked at the largest locations of where their users are. This is a combination of ontap with uh global file cache or now edge cache. Uh this is actually something that we now have deployed in six regions. Two in the US, two in AMIA and two in APAC that are replicating and doing CVO backups to each other. And all of those are doing Windows file serving to users throughout the world. Their largest data centers go directly to the cloud because the performance, the latency is great. Lots of smaller ones, different locations, small um cities or just a handful of users, they actually deploy global file cache at the edge. But CVO is maintaining that global file consistency across each of those pairs of regions. So the users have a local Windows file sharing experience. But from an administration perspective, you as storage admins and cloud architects are now thinking about file serving in a cloud perspective. Right? So this is something you can do today. It's not a managed service. You're doing it yourself, but it's most flexible, I would say. Um, but if you do want to go down to managed service, you can do something similar where you have on premises and cloud-based resources for something like file serving as an example.Um, and then you know this is uh well I just reiterated this one. So yeah. Um, so let's take um I guess we have a few minutes left. Um, do you want to take some questions? Gonna >> Yeah. >> Yeah. >> Yeah.So depending on where you're coming from, so and from a uh application deployment perspective, you know, you can lift and transition applications or lift and shift applications to Google cloud. So if you look at the lefth hand side, uh GCVE on the bottom is our Google cloud VMware engine. This is a man a Google managed VMware environment for you to deploy VMware as you would, right? You still use Vsenter. Uh we're going to support NetUP volumes as NFS data stores to augment the block storage capabilities that come with that. uh and that'll be planned this quarter uh because as you deploy a number of vCPUs in memory, you get block storage with each node as you scale out. Um however, there can be data intensive VMs that need more storage and so you now have NetApp volumes as an alternative to augment that block storage capacity. So you can keep the premium low latency database VMs on block storage and use uh NetApp volumes for tier 2, tier three volumes uh data stores I should say specifically. Um, we integrate with active directory and LDAP capabilities are going to mention whether it be AD your own AD on premises your AD in Google cloud or our managed AD environment you can do that dual protocol environment obviously supported thereas well uh NFS v3 41 all the things you know and love about NetApp on premises you have that capability in the cloud if you're going to move to more of the uh transform you can actually deploy this in Kubernetes as well uh so we have the Trident driver today for GKE pods and clusters where you can provision as small as a 100 gig volume to a pod and because this is an NFS volume it now supports a multi-riter use cases as well as persistent storage across any number of pods and clusters uh in a given region. So depending on where you are in your journey you can start from where you are and get to where you want to be over number of years. You don't have to go to GKE obviously at some point uh you can stay with VMware or comput engine but that flexibility of how you're going to use NFS and SMB storage services there across the portfolio. >> Yeah. >> Yeah. >> Yeah. Important thing is it'sthe service is fully available. Um you know we g in August you can actually use it today. >> Yeah. And I'll leave this uh last slide here for additional sessions. and the 17602 the second one from the top there that's the one that G and I will be doing tomorrow at 4:30 where hopefully the demos will work and uh we we'llgo more in depth about snapshots and replication and actually how you actually experience that through the cloud console andor API >> APIs of course >> any questions we do have a few minutes left I know we got started late but definitely wanted to uh maybe have five minutes or so Yes, moderate the chief pile of EPI subs curious like that you create a I don't like a script or something like thattalks in the king file the par three space on the volume on periodic resizes it so it all yeah sothe question is uhbasically I think he's talking about like can we dynamically ly resize the volume. I talked about know growing versus grow and shrink. You know typical storage ad means always think about first of all shrink is you always think about like you can always shrink a file system for the capacity that's not used. You don't want to lose any data. So it's a uh it's just not a volume semantic at the time. It's actually a file shared semantic. You got to be careful. Um from a growing perspective assuming that's not the case you can resize this. the service itself doesn't do but you can actually write a wrapper on the top so that you can resize uh the volume to your current needs both grow and shrink there are some constraints kind of Sean already talked about the minimum volume size is 100 gig you cannot really go less than 100 gig the max volume size is 100 tib so you cannot go more than that um >> and theother the other concept it's a little bit different than the on premises environment is with net volumes you have the notion of a storage pool. >> Yeah. >> Yeah. >> Yeah. >> That storage pool is a given size up to 10 pabytes and that storage pool is assigned to one of the service levels standard pre or extreme that you then create volumes and assign those to different storage pools. So if you want to automate a growth of a volume you can do that but it can't exceed the size of the storage pool. So there are a few constraints like that but that's the that's one of the benefits of cloud right expanding and contracting as you need resources. >> Yeah. And you can actually resize the pool if you want to increase the pool size.>> But all the Kubernetes environment, container environment, that would be a way to off. >> Yeah, exactly. >> Yep. It's a way to optimize the profiling GKE. Yes. >> Any other questions? Well,>> thank you. Sorry for holding your time back u with the logistics issues, but hope to see you in our next session. Great. Thank you everybody. Thank you.