Technical workflow for hybrid cloud solutions in manufacturing and logistics
In the evolving landscape of logistics and manufacturing, digital transformation is key to staying competitive. This technical blog post explores a hybrid cloud solution on AWS developed through the collaboration of SVA. The solution addresses industry-specific challenges for predictive maintenance tasks.
Integral to this solution is cloud bursting—a feature that’s crucial for manufacturing companies facing variable workloads, especially during peak demand periods. Cloud bursting allows these companies to seamlessly scale their computational and storage resources by extending their on-premises infrastructure to the cloud. This elasticity ensures that manufacturing operations remain uninterrupted and efficient, even under fluctuating demand. By using this hybrid cloud approach, businesses can maintain their core operations on premises—which is stable and predictable—while leveraging the cloud for additional resources as needed. This approach not only optimizes costs by avoiding overprovisioning but also keeps operations agile and scalable, which is vital in the modern manufacturing sector.
Establishing the hybrid connection
The first step involves setting up a connection between the on-premises environment and the AWS Cloud. The configuration must permit communication between both environments.
SVM peering and the FlexCache volume
Next, you need to establish SVM (storage virtual machine) peering between the two NetApp® ONTAP® systems, as detailed in the NetApp ONTAP documentation. Then you need to create a NetApp FlexCache® volume, a task explained in the documentation. To carry out these steps, you’ll need access to the Amazon FSx for NetApp ONTAP command-line interface (CLI).
Mounting the volume to the GPU instance
After these configurations, you can mount the volume with the on-premises data to the GPU-accelerated instance in AWS. This step is necessary for applying GPU capabilities to machine learning tasks.
The NetApp DataOps Toolkit
To integrate NetApp DataOps Toolkit features, consult the guide for installing and configuring the NetApp DataOps Toolkit for traditional/Kubernetes setups, which is available on GitHub. If you aren’t a storage expert, this toolkit is essential for using the data functionalities of FSx for ONTAP efficiently.
After the cloud architect configures the environment, data scientists have several options for their working environment, including Jupyter Notebooks on Elastic Cloud Compute (EC2) instances, AWS SageMaker notebooks, or machine learning operations (MLOps) platforms.
Use case: Jet engines dataset
In this scenario, data scientists work on a predictive maintenance task that uses jet engine telemetry data collected on an on-premises system. The goal is to develop a machine learning model that predicts the remaining useful life (RUL) of these engines (until they need an overhaul) and take advantage of fast GPU access.
Leveraging the NetApp DataOps Toolkit
The DataOps Toolkit is used to import and manage datasets efficiently:
Data access and management
The solution offers several commands for managing and accessing data:
Data preparation and sharing
To prepare and share data, you can use these features:
The joint solution from SVA & NetApp harnesses the strengths of both on-premises infrastructure and public cloud platforms, bridging a crucial gap. On-premises setups offer low latency and robust data governance, whereas public cloud platforms provide scalability and advanced capabilities. This hybrid cloud approach allows you to manage resources seamlessly, so manufacturing companies can use machine learning models, process large datasets in real time, and deploy applications with agility.
SVA and NetApp technologies create value by:
For more information, visit Empowering the logistics industry with Data-Driven Platforms.
Thank you to everyone who contributed to this blog. A special acknowledgment goes to our partner, SVA, for their invaluable contributions. Further, special thanks to my colleague, Max Amende, for co-authoring this blog.
Tilman joined NetApp in 2018 where he now holds the role of Cloud Lead Automotive. Here, Tilman is the technical lead for emerging technology in the automotive industry and responsible for developing and implementing service architectures for emerging use cases such as Product Lifecycle Management, Machine Learning and Autonomous Driving. At NetApp, Tilman can pursue his passion and support global automotive companies in establishing an enterprise-proven hybrid cloud data layer for their most innovative workloads.