Hi everyone, welcome to our session AI in practice. I'm Rick Huang, data scientist and technical marketing engineer at ADAP. Today I'm joined by Davidid from Nvidia. Here's the agenda for today. We'll first look at four industry verticals for leading AI use cases. Then I'll answer the question on your mind. Why NetApp for AI? will look at high level view of AI solutions in recommener systems, computer vision, AI in the cloud and conversational AI. In healthcare, data is increasing at an exponential rate with omniresent sensors, wearables, mobile applications, and the digitization of healthcare records including medical imaging records such as radiology and pathology. Distributed learning is one way to mitigate the challenge of having sufficient data to train AI models. And federated learning may provide a solution for building robust AI models from diverse data across industries, patient types, and countries. We'll cover two healthcare solutions in the following sections. For FSI, there are an abundance of research papers to predict stock volatility using vocal verbal cues with multimodal deep regression and other models. Moreover, AI augmented global customer support centers can provide exceptional user experience via chatbots, audio video conferencing, real-time customer and support agent sentiment analytics for escalation, dispute resolution, and objective long-term employee performance evaluation.Autonomous vehicles have incredible potential to improve roadway safety and efficiency.Deep neuronet networks and convolutional neuronet networks enable this autonomy. Each level of autonomy from level two and above requires the development of multiple AI models each having complex neuronet network architectures and the need for a very large training data set. I listed an example from a survey car. It will generate two pabytes of data a year because of the camera resolution and you also need 5 million to 8 million images to train the network as a best practice. We can help our automotive customers satisfy the data and accelerated computing requirements for achieving higher levels of autonomy. I'll cover autonomous driving lane detection solution in the computer vision section. We have published a technical report on customer journey monetizationin which you'll see examples on combining different data types and streams for better customer segmentation and behavior prediction. From a data scientist's perspective, compliance and access to data can sometimes be tedious and cumbersome. As a result, shuttle it and shuttle AI is a big problem. We're seeing around 80% of a data scientist time is spent on manually cving data from silos, production and prototyping environment. these tasks is time consuming and that's a reason why 87% of AI models don't make it into production. How convenient it is if an experiment code snippet can be versioned with the associated training data set. What would you say if there is a Jupyter Lab workspace with builtin rapid prototyping and traceability? Here's how NetApp AI can help in your journey to realizing AI in business. We help customers create end-to-end pipelines with the best machine learning platform. NetApp enables seamless hybrid cloud training and inferencing. I'll touch up on that in recommener system architecture.Customers can start small with a PC and scale efficiently on premises and or in the cloud. You get a faster ROI from your AI investments by eliminating hours and days of configuration. You can use our Anible playbook, Jupiter notebook examples, or other good stuff in the NetLab data ops toolkit. You can click on each link to see our related documents online. As a data scientist at NetApp, I really appreciate how our technologies resolve pain points in different aspects. Our data ops toolkit is used as a key accelerator in many of the solutions I'll present today. It is a Python library that makes it simple for developers, data scientists, DevOps engineers, and data engineers to perform various data management tasks. Let me show you a movie recommener from Nvidia. >> Hi everyone, I am Dane presenting on behalf of Shashank and Win. With this demo, we trying to demonstrate the power of GPUs for inference. To that end, we are using a DLR model being served using Triton for movie recommendations. We are running a CPU and a GPU model on two systems, each having a V100 GPU and a Zenon CPU. In our testing, the CPU model is able to serve four users versus 250 users being served by a GPU. To be more specific, CPU is able to perform about 4,000 inferences per second, while the GPU is performing about 260,000 inferences. So now that we have some context, let's discuss what we are looking at. Each cell in this grid [clears throat] represents a user and the cell is occupied by the poster of the movie with the highest conference rating for each user. On the right, you can see the top five recommendations for the highlighted user. This should give some context for the taste profile of the user. Up till now we have been watching a CPU serve the recommendations. So now let's switch gears and have a look at a V 100 in action. As you can see that is a massive 60 time speed up. Let's continue to watch it. Thank you for your time. Thank you for the wonderful demo. NVIDIA mering for recommener systems provide fast feature engineering and prep-processing for operators that are common for recommendation data sets including models like DRIM. However, there are some design challenges and bottlenecks in recommener systems. The most critical three are data sparity, scalability and co-start. To resolve these, let's look at NetApp data pipeline for recommener systems. Commercial recommenders are trained on huge data sets, often several terabytes in scale with millions of users and products from which these systems make recommendations.This architecture needs high performance storage, compute, and network. NetApp AFF provides all the flexibility that ONAB delivers to keep up with fast networking capabilities and the high IO demands of GPU enabled training and inferencing clusters. Storage grid object store can be your code data tier as a unified data lakeink to house different streams of data with specific life cycle management policies. Cloud volumes on tap is a highly available storage solution on public clouds that supports grow as you go fileshares. We have cloud sync for rapid and secure data synchronization. Whether you need to transfer files between on premises fileshares, storage grid, untap s3 or public cloud services like azure neta files, azure blob, AWS, fsx, s3, google cloud storage or IBM cloud object storage, cloud sync moves the files where you need them quickly and securely. Our customers are currently modernizing their workflow and data pipelines with NetApp's hybrid cloud technologies. One of the largest American retail corporations, a NetApp customer, is leveraging AI with natural language processing and multimodal recommendations, allowing their customers to interact with products virtually using augmented reality. We will discuss customer journey monetization in the following clickthrough rate prediction section. Next, let's transition to our AI solutions in computer vision. In this talk, we'll go through a coid9 lung CT segmentation AI augmented radiology workflow with transfer learning. The chest computed tomography images can aid the monitoring of the progression of the infection during clinical treatment. Fast analysis of CT images will support rapid triage especially critical in resource restricted environments. NetApp and SFL Scientific have jointly developed an end-toend CT segmentation workflow using Nvidia Claraara and NetApp data ops toolkit on a DGX system. The complete deep learning solution architecture is on the following slide. We also have a flexible approach for face mask usage monitoring in healthcare settings. I'll discuss the challenges and how we resolve them. Remember the challenges in automotive vertigo to address it for enterprise data science teams. We have a distributed training solution for autonomous driving lane detection. We work with one of the NetApp AI partner network independent software vendors run AAI for this use case leveraging the best of Azure cloud Horovo distributed training and run AI's orchestration platform. This deep learning computer vision solution accomplished the following. CT image data processing rapid prototyping via snapshots of models with data transfer learning to fine-tune model parameters achieving a 3% dice score increase in already state-of-the-art 3D SEC Resnet model. It could be useful in various computer vision healthcare scenarios where patient privacy and compliance are mandatory. Our goal is to provide AI augmented capabilities to save clinicians time so they can focus on what's really important inerson patient care. Feel free to click on the titles to see more details. Due to the fourth wave of COVID delta variant, some countries are still having approximately 200,000 new cases in September 2021. Use of face masks and adequate social distancing are essential in hospitals and congregate living spaces. Proper masking is critical in clinical settings where distancing isn't always practical, especially when rates of infection are high.We can enhance an AI solution to deny entry to a facility or a particular area to anyone who isn't properly masked. For example, AWS Sage Maker Neo can be used to deploy AI models on edge devices so that doors don't open unless entrance are properly masked. For the system to work, object detection and localization must occur in near real time. Our solution can be implemented on premises or in the cloud. In a typical training effort, data scientists experiment with AI configurations and data transformation methods using multiple model tuning experiments to maximize performance and accuracy. After they select the model that performs best, they might retrain it at regular intervals by using reason video samples to minimize errors, increase accuracy, and reduce bias. For example, several mask styles are typically in use in any facility, and styles can change when new PPE arrives. Mask styles for personal use have been evolving steadily throughout the pandemic. Some patient masks might be deemed unacceptable. Other facilities requires that patients and staff wear only masks that are approved and supplied by a facility. Retraining keeps a model performing well on mass detection tasks even as transitions like these occur. During experimentation and retraining, efficient data management and traceability are a significant part of the total effort. It becomes essential to have the right tools to construct unified data pipelines that incorporate different data sources and move and manage data quickly and efficiently. It is also important that your data pipeline can deliver real time inferency. How can we do that? On your right hand side, the NetApp data apps toolkit and AI control plan integrate intelligent data management into data science workflow. The NetApp data science toolkit helps data scientists and AI engineers to seamlessly replicate data across sites and regions to create a cohesive and unified AI machine learning deep learning data pipelines for traceability model versioning and AB testing. This toolkit is easy to install using pip. Furthermore, integrating the data apps toolkit with NetApp Astra control center will enable users to deploy and run business critical Kubernetes workloads.Our cloud approach is well suited to sites that already store or archive video surveillance data in the Amazon cloud. We prototype our cloud implementation on AWS by using Amazon Kinesis video streams and Amazon recognition. Kinesis interacts with recognition to tag your video with object detection metadata. Recognition offers the built-in ability to detect faces and track the path of people in a video for social distancing studies. storing detective facial information in collections. Note that recognition is not used to identify people in this implementation.There are more compliance solutions for handwashing and other essential healthcare use cases. Visit ww.netapp.com/ai and look for our healthc care section for more information. Next, let's transition to another vertical, our cloud AI solutions for autonomous driving lane detection. Since 2019, Microsoft delivers an Azure native firstparty portal service for enterprise file services based on our untap technology. This development is driven by a strategic partnership between Microsoft and NetApp. From last year, we've teamed up with run AAI, a company virtualizing AI infrastructure to allow faster AI experimentation with full GPU utilization. The partnership enables teams to speed up AI by running many experiments in parallel. We offer customers a futureproof platform for your AI journey in Azure. From analytics and high performance computing to autonomous decisions, the alliance provides a single unified experience in the Azure cloud for autonomous driving lane detection. Spatial convolutional neuronet network SC CNN generalizes the convolutional neuronet network based approaches to a high reach spatial level. It allows information propagation between neurons in the same layer. However, there are three challenges associated with delivering and deploying a SCNN solution. First, can you use simple few lines of code to access and configure your hybrid cloud compute storage and network? Second, are you currently using or plan to use AI or MLOps pipeline tools? How would you feel if by tomorrow your data pipelines can scale up to tens or even hundreds of times its current magnitude? This is the solution that resolves all three previous challenges. Working with run AI to provide a single pane of class platform for data teams. Your AIM ML data pipeline and associated artifacts are clearly displayed and may be tweaked in stages and minute steps thanks to Qflow integration. Leveraging Horavt Azure Neta files and Nvidia cloud GPUs for distributed training, your resources are fully utilized, enabling automated service level change without disrupting underlying workflows and data. For data scientists and engineers, they can choose to interact with the cluster using Jupyter Lab workspace, command line interface, or their favorite IDE. We achieved 99% cluster GPU utilization with three GPU accelerated worker nodes in Azure Kubernetes cluster running numerous simultaneous message passing interface operations.Please check out our technical report lane detection distributed training with rung AI listed in the resources slide for detail setups and testing instructions. Do you like our cloud AI solution in Azure? Feel free to leave a comment after the session. Now, let's pivot to customer journey monetization. We've all heard of the saying data is the new oil. How do you leverage first and third-party data in your business to differentiate yourself among competitors in the global market? Here's how our customer journey monetization solution applies in three major areas. Click-through rate, CTR is defined as the average number of click-throughs per 100 online ad impressions. It is widely adopted as a key metric in retail and e-commerce. Brick and mortar retailers can correlate the number of visitors and customers to CTR. The number of customers can be seen from their point of sale history. CTR from retailers websites or ad traffic might result in sales. Another use case is loyalty programs. Customers redirected from online ads or other websites might join to earn rewards. Retailers can record behaviors from sales histories of rewards customers to build a recommener that not only predicts customer buying behaviors, but also personalizes coupons and decreases churn. In an e-commerce back end, there are visitor statistics. These stats are typically easy to read and on average more accurate than third-party data. Firstparty data sets from such stats are proprietary and thus the most relevant to e-commerce sellers, buyers and platforms. These data sets are used for setting benchmarks, comparing results to the past by constructing a time series for further analysis. Telecom's and internet service providers have an abundance of firstparty user telemetry data for insightful AI, ML, and analytics. A telecom can leverage its mobile subscribers web browsing top level domain history logs to fine-tune existing models to produce up-to-date audience segmentation, predict customer behavior and collaborate with advertisers to place real-time ads for better online experience. In such datadriven marketing workflow, CTR is an important metric to reflect convergence. Let me show you our CTR solution architecture. We have validated this distributed training solution using Rapids AI and desk resulting in more than 200 times faster Jupiter workspace wall time versus conventional machine learning approach. Read our technical report distributed training in Azure click-through rate prediction listed in resources slide for more details. The solution achieves four major benefits. Simple setup for data scientists, data engineers, and ML ops. Cost control by leveraging ANF dynamic volume shaping as per your AI workload demand.Data management, MF versioning via snapshots. Our data ops toolkit works out of the box. In this session, we'll review a developed conversational AI solution specifically for call centers which as you can imagine are prevalent in nearly every industry from FSI to healthcare. Conversational AI in call centers require technology which can extract insights from calls or audio recordings. In other words, it involves transition between a time series like signal to digestible insights like our call centers have better conversation when they express positivity in the middle of their calls. Extracting key information from conversations is underutilized. But by developing a process to automate these analysis, new insights can be generated to develop pathways to better products, tools, and services. To demonstrate the process, we built a pipeline to use storeofthe-art tools to classify and detect sentiment in financial call data and showcase call center analytics. At a high level, we have audio data coming in whether batched or in real time streaming which is uploaded to any data store like computer cluster or our net app storage with a process set of audio calls. The pipeline applies a speech to text or automated speech recognition model. The transcribed text from those codes is then evaluated using an NLP classification algorithm. In this case, a sentiment analysis algorithm trained to detect positive, neutral, and negative sentiment. Analyzed and aggregated sentiment can be compiled and displayed in a weekly track dashboard. For instance, these aggregated outputs will provide the insight needed to make actionable conclusions. There are a few main components to develop such solutions. Automatic speech recognition.We can use a model or pre-trained apply transfer learning create custom labels and data sets specific to our keywords and then we can get the results from the transcripts. We create name entity recognition models and perform highlevel sentiment analysis or topic modeling. We choose to do sentiment analysis on a public data set as an example that can demonstrate the workflows and power of the tools here for complex terminology and across industries. With this solution, we're trying to streamline and solve the problem of capturing audio data from any source. [snorts] Save and manage it in high performance storage solutions. process that data using AI, machine learning for relevant information, customer or patient analysis such as sentiment analysis, topic modeling or trend analysis. And within that domain, we have a modern scalable environment with both cost, speed and efficiency in mind. There are three main components to this process. All the models have a simple and effective architecture based on birdlike models. Text classification and models consistent of two main modules. An encoder module which is a pre-trained bird-like model such as bird robert or megatron. A decoder module which is an MLP classifier on the output of the first token. There are three main challenges for this solution. For data relevance, we use NVIDIA toolkit for transfer learning to satisfy the requirement for your customized data set. In production architecture, we leverage priorization for speedy process times particularly for ASR using Nvidia Reva toolkit and the data security is guaranteed by using NetApp storage. For business relevance, you have to capture and adopt new users depending on targeted insights and other considerations.For more information, please refer to our published technical report listed in the ref resource session. Here are three key takeaways for you. From edge to core to cloud, our proven expertise, products, and services alleviate pain points for you on all aspects of constructing workflows and data pipelines across healthcare, FSI, retail, and other industries. We have end-to-end conversational AI and computer vision solutions that perform analytics, learning, and prediction at scale. NetApp's proven worldleading data management technologies optimize your workflows, reducing unnecessary costs while maintaining security, compliance, low latency, high throughput, and efficiency. Our data ops toolkit together with Azure NetApp files, cloud volume services, Amazon FSX and ONAP enable simple setup, faster training and inferencing, high compute utilization for data teams and enterprise business users. Your data pipeline can be orchestrated to reduce time to production, giving you actionable insights no matter where you choose to house your data. Here are some related resources. Thank you.