Optimize Data Management and Analytics with NetApp Solutions for Hadoop

With NetApp® technology, you get flexibility and choice based on your use case. If you want to use data management by Hadoop Distributed File System (HDFS), we provide NetApp FAS and AFF systems and NetApp E-Series and EF-Series systems. FAS and AFF systems are certified with Hortonworks Data Platform (HDP) 3.0.0, and E-Series and EF-Series systems are certified with HDP 2.6.5. E-Series and EF-Series systems are also Cloudera certified.

If you need a more optimized in-place analytics solution and you don’t require HDFS, you can use the NetApp In-Place Analytics Module. You gain the following key advantages from the NetApp In-Place Analytics Module compared with competitors’ offerings:

• A single source or copy of data versus a minimum of two or three copies
• A reduced data footprint by using deduplication and compression; data reduction of between 5 and 10 times compared with HDFS deployments
• A unified data lake architecture between Hadoop and artificial intelligence (AI) and deep learning (DL) workloads
• Quick NetApp Snapshots™ copies for testing and development and easy space-efficient backups
• Ability to run analytics on data that’s in NFS storage
• Support for new Hadoop features without upgrading the storage
• Ability to scale storage and compute independently
• Support for data security by using Ranger and Kerberos
• Identical architectures between on-premises deployments and cloud deployments by using NetApp Cloud Volumes Service

The following figure depicts in-place analytics on a Hadoop/Spark cluster.

HDP 3.0.0 Certified with NetApp Cloud Volumes

In the cloud, the configuration with HDP 3.0.0 is certified with NetApp Cloud Volumes ONTAP® technology and Cloud Volumes Service running HDFS over NFS. Cloud Volumes Service is a cloud-native service that gives you high-performance file storage in the cloud, with NFS and SMB connectivity. You get rich data management, such as efficient Snapshot copies and clones, as well as an integrated backup service. You can deploy Cloud Volumes Service in seconds with three different performance tiers on AWS or on Google Cloud Platform. If you use Azure, you can deploy a similar native Azure service called Azure NetApp Files, which is also built on NetApp technology.