Optimize Data Management and Analytics with NetApp Solutions for Hadoop
Mike McNamara
With NetApp® technology, you get flexibility and choice based on your use case. If you want to use data management by Hadoop Distributed File System (HDFS), we provide NetApp FAS and AFF systems and NetApp E-Series and EF-Series systems. FAS and AFF systems are certified with Hortonworks Data Platform (HDP) 3.0.0, and E-Series and EF-Series systems are certified with HDP 2.6.5. E-Series and EF-Series systems are also Cloudera certified.
If you need a more optimized in-place analytics solution and you don’t require HDFS, you can use the NetApp In-Place Analytics Module. You gain the following key advantages from the NetApp In-Place Analytics Module compared with competitors’ offerings:
- A single source or copy of data versus a minimum of two or three copies
- A reduced data footprint by using deduplication and compression; data reduction of between 5 and 10 times compared with HDFS deployments
- A unified data lake architecture between Hadoop and artificial intelligence (AI) and deep learning (DL) workloads
- Quick NetApp Snapshots™ copies for testing and development and easy space-efficient backups
- Ability to run analytics on data that’s in NFS storage
- Support for new Hadoop features without upgrading the storage
- Ability to scale storage and compute independently
- Support for data security by using Ranger and Kerberos
- Identical architectures between on-premises deployments and cloud deployments by using NetApp Cloud Volumes Service
The following figure depicts in-place analytics on a Hadoop/Spark cluster.
HDP 3.0.0 Certified with NetApp Cloud Volumes
In the cloud, the configuration with HDP 3.0.0 is certified with NetApp Cloud Volumes ONTAP® technology and Cloud Volumes Service running HDFS over NFS. Cloud Volumes Service is a cloud-native service that gives you high-performance file storage in the cloud, with NFS and SMB connectivity. You get rich data management, such as efficient Snapshot copies and clones, as well as an integrated backup service. You can deploy Cloud Volumes Service in seconds with three different performance tiers on AWS or on Google Cloud Platform. If you use Azure, you can deploy a similar native Azure service called Azure NetApp Files, which is also built on NetApp technology.
The certified configuration with HDP 3.0.0 enables you to take advantage of the performance and reliability of Cloud Volumes Service in the cloud. At the same time, you benefit from the data management capabilities that are built into HDFS.
NetApp has been supporting Hadoop for many years to deliver advanced solutions for big data analytics. With NetApp technology, you get industry-leading storage and data management that complements your Hortonworks Data Platform that’s built on Hadoop.
Find out how you can optimize data management and accelerate data analytics with NetApp. For more information, visit netapp.com/bigdata.
Mike McNamara
Mike McNamara is a senior product and solution marketing leader at NetApp with over 25 years of data management and cloud storage marketing experience. Before joining NetApp over ten years ago, Mike worked at Adaptec, Dell EMC, and HPE. Mike was a key team leader driving the launch of a first-party cloud storage offering and the industry’s first cloud-connected AI/ML solution (NetApp), unified scale-out and hybrid cloud storage system and software (NetApp), iSCSI and SAS storage system and software (Adaptec), and Fibre Channel storage system (EMC CLARiiON).
In addition to his past role as marketing chairperson for the Fibre Channel Industry Association, he is a member of the Ethernet Technology Summit Conference Advisory Board, a member of the Ethernet Alliance, a regular contributor to industry journals, and a frequent event speaker. Mike also published a book through FriesenPress titled "Scale-Out Storage - The Next Frontier in Enterprise Data Management" and was listed as a top 50 B2B product marketer to watch by Kapos.
