Think Global, Act Local: A Buffer Cache Design for Global Ordering and Parallel Processing in the WAFL File System

2016 International Conference on Parallel Processing (ICPP 2016) Philadelphia, PA, USA.

Given the enormous disparity in access speeds between main memory and storage media, modern storage servers must leverage highly effective buffer cache policies to meet demanding performance requirements. At the same time, these page replacement policies need to scale efficiently with ever-increasing core counts and memory sizes, which necessitate parallel buffer cache management. However, these requirements of effectiveness and scalability are at odds, because centralized processing does not scale with more processors and parallel policies are a challenge to implement with maximum effectiveness. We have overcome this difficulty in the NetApp® Data ONTAP® WAFL® file system by using a sophisticated technique to simultaneously allow global buffer prioritization while providing parallel management operations. In addition, we have extended the buffer cache to provide a soft isolation of different workloads’ buffer cache usage, which is akin to buffer cache quality of server (QoS). This paper presents the design and implementation of these significant extensions in the buffer cache of a high-performance commercial file system.

Resources

The definitive version of the paper can be found at: http://dx.doi.org/10.1109/ICPP.2016.51.