Enhancing the Linux Memory Architecture to Support File Systems over Heterogeneous Devices

This position paper was selected for presentation at the invitation-only Linux Storage and File System Workshop.

The Linux kernel must deal with the ever-growing performance heterogeneity of network and I/O devices. In a heterogeneous environment a single, policy-based framework for memory management does not provide good write performance to all storage resources. Currently, Linux treats all memory pages uniformly without considering the capabilities of the underlying device.

New implementations of traditional file sharing mechanisms such as zero-copy NFS over RDMA make this problem more apparent. We have conducted a series of experiments using NFS over RDMA that show that write throughput for cached I/O lags far behind the available bandwidth. This is an indication that the current memory management scheme is not optimized for low-latency, high-bandwidth interconnects such as Infiniband or 10Gb Ethernet. Although the memory manager can be bypassed via direct I/O, this solution requires changes to existing applications and often yields lower system performance because it loses the benefit caching.

In Proceedings of Linux Storage and Filesystem Workshop 2007

Resources

A copy of the paper is attached to this posting.Linux-memory-lsf07.pdf