Bridging the Memory-Storage Gap

Abstract:

The Internet has become indispensable in the developing world. It has
become an important tool for providing entertainment, for enabling and
bettering human communication, for delivering effective education, for
conducting research, for spreading news, and for organizing people
to rally to various causes. However, today, only a third of the world's
population has quality access to the Internet. The Internet has two
primary hindrances for expansion. First, the high cost of network
connectivity in developing regions and second, the high cost of
establishing new data centers to reduce the load on the existing data
centers. Fortunately, caches in various forms help address both these
problems by storing reusable content near the clients. Despite their
importance, today's caches are limited in their scale because of the
trends in the evolution of current memory-storage technologies.

The widening gap between memory and storage is limiting the
performance of applications like caches. The ever-increasing
amount of data and the need to access more of it quickly have further
magnified the gap. Limited DRAM capacity of servers makes it
difficult to obtain good in-memory hit-rates that are vital for
avoiding high-latency disk accesses. This dissertation presents two
new complementary methods (HashCache and SSDAlloc) to bridge this gap
for caches that affect the performance of many applications including
HTTP Web Proxy caches, wide area network accelerators, content
distribution networks, and file backup services. First, we will
develop HashCache, a novel method to drastically improve the memory
efficiency of caches. By reducing the amount of memory needed for
caching by up to 20 times, it reduces the cache's total cost of
ownership. While HashCache makes more effective use of limited RAM in
a system, SSDAlloc introduces a tier of new memory technology like
NAND-Flash between RAM and disk to further bridge the gap.

SSDAlloc removes the impediments to the integration of new
high-capacity memory technologies like NAND-Flash into the memory
hierarchy. SSDAlloc is a novel memory manager that helps applications
like HashCache tier data transparently and efficiently between DRAM
and NAND-Flash. With only a few modifications to an existing
application, restricted to the memory allocation portions of the code,
one can reap the benefits of new memory technologies. Additionally,
with SSDAlloc, applications can obtain 90\% of the raw performance of
NAND-Flash, while existing transparent tiering mechanisms deliver only
6-30\% of that. Furthermore, by cleverly discerning application
behavior, SSDAlloc writes up to 32 times less data to NAND-Flash when
compared to similar existing mechanisms. This greatly increases the
reliability of NAND-Flash that has a limited lifetime unlike DRAM.