We have built a distributed file system called Archipelagic File System (APFS). APFS is distinct from other file systems in its approach to failure isolation and low-cost consistency maintenance. The building blocks of APFS are smaller self-contained file servers called islands. The main idea underlying APFS design is the one-island principle: almost every atomic operation should require the participation of exactly one island or server. The one-island principle improves partial reliability and availability because each island is self-contained and therefore can function independently of others' failures. The one-island principle allows an archipelagic file system to scale well with the number of islands and workloads because cross-island communication and synchronization are minimized. Reconfiguration (addition or removal of islands, or dynamic load balancing) is efficient because it only requires a minimal amount of data to be migrated between islands. Existing file system structures cannot satisfy the one-island principle. We built a new system structure in which data is partitioned to servers at directory granularity by hashing the pathnames of directories.

Experiments on the contents of five existing file systems in use show that the hashing algorithm in APFS can evenly distribute workload to islands. Statistics and scalability measurements show that on average 99.8% of operations satisfy the one-island principle and that performance of these one-island operations scales efficiently with the number of islands and workloads. A trace-driven study of online reconfiguration for a web server running on APFS shows that data migration during reconfiguration is made transparent to the web server and imposes a performance penalty of only 4.5%.