Using parity to more efficiently store objects.

Swift can provide more efficient storage with tunable resiliently by storing objects in parity encoded chunks. (like an arbitrary cluster wide RAID level). This comes with a cost of higher latency but for longer term archival storage of large amounts of data this is often preferred to replicas (or can be combined with replicas for even more control over space vs. resiliency).

The zfec library (http://pypi.python.org/pypi/zfec) provides fast parity calculation and would allow a tunable amount of control over resiliency for a given resiliency class (i.e store a chunk across 10 systems with 8 chunks needed to reconstruct the original). A 100MB file stored in this way would only require 125 MB(100*10/8) across 10 systems vs 300MB across 3 systems for standard 3x replication.