Be able to adjust ring replica counts up and down gradually

As part of supporting globally-distributed clusters, Swift needs to allow operators to change the replica count for a running cluster.

Consider a 3-replica, 3-region cluster. When the owning company grows and wants to add a fourth Swift region, they may want to make the cluster a 4-replica cluster as well, so that there is one replica of each object local to each cluster.

The naive approach of simply adding an entire new replica at once is not adequate to address this requirement. A 3-replica cluster with 30 PB of data in it (10 PB user data) will suffer a very extended period of degraded performance if the replica count is simply incremented.

Instead, a new replica should be able to be added gradually, essentially having fractional replicas. Of course, no one object will have fractional replicas; the idea is nonsensical. Instead, different objects will have different replica counts depending on their hash values. For example, a ring with 3.25 replicas would give 4 replicas to 25% of objects and 3 replicas to the remaining 75%; objects with numerically-smaller hash values would be the ones with more replicas.

A proposed interface:

$ swift-ring-builder foo.builder set-replicas <value>

To gradually add a replica, a cluster operator could run

$ swift-ring-builder foo.builder set-replicas 3.01
$ swift-ring-builder rebalance
<push rings and wait>

$ swift-ring-builder foo.builder set-replicas 3.02
$ swift-ring-builder rebalance
<push rings and wait>

$ swift-ring-builder foo.builder set-replicas 3.03
$ swift-ring-builder rebalance
<push rings and wait>

[...]

$ swift-ring-builder foo.builder set-replicas 4
$ swift-ring-builder rebalance
<push rings>