Ceph: default value for pool placement groups number parameters

Fuel sets the following placement group parameters in ceph.conf

  osd_pool_default_pg_num
  osd_pool_default_pgp_num

using the formula:

  Nearest power of 2 (num_of_OSDs * 100 / num_of_replica)

This formula is correct only for clusters with one pool. For clusters with many pools it overestimate the number of placement groups by the number of the pools times (the goal is to have 100 placement groups per osd or a little high).

As it is unknown how many pools a user plans for a cluster, it is not possible to add number of pools to the formula, to calculate the value correctly. Still we need to change the formula, because large amount of placement groups per osd leed to long peering times after any changes to the cluster, during this period operations that need data from peering placement groups hang.

Taking into consideration that it is much easier to increase pg num values for existing pools (pg splitting is supported) than decreasing them (currently pg merging is not supported, so the only way is to recreate the pool and copy data), it is much better if we underestimate the parameters than overestimate.

In many cases we can expect that a cluster will have at least 10 pools (images, volumes, compute + 7 pools for radosgw). So, the formula can be changed to:

 max(128, Nearest power of 2 (num_of_OSDs * 100 / num_of_replica / 10))

Another option is to add a possibility for users to set this parameter manually via GUI, similarly to what we already do with ceph replication factor.