Determining Replication Factor

Volumes are stored as pieces called containers that contain files, directories, and other data. By default, the maximum container size is 32 GB. The maximum container size is set by the cldb.container.sizemb parameter (see the config commands). Containers are replicated to protect data. Normally, each container has three copies stored on separate nodes to provide uninterrupted access to all data, even if a node fails.

For each volume, you can specify a desired and minimum replication factor, a desired and minimum namespace replication factor, and an optimization replication factor.

NOTE: When enabled, the CLDB manages the namespace container replication separate from the data container replication. This capability is used when you have low volume replication but want to have higher namespace replication. Note: The namespace container parameters, nsreplication or nsminreplication. must be the same or larger than the equivalent data replication parameter, replication or minreplication.
  • The replication factor is the number of replicated copies you want for normal operation and data protection. When the number of copies falls below the desired replication factor, but remains equal to or above the minimum replication factor, the CLDB actively creates additional copies of the container while trying to minimize the impact of making an additional copy of the container. Re-replication occurs after the timeout specified in the cldb.fs.mark.rereplicate.sec parameter (configurable using the config API). The minimum replication factor is 1 and the maximum is 6 (default: 3).
  • The minimum replication factor is the smallest number of copies you want in order to adequately protect against data loss. When the replication factor falls below this minimum, re-replication occurs as aggressively as possible to restore the replication level. The minimum replication factor is 1 and the maximum is 6 (default: 2). In all cases, the minimum replication factor cannot be greater than the replication factor.
  • The namespace replication factor is the minimum number of namespace container replicated copies you want for normal operation and data protection. When the number of copies falls below the desired replication factor, but remains equal to or above the minimum replication factor, the CLDB actively creates additional copies of the container while trying to minimize the impact of making an additional copy of the container. Re-replication occurs after the timeout specified in the cldb.fs.mark.rereplicate.sec parameter. The minimum replication factor is 1 and the maximum is 6 (default: 3).
  • The minimum namespace replication factor is the minimum number of namespace container replicated copies you want in order to adequately protect against data loss. When the replication factor falls below this minimum, re-replication occurs as aggressively as possible to restore the replication level. The system will not wait for lost replicas to become available again. The minimum replication factor is 1 and the maximum is 6 (default: 2). In all cases, the minimum replication factor cannot be greater than the replication factor.

If any containers in the CLDB volume fall below the minimum replication factor, writes are disabled until aggressive re-replication restores the minimum level of replication. If a disk failure is detected, any data stored on the failed disk is re-replicated without regard to the timeout specified in the cldb.fs.mark.rereplicate.sec parameter.

If namespace (NS) replication and minimum namespace replication are not set explicitly, they assume the same values as replication and minimum replication respectively. This means that all changes to replication and minreplication parameters are also reflected in nsreplication and nsminreplication. If nsreplication or nsminreplication is modified or specified during creation, nsreplication and nsminreplication start assuming values different from replication and minreplication.