CRUSHDIFF(8) Ceph CRUSHDIFF(8)

crushdiff - ceph crush map test tool

crushdiff [ --osdmap osdmap ] [ --pg-dump pg-dump ]
[ --compiled ] [ --verbose ] command crushmap

crushdiff is a utility that lets you test the effect of a crushmap change: number of pgs, objects, bytes moved. This is a wrapper around osdmaptool(8), relying on its --test-map-pgs-dump option to get the list of changed pgs. Additionally it uses pg stats to calculate the numbers of objects and bytes moved.

By default, crushdiff will use the cluster current osdmap and pg stats, which requires access to the cluster. Though one can use the --osdmap and --pg-dump options to test against previously obtained data.

The input/output crushmap is compiled. If the options is not specified the expected/returned crushmap is in txt (decompiled) format.

JSON output of ceph pg dump. If not specified crushdiff will try to get data running the command itself.

The cluster osdmap, obtained with ceph osd getmap command. If not specified crushdiff will try to get data running the command itself.

Produce diagnostic output.

Compare the crushmap from crushmap file with the crushmap from the cluster osdmap. The output will show the expected number of pgs, objects, bytes moved when the new crushmap is installed.
Export crushmap to crushmap file from the cluster osdmap.
Import crushmap from crushmap file to the cluster osdmap.

Get the current crushmap:

crushdiff export cm.txt


Edit the map:

$EDITOR cm.txt


Check the result:

crushdiff compare cm.txt
79/416 (18.99%) pgs affected
281/1392 (20.19%) objects affected
80/1248 (6.41%) pg shards to move
281/4176 (6.73%) pg object shards to move
730.52Mi/10.55Gi (6.76%) bytes to move


When running with --verbose option the output will also contain detailed information about the affected pgs, like below:

4.3     [0, 2, 1] -> [1, 4, 2]
4.b     [0, 1, 3] -> [2, 1, 3]
4.c     [4, 0, 1] -> [4, 1, 2]


i.e. a pg number, and its old and the new osd active sets.

If the result is satisfactory install the updated map:

crushdiff import cm.txt


crushdiff is part of Ceph, a massively scalable, open-source, distributed storage system. Please refer to the Ceph documentation at https://docs.ceph.com for more information.

ceph(8), crushtool(8), osdmaptool(8),

2010-2024, Inktank Storage, Inc. and contributors. Licensed under Creative Commons Attribution Share Alike 3.0 (CC-BY-SA-3.0)

April 19, 2024 dev