Note that most of this process is actually well documented, but I added in a few extra steps good to run if you want to start from scratch without having to reinstall an OS.
First remove all CEPH rpms from your CEPH hosts, this includes Monitor nodes and OSD nodes. Note that I am in /root/ceph-deploy on my monitor/admin server. If you have separate admin and monitor nodes then run these commands from your admin node.
# ceph-deploy purge mon01 osd01 osd02 osd03
Now purge all config files.
# ceph-deploy purgedata mon01 osd01 osd02 osd03
Now remove all keys
# ceph-deploy forgetkeys
Now delete any remaining files or keys from /root/ceph-deploy, if there are any files in here that you may want to keep, just copy them to /root.
# rm -rf /root/ceph-deploy/*
Now remove Calamari. Note: that in my test lab I am running Calamari on my one MON node which also happens to be my admin node, so I run this command there
#yum remove ceph-deploy calamari-server calamari-clients
You can now start over. I am installing Red Hat Ceph 1.3, so I am using the instructions here.
You might also want to uninstall salt from your servers. If that’s the case just look for any of the packages below and rip them out. Your versions will vary.