ssd

Ceph: Show OSD to Journal Mapping

/ Christopher Paquin / 3 Comments

calamari

In Ceph, when you create an OSD (Object Storage Device) you also need to create its Journal, which is where data is initially written before it is flushed to an OSD. Note that too maximize I/O it is suggested to use SSD drives as the journal partitions for your OSDs (see this link for reference).

So this is exactly what I did. I basically followed the instructions here regarding the creation of OSDs and Journals.

However post-deployment, I wanted to verify that my journal partitions were actually created properly and were being used as expected. That was a little bit tougher to figure out.

First you need to ssh directly to one of your OSD Servers, this command cannot be run from the monitor/admin node.

[root@osd01 ceph-20]# ceph-disk list
WARNING:ceph-disk:Old blkid does not support ID_PART_ENTRY_* fields, trying sgdisk; may not correctly identify ceph volumes with dmcrypt
/dev/sda :
/dev/sda1 other, xfs, mounted on /boot
/dev/sda2 other, LVM2_member
/dev/sdb :
/dev/sdb1 ceph data, active, unknown cluster 6f7cebf2-ceef-49b1-8928-2d36e6044db4, osd.19, journal /dev/sde1
/dev/sdc :
/dev/sdc1 ceph data, active, unknown cluster 6f7cebf2-ceef-49b1-8928-2d36e6044db4, osd.20, journal /dev/sde2
/dev/sdd :
/dev/sdd1 ceph data, active, unknown cluster 6f7cebf2-ceef-49b1-8928-2d36e6044db4, osd.21, journal /dev/sde3
/dev/sde :
/dev/sde1 ceph journal, for /dev/sdb1
/dev/sde2 ceph journal, for /dev/sdc1
/dev/sde3 ceph journal, for /dev/sdd1

In the output above you can see I have three OSDs (sdb1, sdc1, sdd1) and you can see that my journal disk (sde) has three partitions and you can see how they are mapped to echo SSD.

Poor Man’s eSATA Drive Hot Swap without AHCI or Hotplug Support Under Linux

/ Christopher Paquin / Leave a comment

 

 

ServerRoomFireWeb

Unfortunately, hot-swapping an eSATA drive is a bit more complicated than hot swapping a USB drive.

First off,  your BIOS needs to support AHCI (click here for more info on AHCI), and your SATA controller also needs to support it as well. Secondly your OS, needs to specifically support hot plug, and in the case of Windows 7, it wont boot if you change to AHCI after the OS has been installed.

So, In my case I need to update firmware on lots of SATA SSDs and want to do so without rebooting, and without worrying about changing bios settings. So in order to keep things simple, I followed the procedure below.

First, you need to detect your drive. So watch dmesg to see what drive letter is assigned to your new disk upon initial connection.

#dmesg

[86527.985994]  sdd: unknown partition table
[86528.012820] sd 8:0:0:0: [sdd] Assuming drive cache: write through
[86528.012823] sd 8:0:0:0: [sdd] Attached SCSI disk
[86528.456281] device label btrfs devid 1 transid 11 /dev/sdd

Then, when its time to remove the disk device to the following. Subsitute your disk device letters.

# echo 1 > /sys/block/sdd/device/delete

Now you are free to swap your disk. No reboot, no bios changes, required.

LSI MegaCLI — Check For Failed Raid Controller Battery

/ Christopher Paquin / Leave a comment

701590_rusty_batteryThere are several tools that you can use to monitor and configure and LSI SAS controller, however as I have found, some are easier than others to use and some do not always display the correct information.

In my case my controller is a SAS 9260-8i, and when building a server I always make sure that I install the MegaRaid Storage Manager gui for configuring disks and setting up email alerts. However I have often found that this tool is sometimes confusing to use for other tasks so I also make sure that I install the MegaCLI (command line interface). Both utilities can be downloaded directly from LSI here.

MegaRaid Storage Manager installs to /usr/local/MegaRAID Storage Manager, while the cli installs via rpm to /opt/MegaRAID/MegaCli.

Anyway to check the battery status run the following (note i am running 64 bit os)

#>./MegaCli64 -AdpBbuCmd -aAll

Your output will be lengthy – but look for the line below to know if you need to replace your BBU.

Battery Replacement required            : Yes

Two additional usefully commands are:

  • megacli -AdpAllInfo -aALL lists all the adapters in the machine
  • megacli -PDList -aALL lists all disks and enclosures

Note that there is an open source CLI called Megactl, and while its quick and easy to use to see a quick list of your disks and their statuses, its not shown itself to be accurate when it comes to detecting whether or not a battery has failed. You can get it here

Additonal Megacli command can be found here; https://twiki.cern.ch/twiki/bin/view/FIOgroup/DiskRefPerc