Extending GPT/LVM disk on CentOS

Friday turned out to be a tinker day, one of those where you come home a little wiser than when you left 🙂

In production we have some HPE servers running large data collections, something I guess would qualify as big-data. They are based on CentOS 7.2 and each have a data volume of 15TB that was running low.

What’s the plan Doc?

The plan was to expand the volumes as there were available drive slots in the server for additional drives. The extra drives was installed, added to the RAID set and finally the logical drive was expanded. If you want to know how to do this on a HPE server, check out my other article I wrote on the subject – Install HPE Management Components on CentOS

So far so good, now come the tricky part.

When you find them, there are many ways to complete the objective but first you need to know what should be done.

First we need to deal with the disk as we would with any other physical drive, we need to either expand the partition one the disk or create a new one. This is only required as the disk was created with a GPT (GUID Partition Table) partition before LVM was used to manage the disk.

We don’t actually need a partition on the disk for LVM to use is, but as the disk would appear empty to legacy tools if we didn’t fill it with a partition, the risk exists that another sysadmin could claim the disk by accident and effectively destroy the data.

There are plenty of articles that explain the LVM (Logical Volume Manager) technology and surely I’m no expert on the subject, so if you want to dig deeper go ahead and google it.

But how?

So how do we accomplish the task – as my colleague was nice enough to point out, you can just use GParted but only if you have a GUI installed on the server or have another system with a GUI that can access the system remotely via an SSL connection.

Not having a GUI available to me, I had to go ahead and find out how to do it using the CLI and the tools available on the distribution.

When you google the subject, you will learn that you spend a lot of time searching through articles describing how to use fdisk to accomplish the task – while this was the correct approach using MBR drives (< 2TB) the fdisk tool is not supported with GPT although it can be used to see information about the disks.

My first approach what to use Parted as it was available on the system and I also referenced in various articles.

I then found another tool called GDisk also know as “GPT fdisk” that is described as “more compatible with GPT than parted on certain systems” – This is confusing but let me break it down for you.

Parted lastest version is 3.2 and it has not been updated since 2014 – the latest version available in the CentOS repository is version 3.1 that obviously is even older.

GDisk latest version is 1.0.3 and has been updated late 2017 – though the latest version available in the CentOS repository is version 0.8.7 that was released in 2013.

So to answer the question, GDisk is maybe not the most GPT compatible tool for CentOS 7.2 but it will work. So what you choose to use is a personal preference.

If you want to find out what versions of a specific tool is available for your system you can use a site like pkgs.org to search for it.

Partition alignment

Just a quick word on partition alignment before we get started, as you may come across it when using the tools or in various articles.

Partition alignment is the proper alignment of partitions to the boundaries available in a data storage device.

It’s possible to align a partition in two ways, either optimal or minimal – it’s also possible not to align the partition, but event though it’s not important when using LVM, I’m sure you don’t want to do this.

There are several articles describing how to calculate alignment or how to get the tools to help you.

By default the partition will be placed next to the previous when using GDisk below, so you don’t have to worry about this. If the first partition is not aligned you need to remove and recreate it for it to be aligned. That means losing you data, so you properly don’t want to do that. Since we are expanding the drive we can just leave it as is and proceed with the next steps.

Let’s dive in

CAUTION: Always understand what you are doing and research before you being. If your situtation does not match this scenario please only use a reference. You could potentially face data loss or corruption – you have been warned!

Before we start our adventure you need to become root on your server if you are not already logged in as root (hopefully you are not). So start with

sudo su

Now start you weapon of choice and point to the disk that have been expanded.
(you can use the ‘lsblk’ to get an overview of the disks and device names)

gdisk /dev/<device> OR parted /dev/<device>

If you don’t have GDisk installed, you can get it using yum:

yum install gdisk

The first thing we need to do is move the secondary GPT header (also called the backup header) because we have expanded the physical disk (which actually is a logical raid volume) and therefore the header is no loger located in the end of the disk as it should be.

• Parted
  1. print
     – This command will give a warning if the GPT header is not at the end of the disk.
  2. We expect parted will return the warning and choose ‘Fix‘ to relocate the header.
  3. Next we will receive a warning that not all space available is usable and if we would like to fix it, choose ‘Fix‘.
• GDisk
  1. v
     – Verify that we need to relocate the GPT header.
  2. x
     – Enter the expert menu.
  3. e
     – Relocate the GPT header to the end of the disk.
  4. m
     – return to the previous menu.
  5. You need to write the changes for GDisk to save them – either do this now or proceed with partition creation and then save all the changes.

Now we can go ahead and create the new partition.

It’s worth mentioning that if you have access to parted version 3.2, you should be able to expand the existing partition using the ‘resizepart’ command as described in this article

• Parted
  1. unit s
     – Change to sector view
  2. print
     – Find the sector end of the existing partition and make a note of it
  3. mkpart primary <endSectorExistingPartition+1>s 100%
     This will make a new partition that is starting one sector after the existing end to the end of the disk
  4. print
     To see the new partition number
  5. set <partitionNumber> lvm on
     – This will set the partition type to LVM
  6. quit
     – Exit the program
• GDisk
  1. n
     – Create a new partition
  2. Verify partition start
  3. Verify partition end
  4. 8E00
     – Set the partition type to Linux LVM
  5. w
     – Write the changes to disk

Extending LVM

Now we have created the new partition you need to make the kernel aware of the updated partition table otherwise you could be facing critical issues such as data loss or corruption. You can achieve this by either rebooting the server or issue the following command from the CLI:

partprobe

That’s it for the GPT part and can now proceed with the LVM part. These next steps are actually pretty common and well documented and you can easily find articles describing the procedure. If you used the ‘resizepart’ command in parted and resized the existing partition, you can use the rest of the procedure described in the article above. Otherwise proceed as follows.

First we need to find the name of the newly created partition, so go ahead and issue the ‘lsblk’ command

lsblk

Find the device name of the new partition and make a note of it. Issue the ‘pvcreate’ command to create a physical volume.

pvcreate /dev/<partitionDeviceName>

Extend the volume group. You can find the name of the volume group with the ‘vgs’ command.

vgextend <volumeGroupName> /dev/<partitionDeviceName>

You can confirm that the new space have been added to the group with the ‘vgs’ command.

Next step is to expand the logical volume.

lvextend -l +100%FREE /dev/<volumeGroupName>/<logicalVolumeName>

The command is issued with the parameter ‘-l’ and ‘+100%FREE’ to instruct lvextend to increase the size of the logical volume with all available extends in the volume group.
Note that the ‘+’ in the command is very important as it tells lvextend to add space to the group.

Wrapping up

Last thing we have to do is resize the file system that resides on top of the logical volume by using the ‘resize2fs’ command.

resize2fs /dev/<volumeGroupName>/<logicalVolumeName>

Please note, if your filesystem is XFS (i.e. CentOS 7), you will receive the following error message:

Bad magic number in super-block while trying to open /dev/<volumeGroupName>/<logicalVolumeName>
Couldn't find valid filesystem superblock.

You will then need to use this command instead.

xfs_growfs -d /dev/<volumeGroupName>/<logicalVolumeName>

By default, most file system resizing tools will increase the size of the file system to be the size of the underlying logical volume so you do not need to worry about specifying the size.

You can check that the additional space is available by using the ‘df -h’ command.

Other references:

https://www.lifewire.com/how-to-use-linux-to-find-the-names-of-the-devices-on-your-computer-4078068

https://www.ibm.com/developerworks/library/l-resizing-partitions-2/index.html

https://wiki.archlinux.org/index.php/GNU_Parted#Growing_partitions

https://www.howtoforge.com/logical-volume-manager-how-can-i-extend-a-volume-group

Live Resizing LVM on Linux

How to Extend/Reduce LVM’s (Logical Volume Management) in Linux – Part II