How to recover a broken LVM configuration

Hello and Welcome ,

Today ,I want to show you how to recover a broken or mis-configured LVM configuration .

If you have worked with LVM at some point in your life, you may have encountered situations where you could potentially lose data on a logical volume. However, what most people don’t know is that these problems can be fixed.

So, let’s take a look at how we can restore a previous state if something goes wrong with our logical volume.

I want to keep this article short, so without further ado, let’s get started.

I’m demonstrating this procedure on a CentOS system (the procedure is the same in any Red Hat-based distribution).

The first thing I’m going to do is create the LVM three-layer structure on a fresh disk (in my example, the disk is vdc).

For the volume group, I named it “vgtest,” and for the logical volume, I named it “ lvtest “ with a size of 2G.

Next, let’s create an XFS filesystem on top of it, create a mount point, and mount it.

Now , we can verify that the volume is mounted successfully , using the list block command or the mount command , both works .

And as a final step before we dig in the troubleshooting procedure , let’s fill our disk with some random files .

Before we break our logical volume, there are a couple of lines in the LVM configuration file that we need to check and ensure they are uncommented.

Specifically, we need to ensure that all the parameters in the backup section inside the config file are uncommented.

Ok , Now let’s break our Logical Volume ,

The trick here is that when we don’t specify the “+” sign in front of the number, the lvresize command will change the volume to that number. This means that instead of growing the volume by an additional 1G, we actually reduce it to 1G.

I know, you can see the warning message here. However, sometimes accidents happen, and some people might type “yes” directly without reading the warning.

So, at this point, if we try to mount it again, we get an error.

So, how do we fix that? Is there a way to roll-back to the previous state?

And the answer is yes! 😁 We can fix it .

The first step is to isolate the affected LV (😂 😂 No, I’m just kidding ,we are not in a zombie movie).

Among all those archives, the first one is too early (before the creation of the LV). The second archive is also too early. However, the third archive seems good to me because it was created before the lvresize command .

Now that we have decided which one to use , let’s go back to that state using the same command with the “ -f “ parameter to specify the archive file .

And as you can see, the restoration was successful.

Finally , let’s disable and enable the logical volume using the lvchange command.

Et voilà! , The volume is back to its previous state with 2G of capacity .It is mounted successfully, and the files we created are still there.

To conclude , when an LV configuration becomes broken or corrupted, we can use these archives to restore to a previous known working state of the LV configuration. These archives serve as backups because they store earlier versions of the metadata configuration and they allow us to revert back to a stable configuration in case of issues or accidental changes.

If you found this article helpful, please consider liking and sharing it . Thank you for your support, and I’ll see you in the next one .