Unified Extensible Firmware Interface (UEFI) and Logical Volume Manager (LVM) adventures: How upgrading to UEFI resulted in this epic post

This is a work-in-progress…

It all started when I found my backup plan lacking in all ways imaginable. To be really honest, I never had one. This is not because I am digitally retarded or lazy (this is debatable) or lack resources. I did not have much to back up. Yes, in this age where digital asset accumulation is natural and people tend to have terabytes of data without even realizing, I had nothing worth a planned back-up:

• The important documents, that are very few in number, have been sitting cosily in my mails
• I, somehow, relied only on text based learning materials which were available online

In the past few months, I have seen myself learning a lot using videos – Mathematics, Computer Science, Arts etcetera. This, inevitably, means a growing digital wealth. One fine day, last week, I found that my 1 Tera Byte (TB) disk was more than 50% full. That is when I realized the folly in my ways.

I purchased a couple of the Transcend 2 TB Hard Drives:
[youtube https://www.youtube.com/watch?v=f0gzPZDflw8]

I immediately backed-up all my digital belongings. With all my data safe, my brain started its creative endeavours. What was unthinkable became a real possibility now. Suddenly it occured to me:

• That I was using Legacy Boot Options on my Dell Optiplex 9020 machine when UEFI was very much waiting for me
• That the BIOS my machine was shipped with was version A01 while A07 was available on Dell’s website
• That I was using the old partition scheme on my machine and had never experimented with Logical Volume Manager (LVM)
• That Manjaro installation had some troubles as the GUI windows weren’t refreshing properly and making me mad (this was a minor inconvinience but humans are just insatiable)

Some History

When I installed Manjaro in the first week of October 2013, there was no UEFI or LVM support in the distro. They added these to the distro only in December 2013. I had very little idea as to how UEFI and LVM actually work and was not bothered to learn because I was already burdened with a lot of stuff. With no knowledge to bank on, using advanced features would have been foolish. So, I decided not to play with them.

Unified Extensible Firmware Interface Upgrade

This is quite easy. Just hit F2 after powering on the machine and the BIOS menu appears. Change the boot option to UEFI. Make sure that the Secure Boot option is not touhed at all. But, I knew that my BIOS version was very old and had to be upgraded compulsarily. This is a problem when you are on the Linux distro. This has nothing to do with Linux. This issue exists due to the attitude of large corporations towards FOSS.This is what Dells’s website instructs:

Installation instructions BIOS Update Executable for Windows/DOS Download 1. Click Download File to download the file. 2. When the File Download window appears, click Save to save the file to your hard drive. Run the BIOS update utility from Windows environment 1. Browse to the location where you downloaded the file and double-click the new file. 2. Windows System will auto restart and update BIOS while system startup screen. 3. After BIOS update finished, system will auto reboot to take effect. Run the BIOS update utility from DOS environment if Legacy Boot Mode(Non-Windows users). 1. Copy the downloaded file to a bootable DOS USB key. 2. Power on the system, then Press F12 key and Select "USB Storage Device" and Boot to DOS prompt. 3. Run the file by typing copied file name where the executable is located. 4. DOS System will auto restart and update BIOS while system startup screen. 5. After BIOS update finished, system will auto reboot to take effect. Run the BIOS update utility from DOS environment if UEFI boot mode with Load Legacy Option disabled (Non-Windows users) 1. Copy the downloaded file to a bootable DOS USB key. 2. Power on the system, then go to BIOS Setup by pressing F2 and go to "General-Boot Sequence - Boot List Option". 3. Change "UEFI" to "Legacy" of Boot List Option. 4. Click "Apply","Exit" to save changes and reboot system. 5. Press F12, then Select "USB Storage Device" and Boot to DOS prompt. 6. Run the file by typing copied file name where the executable is located. 7. DOS system will auto restart and update BIOS while system startup screen. 8. After BIOS update finished, system will auto reboot. 9. Go to BIOS Setup by pressing F2 and go to "General > Boot Sequence > Boot List Option". 10. Change "Legacy" to "UEFI" Boot Option. 11. Go to "Exit > Exit Save Changes" and reboot system. -------------------------------------------------------------------------------- Note 1: You will need to provide a bootable DOS USB key. This executable file does not createthe DOS system files. Note 2: Please make sure you suspend BitLocker encryption before updating BIOS on a BitLocker enabled system. If you don't enable BitLocker on your system you can ignore it.

You need Windows/DOS to flash the new version of the BIOS. Linux will NOT work. I had some choices, though:

• Install Windows temporarily on the machine and flash the BIOS. Then, reinstall Manjaro. What happens if another update arrives in the near future? Will I be willing to install Windows temporarily and suffer the consequential pain? NO.
• Use Virtualbox + Windows guest + Rufus or UNetBootin utility in Windows guest to create a bootable FreeDOS. This seems painless and an ideal way to handle the scenario.

The second option might not be clear to many. What is FreeDOS, some may ask. For that let us first see what DOS is:

DOS/dɒs/, short for Disk Operating System,^[1] is an acronym for several closely related operating systems that dominated the IBM PC compatible market between 1981 and 1995, or until about 2000 including the partially DOS-based Microsoft Windows versions 95, 98, and Millennium Edition.

Related systems include MS-DOS, PC DOS, DR-DOS, FreeDOS, ROM-DOS, and PTS-DOS.

So, when Microsoft decided to kill DOS, many users were left in the lurch. FreeDOS was an attempt to help those users:

FreeDOS is a free DOS-compatible operating system that can be used to play games, run legacy software, or support embedded systems. FreeDOS is basically like the old MS-DOS, but better! For example, FreeDOS lets you access FAT32 file systems and use large disk support (LBA) — a feature not available in MS-DOS, and only included in Windows 95 and newer.

I knew I had to go the FreeDOS way and use Virtualbox to get a bootable FreeDOS USB. I decided to install latest Manjaro with UEFI enabled and then upgrading the BIOS.

So, I downloaded the latest Manjaro XFCE  edition with full support for UEFI and LVM. Then I did the following:

• Used the dd command and created a bootable USB flash drive for Manjaro installation
• Changed the Boot option from Legacy to UEFI in my BIOS settings (hit F2 after powering on the machine)
• Started the Manjaro installation procedure

Manjaro Graphical Installation With UEFI and LVM Features

Manjaro’s graphical installer is a very beautifully crafted software. It resembles Ubuntu’s installer and is very practical and intuitive. The Installation type dialog had the options for an LVM partitioning in the automatic mode:

and seemingly the LVM option is not available in the advanced mode:

So, I chose automatic mode and let Manjaro installer choose the best partition scheme for me. There seems to be no way to modify the partition scheme to LVM in the automatic mode. This is limiting and would surely be addressed.

The lsblk command spits out this:

NAME                 MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINT sda                    8:0    0 931.5G  0 disk   ├─sda1                 8:1    0     2M  0 part   ├─sda2                 8:2    0   100M  0 part  /boot/efi ├─sda3                 8:3    0   256M  0 part  /boot └─sda4                 8:4    0 931.2G  0 part     └─cryptManjaro     254:0    0 931.2G  0 crypt     ├─ManjaroVG-ManjaroRoot                      254:1    0  29.3G  0 lvm   /     ├─ManjaroVG-ManjaroHome                      254:2    0 886.2G  0 lvm   /home     └─ManjaroVG-ManjaroSwap                      254:3    0  15.7G  0 lvm   [SWAP]

Evidently, my 1 TB hard drive (sda) was partitioned automatically into four partitions (sda1, sda2, sda3, sda4). The sda4 is used as an LVM that hosts the logical /, /home and SWAP partitions. I had chosen a separate partition for /home in the Installation type dialog’s automatic option. And yes, the LVM is LUKS encrypted:

Now, what exactly is LVM? This needs to be answered so as to understand whatever has been described in this section:

The full explanation of the concept would be quite a task. Thus, I resort to copying the important facts copied from the Ubuntu Wiki. It would be really benefitial for the reader to also read the following pages to understand LVM properly:

• http://www.markus-gattol.name/ws/lvm.html
• http://www.standalone-sysadmin.com/blog/2008/09/introduction-to-lvm-in-linux/

What is LVM?
LVM stands for Logical Volume Management. It is a system of managing logical volumes, or filesystems, that is much more advanced and flexible than the traditional method of partitioning a disk into one or more segments and formatting that partition with a filesystem.
Why use LVM?
For a long time I wondered why anyone would want to use LVM when you can use gparted to resize and move partitions just fine. The answer is that lvm can do these things better, and some nifty new things that you just can’t do otherwise. I will explain several tasks that lvm can do and why it does so better than other tools, then how to do them. First you should understand the basics of lvm.
The Basics
There are 3 concepts that LVM manages:
Volume Groups
Physical Volumes
Logical Volumes
A Volume Group is a named collection of physical and logical volumes. Typical systems only need one Volume Group to contain all of the physical and logical volumes on the system, and I like to name mine after the name of the machine. Physical Volumes correspond to disks; they are block devices that provide the space to store logical volumes. Logical volumes correspond to partitions: they hold a filesystem. Unlike partitions though, logical volumes get names rather than numbers, they can span across multiple disks, and do not have to be physically contiguous.

The Specifics
One of the biggest advantages LVM has is that most operations can be done on the fly, while the system is running. Most operations that you can do with gparted require that the partitions you are trying to manipulate are not in use at the time, so you have to boot from the livecd to perform them. You also often run into the limits of the msdos partition table format with gparted, including only 4 primary partitions, and all logical partitions must be contained within one contiguous extended partition.
Resizing Partitions
With gparted you can expand and shrink partitions, but only if they are not in use. LVM can expand a partition while it is mounted, if the filesystem used on it also supports that ( like the usual ext3/4 ). When expanding a partition, gparted can only expand it into adjacent free space, but LVM can use free space anywhere in the Volume Group, even on another disk. When using gparted, this restriction often means that you must move other partitions around to make space to expand one, which is a very time consuming process that can result in massive data loss if it fails or is interrupted ( power loss ).
Moving Partitions
Moving partitions with gparted is usually only necessary in the first place because of the requirement that partitions be physically contiguous, so you probably won’t ever need to do this with LVM. If you do, unlike gparted, LVM can move a partition while it is in use, and will not corrupt your data if it is interrupted. In the event that your system crashes or loses power during the move, you can simply restart it after rebooting and it will finish normally. When I got my SSD drive, I simply plugged it in, booted it up, and asked lvm to move my running root filesystem to the new drive in the background while I continued working. Another reason you might want to move is to replace an old disk with a new, larger one. You can migrate the system to the new disk while using it, and then remove the old one later.
Many Partitions
If you like to test various Linux distributions, or just different version of Ubuntu, or both, you can quickly end up with quite a few partitions. With conventional msdos partitions, this becomes problematic due to its limitations. With LVM you can create as many Logical Volumes as you wish, and it is usually quite easy since you usually have plenty of free space left. Usually people allocate the entire drive to one partition when they first install, but since extending a partition is so easy with LVM, there is no reason to do this. It is better to allocate only what you think you will need, and leave the rest of the space free for future use. If you end up running out of the initial allocation, adding more space to that volume is just one command that completes immediately while the system is running normally.
Snapshots
This is something you simply can not do without LVM. It allows you freeze an existing Logical Volume in time, at any moment, even while the system is running. You can continue to use the original volume normally, but the snapshot volume appears to be an image of the original, frozen in time at the moment you created it. You can use this to get a consistent filesystem image to back up, without shutting down the system. You can also use it to save the state of the system, so that you can later return to that state if you mess things up. You can even mount the snapshot volume and make changes to it, without affecting the original.

Manjaro Post-Installation

The installtion procedure of Manjaro is like most other established distros – as Zohan would put it, “Silky Smooth”. Once I was done with it, I was focussed on getting my machine updated. I fired up pacman and well, updated. Now I had Manjaro installed and updated. It is time to prepare for a BIOS update. How am I going to do it? The answer took me on a Virtualbox advenutre.

Once I came back from the Virtualbox adventure, I had with me a USB flash drive readily accessible form the Windows VM guest. I  used Rufus to create a bootable FreeDOS USB flash drive from the Windows guest and copied the BIOS executable that I downloaded from the Dell Website. I restarted my machine and hit F12 to get to the one-time boot menu. I chose the USB flash drive for booting and got into FreeDOS. This is where things got a bit confusing.

I had downloaded the following ISO from the official website. This is lacking in some features which makes it inappropriate for BIOS upgrade. Download this if you need to test or learn etc. but not for BIOS upgrade. For BIOS upgrade you need FULL CD:

FreeDOS 1.1 Base

fd11src.iso

or, via bittorrent

Features

• FreeDOS 1.1 Base CD
• Contains packages from BASE, and several useful utilities
• Includes source code for everything
• Install only, does not include LiveCD
• 40 MB

The menu options available after the booting of this ISO are:

So, I had to search for a full ISO image that could help me. Further net surfing helped me get my hands on this:

http://ftp.gwdg.de/pub/misc/freedos/files/distributions/1.0/fdfullcd.iso at http://www.z80.eu/freedoscd.html

There may be many other sources too! The only difference is that this ISO image is around 150 MB in size.

I created another FreeDOS bootable USB flash drive with this ISO and the menu options changed favourably. Choose the option highlighted in the image below:

Once the highlighted option is selected, the following appers:

With the command prompt at A:>, the BIOS executable that I copied was not available. I used the dir command to look at the files. I had to change the working directory to C: drive. With that done, my file was available. Now, I could type the name of the file and hit enter to execute it. The executable takes control of the situation and flashes the new BIOS, A07 version. Done!