openSUSE 13.2: Day-1 troubles and troubleshooting – VLC, wireless etcetera

Spent a considerable amount of time yesterday, configuring the system to my liking. Installed Virtualbox and VirtualBox Extension Pack to run the various virtual machines I have. Installed VLC hoping to watch a couple of movies during the weekend. Had a wonderful time installing wireless Broadcom BCM43228 driver.

Wireless troubled me again. I have not automated the loading of the wl.ko driver which I compiled yesterday using Broadcom sources. What I do is:

sudo modprobe lib80211
sudo modprobe cfg80211
cd <Path where the wl.ko driver is>
sudo insmod ./wl.ko

I tried loading the wl.ko driver using modprobe but wireless did not work. Do not know why. Need to learn more about it.
Well, wireless worked once the drivers were loaded. But today wireless stopped working. I was not able to connect at all. The NetworkManager notifier provided a list of available connections. So, wireless driver was fine. I removed the wireless connection from the NetworkManager UI and added it again by filling all required values. Wireless is back in business.

With wireless troubles decimated, I tried to launch VLC. No matter what file I chose, I was greeted by errors only. The codecs were missing. So, I did the following:

sudo zypper in vlc-codecs

Now VLC has got rid of its dementia. Maybe, I will go ahead and watch a couple of movies.

Advertisements

Accessing USB devices from Virtualbox Windows guest on Linux host

This is one issue that had troubled me for long but was not important enough for me to put any effort into soving it. All this changed when I had to upgrade my BIOS on my Dell Optiplex 9020 machine. The problems are:

  • One cannot upgrade BIOS from a Linux machine, a Windows installation is a must. At least, a DOS bootable CD, USB Flash drive is needed
  • I run Manjaro on this machine and installing Windows just for the BIOS upgrade is just too much

So, I had to find a way to access USB devices on my Windows VM guest. I have never been able to do that. Primarily because, I never tried. This time I had to. So here is what is needed:

  1. Go to the Virtualbox Downloads page
  2. If the Virtualbox installed on the system is not the latest move to the Virtualbox Old Builds section
  3. Download the Extension PackVirtualboxWebsite_ExtensionPackLink
  4. Hit the link and the Firefox Save File Dialog pops up with Virtualbox selected as the app to open the file with. Just say OK:Firefox_SaveFileDialog_VirtualboxExtensionPack
  5. Virtualbox will try to install the Extension Pack. It will pop up the following EULA. Scroll to the bottom and the I Agree and I Disagree buttons will get enabled. You know what to choose:Virtualbox_ExtensionPack_EULA
  6. If all goes well, Virtualbox will throw this at you:Virtualbox_ExtensionPack_InstalledSuccessfully
  7. It is time for some radical changes in the system. Bring to life a terminal emulator and feed it the following command to add the desired user to the vboxusers group:
     usermod -a G vboxusers <User_Name>
  8. Now move to the settings of the Virtualbox Windows guest and click the USB settings. Make sure that you click on all checkboxes and finally the settings dialog looks like this:Virtualbox_USBSettings_AddFilter
  9. Make sure that the device that you want to access in the Windows guest is NOT mounted on the Linux host. If it is, unmount it. Then, click on the button with the USB icon which has the green colored plus (+) sign. The above picture shows the tooltip for the said button. Click on that and all available USB devices on the machine will be listed. Choose the one you want to add a filter for. Adding a filter will make the device available on the Windows guest. This makes the device completely inaccessible on the host. Every time you bootstrap the Windows guest, the filter will make sure that the device is available in the guest.
  10. Some driver installations will happen in the guest once the filter is added. A restart of the guest and host would be good for the society.

 

 

 

 

Unified Extensible Firmware Interface (UEFI) and Logical Volume Manager (LVM) adventures: How upgrading to UEFI resulted in this epic post

This is a work-in-progress…

It all started when I found my backup plan lacking in all ways imaginable. To be really honest, I never had one. This is not because I am digitally retarded or lazy (this is debatable) or lack resources. I did not have much to back up. Yes, in this age where digital asset accumulation is natural and people tend to have terabytes of data without even realizing, I had nothing worth a planned back-up:

  • The important documents, that are very few in number, have been sitting cosily in my mails
  • I, somehow, relied only on text based learning materials which were available online

In the past few months, I have seen myself learning a lot using videos – Mathematics, Computer Science, Arts etcetera. This, inevitably, means a growing digital wealth. One fine day, last week, I found that my 1 Tera Byte (TB) disk was more than 50% full. That is when I realized the folly in my ways.

I purchased a couple of the Transcend 2 TB Hard Drives:

I immediately backed-up all my digital belongings. With all my data safe, my brain started its creative endeavours. What was unthinkable became a real possibility now. Suddenly it occured to me:

  • That I was using Legacy Boot Options on my Dell Optiplex 9020 machine when UEFI was very much waiting for me
  • That the BIOS my machine was shipped with was version A01 while A07 was available on Dell’s website
  • That I was using the old partition scheme on my machine and had never experimented with Logical Volume Manager (LVM)
  • That Manjaro installation had some troubles as the GUI windows weren’t refreshing properly and making me mad (this was a minor inconvinience but humans are just insatiable)

Some History

When I installed Manjaro in the first week of October 2013, there was no UEFI or LVM support in the distro. They added these to the distro only in December 2013. I had very little idea as to how UEFI and LVM actually work and was not bothered to learn because I was already burdened with a lot of stuff. With no knowledge to bank on, using advanced features would have been foolish. So, I decided not to play with them.

Unified Extensible Firmware Interface Upgrade

This is quite easy. Just hit F2 after powering on the machine and the BIOS menu appears. Change the boot option to UEFI. Make sure that the Secure Boot option is not touhed at all. But, I knew that my BIOS version was very old and had to be upgraded compulsarily. This is a problem when you are on the Linux distro. This has nothing to do with Linux. This issue exists due to the attitude of large corporations towards FOSS.This is what Dells’s website instructs:

Installation instructions
BIOS Update Executable for Windows/DOS
Download
1. Click Download File to download the file.
2. When the File Download window appears, click Save to save the file to your hard drive.

Run the BIOS update utility from Windows environment
1. Browse to the location where you downloaded the file and double-click the new file.
2. Windows System will auto restart and update BIOS while system startup screen.
3. After BIOS update finished, system will auto reboot to take effect.

Run the BIOS update utility from DOS environment if Legacy Boot Mode(Non-Windows users).
1. Copy the downloaded file to a bootable DOS USB key.
2. Power on the system, then Press F12 key and Select "USB Storage Device" and Boot to DOS prompt.
3. Run the file by typing copied file name where the executable is located. 
4. DOS System will auto restart and update BIOS while system startup screen. 
5. After BIOS update finished, system will auto reboot to take effect.

Run the BIOS update utility from DOS environment if UEFI boot mode with Load Legacy Option disabled (Non-Windows users)
1. Copy the downloaded file to a bootable DOS USB key.
2. Power on the system, then go to BIOS Setup by pressing F2 and go to "General-Boot Sequence - Boot List Option".
3. Change "UEFI" to "Legacy" of Boot List Option.
4. Click "Apply","Exit" to save changes and reboot system.
5. Press F12, then Select "USB Storage Device" and Boot to DOS prompt.
6. Run the file by typing copied file name where the executable is located. 
7. DOS system will auto restart and update BIOS while system startup screen.
8. After BIOS update finished, system will auto reboot.
9. Go to BIOS Setup by pressing F2 and go to "General > Boot Sequence > Boot List Option".
10. Change "Legacy" to "UEFI" Boot Option.
11. Go to "Exit > Exit Save Changes" and reboot system.
--------------------------------------------------------------------------------
Note 1: You will need to provide a bootable DOS USB key. This executable file does not createthe DOS system files.
Note 2: Please make sure you suspend BitLocker encryption before updating BIOS on a BitLocker enabled system. 
If you don't enable BitLocker on your system you can ignore it.

You need Windows/DOS to flash the new version of the BIOS. Linux will NOT work. I had some choices, though:

  • Install Windows temporarily on the machine and flash the BIOS. Then, reinstall Manjaro. What happens if another update arrives in the near future? Will I be willing to install Windows temporarily and suffer the consequential pain? NO.
  • Use Virtualbox + Windows guest + Rufus or UNetBootin utility in Windows guest to create a bootable FreeDOS. This seems painless and an ideal way to handle the scenario.

The second option might not be clear to many. What is FreeDOS, some may ask. For that let us first see what DOS is:

DOS/dɒs/, short for Disk Operating System,[1] is an acronym for several closely related operating systems that dominated the IBM PC compatible market between 1981 and 1995, or until about 2000 including the partially DOS-based Microsoft Windows versions 95, 98, and Millennium Edition.

Related systems include MS-DOS, PC DOS, DR-DOS, FreeDOS, ROM-DOS, and PTS-DOS.

So, when Microsoft decided to kill DOS, many users were left in the lurch. FreeDOS was an attempt to help those users:

FreeDOS is a free DOS-compatible operating system that can be used to play games, run legacy software, or support embedded systems. FreeDOS is basically like the old MS-DOS, but better! For example, FreeDOS lets you access FAT32 file systems and use large disk support (LBA) — a feature not available in MS-DOS, and only included in Windows 95 and newer.

I knew I had to go the FreeDOS way and use Virtualbox to get a bootable FreeDOS USB. I decided to install latest Manjaro with UEFI enabled and then upgrading the BIOS.

So, I downloaded the latest Manjaro XFCE  edition with full support for UEFI and LVM. Then I did the following:

  • Used the dd command and created a bootable USB flash drive for Manjaro installation
  • Changed the Boot option from Legacy to UEFI in my BIOS settings (hit F2 after powering on the machine)
  • Started the Manjaro installation procedure

Manjaro Graphical Installation With UEFI and LVM Features

Manjaro’s graphical installer is a very beautifully crafted software. It resembles Ubuntu’s installer and is very practical and intuitive. The Installation type dialog had the options for an LVM partitioning in the automatic mode:

ManjaroGraphicalInstaller_InstallationType_LVM

and seemingly the LVM option is not available in the advanced mode:

ManjaroGraphicalInstaller_AdvancedInstallationMode

So, I chose automatic mode and let Manjaro installer choose the best partition scheme for me. There seems to be no way to modify the partition scheme to LVM in the automatic mode. This is limiting and would surely be addressed.

The lsblk command spits out this:

NAME                 MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINT
sda                    8:0    0 931.5G  0 disk  
├─sda1                 8:1    0     2M  0 part  
├─sda2                 8:2    0   100M  0 part  /boot/efi
├─sda3                 8:3    0   256M  0 part  /boot
└─sda4                 8:4    0 931.2G  0 part  
  └─cryptManjaro     254:0    0 931.2G  0 crypt 
    ├─ManjaroVG-ManjaroRoot
                     254:1    0  29.3G  0 lvm   /
    ├─ManjaroVG-ManjaroHome
                     254:2    0 886.2G  0 lvm   /home
    └─ManjaroVG-ManjaroSwap
                     254:3    0  15.7G  0 lvm   [SWAP]

Evidently, my 1 TB hard drive (sda) was partitioned automatically into four partitions (sda1, sda2, sda3, sda4). The sda4 is used as an LVM that hosts the logical /, /home and SWAP partitions. I had chosen a separate partition for /home in the Installation type dialog’s automatic option. And yes, the LVM is LUKS encrypted:

GnomeDiskUtility_DiskLayout

Now, what exactly is LVM? This needs to be answered so as to understand whatever has been described in this section:

The full explanation of the concept would be quite a task. Thus, I resort to copying the important facts copied from the Ubuntu Wiki. It would be really benefitial for the reader to also read the following pages to understand LVM properly:

What is LVM?
LVM stands for Logical Volume Management. It is a system of managing logical volumes, or filesystems, that is much more advanced and flexible than the traditional method of partitioning a disk into one or more segments and formatting that partition with a filesystem.
Why use LVM?
For a long time I wondered why anyone would want to use LVM when you can use gparted to resize and move partitions just fine. The answer is that lvm can do these things better, and some nifty new things that you just can’t do otherwise. I will explain several tasks that lvm can do and why it does so better than other tools, then how to do them. First you should understand the basics of lvm.
The Basics
There are 3 concepts that LVM manages:
Volume Groups
Physical Volumes
Logical Volumes
A Volume Group is a named collection of physical and logical volumes. Typical systems only need one Volume Group to contain all of the physical and logical volumes on the system, and I like to name mine after the name of the machine. Physical Volumes correspond to disks; they are block devices that provide the space to store logical volumes. Logical volumes correspond to partitions: they hold a filesystem. Unlike partitions though, logical volumes get names rather than numbers, they can span across multiple disks, and do not have to be physically contiguous.

LVMComponents

Copied from http://www.markus-gattol.name/ws/lvm.html, Markus Gattol’s blog

[The five points below have been copied from the Ubuntu Wiki without modifications]

The Specifics
One of the biggest advantages LVM has is that most operations can be done on the fly, while the system is running. Most operations that you can do with gparted require that the partitions you are trying to manipulate are not in use at the time, so you have to boot from the livecd to perform them. You also often run into the limits of the msdos partition table format with gparted, including only 4 primary partitions, and all logical partitions must be contained within one contiguous extended partition.
Resizing Partitions
With gparted you can expand and shrink partitions, but only if they are not in use. LVM can expand a partition while it is mounted, if the filesystem used on it also supports that ( like the usual ext3/4 ). When expanding a partition, gparted can only expand it into adjacent free space, but LVM can use free space anywhere in the Volume Group, even on another disk. When using gparted, this restriction often means that you must move other partitions around to make space to expand one, which is a very time consuming process that can result in massive data loss if it fails or is interrupted ( power loss ).
Moving Partitions
Moving partitions with gparted is usually only necessary in the first place because of the requirement that partitions be physically contiguous, so you probably won’t ever need to do this with LVM. If you do, unlike gparted, LVM can move a partition while it is in use, and will not corrupt your data if it is interrupted. In the event that your system crashes or loses power during the move, you can simply restart it after rebooting and it will finish normally. When I got my SSD drive, I simply plugged it in, booted it up, and asked lvm to move my running root filesystem to the new drive in the background while I continued working. Another reason you might want to move is to replace an old disk with a new, larger one. You can migrate the system to the new disk while using it, and then remove the old one later.
Many Partitions
If you like to test various Linux distributions, or just different version of Ubuntu, or both, you can quickly end up with quite a few partitions. With conventional msdos partitions, this becomes problematic due to its limitations. With LVM you can create as many Logical Volumes as you wish, and it is usually quite easy since you usually have plenty of free space left. Usually people allocate the entire drive to one partition when they first install, but since extending a partition is so easy with LVM, there is no reason to do this. It is better to allocate only what you think you will need, and leave the rest of the space free for future use. If you end up running out of the initial allocation, adding more space to that volume is just one command that completes immediately while the system is running normally.
Snapshots
This is something you simply can not do without LVM. It allows you freeze an existing Logical Volume in time, at any moment, even while the system is running. You can continue to use the original volume normally, but the snapshot volume appears to be an image of the original, frozen in time at the moment you created it. You can use this to get a consistent filesystem image to back up, without shutting down the system. You can also use it to save the state of the system, so that you can later return to that state if you mess things up. You can even mount the snapshot volume and make changes to it, without affecting the original.

Manjaro Post-Installation

The installtion procedure of Manjaro is like most other established distros – as Zohan would put it, “Silky Smooth”. Once I was done with it, I was focussed on getting my machine updated. I fired up pacman and well, updated. Now I had Manjaro installed and updated. It is time to prepare for a BIOS update. How am I going to do it? The answer took me on a Virtualbox advenutre.

Once I came back from the Virtualbox adventure, I had with me a USB flash drive readily accessible form the Windows VM guest. I  used Rufus to create a bootable FreeDOS USB flash drive from the Windows guest and copied the BIOS executable that I downloaded from the Dell Website. I restarted my machine and hit F12 to get to the one-time boot menu. I chose the USB flash drive for booting and got into FreeDOS. This is where things got a bit confusing.

I had downloaded the following ISO from the official website. This is lacking in some features which makes it inappropriate for BIOS upgrade. Download this if you need to test or learn etc. but not for BIOS upgrade. For BIOS upgrade you need FULL CD:

FreeDOS 1.1 Base

fd11src.iso

Bittorrentor, via bittorrent

Features

  • FreeDOS 1.1 Base CD
  • Contains packages from BASE, and several useful utilities
  • Includes source code for everything
  • Install only, does not include LiveCD
  • 40 MB

The menu options available after the booting of this ISO are:

FreeDOS_OfficialCD_MenuOptions

So, I had to search for a full ISO image that could help me. Further net surfing helped me get my hands on this:

http://ftp.gwdg.de/pub/misc/freedos/files/distributions/1.0/fdfullcd.iso at http://www.z80.eu/freedoscd.html

There may be many other sources too! The only difference is that this ISO image is around 150 MB in size.

I created another FreeDOS bootable USB flash drive with this ISO and the menu options changed favourably. Choose the option highlighted in the image below:

FreeDOS_OfficialFULLCD_MenuOptions

Once the highlighted option is selected, the following appers:

FreeDOS_OfficialFULLCD_AfterFreeDOSSafeModeOption

With the command prompt at A:\>, the BIOS executable that I copied was not available. I used the dir command to look at the files. I had to change the working directory to C: drive. With that done, my file was available. Now, I could type the name of the file and hit enter to execute it. The executable takes control of the situation and flashes the new BIOS, A07 version. Done!

Install FreeBSD 10.0 using memstick IMG file on Virtualbox

When FreeBSD 10.0 arrived, I headed straight to the FreeBSD website and downloaded the FreeBSD-10.0-RELEASE-amd64-memstick.img file. I usually do not burn DVDs. It is just not flexible enough. Writing the installation media on to a USB is easiest thing to do when you are addicted to installing OSs in your free time.

Most Linux distros publish their installation media as ISO images while the FreeBSD team creates an IMG image in addition to the ISO. While the Linux ISO images can be written to a USB (ISO -> USB), the FreeBSD ISO cannot be  written to USB. That is where the FreeBSD IMG file comes into picture. While the IMG file is good for writing onto USB drive, one will surely have a horrid time installing FreeBSD as a Virtualbox guest using the IMG file. Virtualbox does not recognize the IMG format.

What is the solution?

Just run the following command:

VBoxManage convertfromraw -format VDI [filename].img [filename].vdi

Once this is done, a VDI file is generated. This is the virtual hard disk file format used by Virtualbox. Now this file can simply be attached as another storage device in Virtualbox. During the boot process, hit F12 key and enter the virtual BIOS menu. One can select the attached drive and boot from it, in case Virtualbox does not automatically boot from the attached drive.

To write FreeBSD IMG file to USB, I use the following command:

dd if=FreeBSD-10.0-RELEASE-amd64-memstick.img of=/dev/<device> bs=64k

The general command is:

dd if=<Any compatible ISO or IMG file> of=/dev/<device> bs=64k

Yes, dd command works on Linux and FreeBSD

Be very careful while using the above command. The changes are irreversible. Thus, if you select a wrong device, you will definitely lose all data.

Further reading:

Virtualbox on OpenSUSE 13.1: vboxdrv troubles and troubleshooting

I installed Virtualbox on my openSUSE 13.1 system using the following command

sudo zypper in virtualbox

I created a Windows 7 VM. Everything worked. I updated the system later and found Virtualbox reluctant to start. It simply said

Kernel driver not installed (rc=-1908)”

AND

WARNING: The vboxdrv kernel module is not loaded. Either there is no module available for the current kernel (3.5.0-18-generic) or it failed to load. Please recompile the kernel module and install it by sudo /etc/init.d/vboxdrv setup You will not be able to start VMs until this problem is fixed.

The kernel was definitely updated and the Dynamic Kernel Module Support (DKMS) did not work. If it had I would not have seen the above errors. I tried running

sudo /etc/init.d/vboxdrv setup

This failed. I understood from the logs that there was no make module installed on my system. I read further and understood that I was missing quite a lot of packages essential for compiling the driver. I did the following:

sudo  zypper install make gcc  gcc-c++ kernel-source kernel-syms

The log had mentioned about the KERN_DIR parameter(kernel directory) being indeterminate. This meant I needed the kernel-source package. kernel-syms(kernel symbols version) package is needed for the following reason:

Kernel symbols such as functions and variables have version information attached. This package contains the symbol versions for the standard kernels. Installing this package before compiling kernel modules outside of the kernel source tree adds symbol version information in these modules. Modules without symbol version information can only be loaded on exactly the kernel version for which they were compiled. Modules with symbol version information can also be loaded into more recent kernels as long as none of the symbols exported by the kernel changes. (This provides a reasonable level of confidence but does not guarantee that the module will still work.)

Once this was done, I executed

sudo /etc/init.d/vboxdrv setup

Virualbox has no starting troubles now.

Virtualbox Guest Additions for Debian sid guest

The whole process is very simple:

  • Login as root or do a su root or better do a sudo
  • Update APT database with
apt-get update  
  • Update the system with
apt-get dist-upgrade  
Note: The next step is needed the first time only
  • Install the following packages
apt-get install build-essential module-assistant  
  • On the Virtualbox Devices menu, click Insert Guest Additions CD image…
  • Mount the CD via command line or File Manager
  • Open terminal and run
sh /media/cdrom/VBoxLinuxAdditions.run  

If, for some reason, the Insert Guest Additions CD image… does not make the CD available in the guest, it can be from the .VirtualBox directory in the home directory of the host machine:

/home/<USERNAME>/.VirtualBox/
VirtualBox_ChooseCD-DVDtoMountInGuest

Virtualbox Guest Additions for FreeBSD Guest

I had to test something on FreeBSD. I was not sure how the test would turn out. Virtual machines help a lot in this regard. Virtualbox is one of the best tools available that helps in creating virtual machines. I can easily create virtual machines on Virtualbox and test to my heart’s content. Back-ups are possible. Throwing away the virtual machine too would cost nothing(apart from the time one invested in creating the virtual machine).

I installed FreeBSD on Virtualbox and everything went fine. I installed XFCE 4.10 on it. But most of the features that make Virtualbox a pleasure to work with need Virtualbox Guest Additions to be installed on the guest OS. This includes sharing the clipboard, mouse pointer integration, shared folders, full screen/scale-mode etc. Installing Guest Additions is a trivial step in the Windows guests and on most Linux distributions. It is however a different story with FreeBSD. It is still simple but one needs to go through the FreeBSD’s unparalleled documentation.

The FreeBSD Wiki and FreeBSD Handbook have clear details of how to install Virtualbox Guest Additions to the FreeBSD guest. The gist is:

  • First, install the emulators/virtualbox-ose-additions package or port in the FreeBSD guest. This will install the port:
    cd /usr/ports/emulators/virtualbox-ose-additions && make install clean
    

The above process needs the port to be installed on the system. Each port has complete data needed for compiling and installing the application. However, the source code is not downloaded when you install the ports collection. The Makefile in the port will fetch and compile the source code and then install. This may not be suitable for all. About ports:

Each port’s Makefile automatically fetches the application source code, either from a local disk, CD-ROM or via ftp, unpacks it on your system, applies the patches, and compiles. If all went well, a simple make install will install the application and register it with the package system.

I did not have the ports tree with me and did not want to compile. I decided to install the virtualbox-ose-additions binary package. Luckily, it existed and did not have to go through the process of compiling everything. I did the following:

pkg_add -r  virtualbox-ose-additions

That was it. Done.

Add these lines to /etc/rc.conf:

vboxguest_enable="YES" 

vboxservice_enable="YES"

That is all. There is nothing more to it. Everything now works as expected.

Edit: for FreeBSD 10.0

When I installed Xorg packages on the FreeBSD guest, I ran the Xorg -configure command too. After that I installed the guest additions package. But the FULL SCREEN mode did not work. That is when I re-ran the Xorg -configure. The vboxvideo driver was recognized and xorg.conf.new file was generated at some temp location. I copied this file as /etc/X11/xorg.conf. Everything works now.