Using Linux to Upgrade a Windows® Hard Disk

Background

Upgrading the hard disk on an existing Windows installation can be one of the trickiest upgrades to perform. When adding more memory, adding a faster CPU, or even adding an additional (second, third, etc.) hard disk, Windows takes care of most of the work for you by autodetecting the new hardware and integrating it seamlessly. But, when replacing an existing hard disk, probably with a larger or faster one, there are a host of other issues to consider. You probably have a bunch of data on your old hard disk that you want to preserve. Luckily, there are a host of backup options that make that easy to deal with. But, what about the OS installation itself, and the installation and configuration of all your favorite programs? If you want to perform the hard disk replacement, and in one step, have everything (programs, preferences, settings, etc.) exactly how they were, you have a problem. If you have licensed software (e.g. Windows, or other application) that you either don't have the license key for, don't want to have to manually install again, or fear might require you to buy another license for, you have a problem. For example, Microsoft licensing uses a rather complicated scheme of identifying different components within a computer to determine when you've simply made an upgrade, and when you have a new computer (not a black-and-white issue). Changing hardware may trigger their software to require a new license to be purchased. Luckily, there are some options.

First of all, you should always have a good system for backing up your important data, regardless of whether or not you plan on changing out a hard disk. Secondly, before making an upgrade like this, it's wise to try to compile a list of all the license keys for the licensed software you have (and wish to keep using). For some apps, you can recover the license key if you've misplaced it. A good idea is to use a key finder program to recover any lost keys before embarking on a hard disk upgrade. Search for "license key finder" and you'll be presented with many options.

Once you've setup a backup system, and collected your license keys, you're ready to attempt the hard disk upgrade. One option, if you're using Windows Vista, is to use Microsoft's Ximage and Sysprep tools. I considered these for my hard disk upgrade, on a machine that had Vista (no Service Pack) installed. However, I found the process to be overly complicated. It also required installing the WAIK (Windows Automated Installation Kit), which for some odd-reason, can only be installed via a downloaded .iso image, which requires a blank DVD to burn the image. As it turns out, I didn't have a blank DVD available, and needed to get the upgrade completed ASAP. This will usually not be a constraint, however, I also found the rest of the Ximage/Sysprep process to be overly involved. This lead me to Linux, and dd.

To clarify, using Linux and dd does not require you to have Linux installed on any computer, or have some dual-boot configuration on the computer on which you're going to install the new hard disk. What you do need is a linux boot disk (CDROM). Of course, this presents a similar requirement to the WAIK .iso image, but as it turns out, I had several flavors of linux boot disk laying around! For this project, I used an old SuSE linux 8.1 boot disk, because it quickly found the correct drivers for all of my hardware. However, you should be able to similarly complete this process with most types of linux boot CDROMs. It should be noted, however, that you may have trouble with certain distributions not recognizing all of you hardware without manual intervention. Thus, trying a couple distributions may be needed (as it was for me). The important hardware to be recognized would be a network card (if you don't have a spare disk in the computer you are upgrading), and possibly a disk controller (e.g. SCSI) if your hardware uses such a card. In my case, the upgrade involved starting with a 36GB SCSI disk drive, and adding a second identical SCSI drive, making a RAID 0 array. Thus, the old (source) disk, was also part of the new (target) configuration. This meant that I could not directly copy my old hard disk contents to the new hard disk, but instead needed to make a temporary copy of the disk, and then write that copy to the new RAID array. So, this represents one of the most complex disk upgrade scenarios, which is why I'm describing the technique here!

The basics steps are as follows:

1. boot the system with the linux boot CDROM
2. mount a network drive with enough space to store the (compressed) contents of my old disk
3. use gzip and dd to copy the disk to the network location
4. install the new disk in the computer chassis
boot the system with the linux CD again
5. use gzip and dd to copy the image from the network location, to the new disk
6. resize the disk partition to take advantage of all the new disk's space

Backup

linux>> fdisk -l /dev/sda

Disk /dev/sda: 255 heads, 63 sectors, 4462 cylinders
Units = cylinders of 16065 * 512 bytes

   Device Boot     Start      End    Blocks    Id  System
/dev/sda1              1       16    128488+   83  Linux
/dev/sda2   *         17     4462  35844717     7  HPFS/NTFS

This particular disk has two partitions ... a linux partition used to hold my bootloader (GRUB), and the Vista (NTFS) partition. Next, we need to mount a network drive with space for the backup:

linux>> mkdir /mnt/backup
linux>> ifconfig -a
eth0      Link encap:Ethernet  HWaddr 00:08:74:4F:2A:C6
          BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:100
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)
          Interrupt:11 Base address:0xdcc0 Memory:ff6e0000-0

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)

linux>> ifconfig eth0 192.168.0.10 netmask 255.255.255.0 up
linux>> ping 192.168.0.44 
PING 192.168.0.44 (192.168.0.44) from 192.168.0.10 : 56(84) bytes of data.
64 bytes from 192.168.0.44: icmp_seq=1 ttl=64 time=0.377 ms
64 bytes from 192.168.0.44: icmp_seq=2 ttl=64 time=0.296 ms
64 bytes from 192.168.0.44: icmp_seq=3 ttl=64 time=0.135 ms

--- 192.168.0.44 ping statistics ---
3 packets transmitted, 3 received, 0% loss, time 1999ms
rtt min/avg/max/mdev = 0.135/0.269/0.377/0.101 ms

linux>> mount -t nfs 192.168.0.44:/export/path/ /mnt/backup

We used ifconfig to configure the ethernet card to communicate on the same subnet as the server where we'll store the temporary disk image.

linux>> dd if=/dev/sda conv=sync,noerror bs=64k | gzip -c > /mnt/backup/sda.img.gz 
560057+1 records in
560058+0 records out

Restore

linux>> mkdir /mnt/backup
linux>> ifconfig -a
linux>> ifconfig eth0 192.168.0.10 netmask 255.255.255.0 up
linux>> ping 192.168.0.44
linux>> mount -t nfs 192.168.0.44:/export/path/ /mnt/backup
linux>> gzip -cd /mnt/backup/sda.img.gz | dd of=/dev/sda
560058+0 records in
560058+0 records out
linux>> fdisk -l /dev/sda

Disk /dev/sda: 255 heads, 63 sectors, 8836 cylinders
Units = cylinders of 16065 * 512 bytes

   Device Boot     Start      End    Blocks    Id  System
/dev/sda1              1       16    128488+   83  Linux
/dev/sda2   *         17     4462  35844717     7  HPFS/NTFS

Final Steps

At this point, you can reboot your computer, without the linux boot disk. You should be able to get back into Windows, just as before. When, you first open My Computer, you may notice that you seem to have the same sized hard disk as before the upgrade. Presumably, you actually added a larger hard drive. This is not a mistake. You backed up the entire hard disk, which include the partition table that specifies the size of the disk partitions. So, at this point, you probably have a bunch of unused space on your new hard disk (equal to the amount by which the new disk is bigger than the old one). You can either use the linux boot CDROM to resize the partitions (even if they are Windows partitions), or probably easier, just use the Windows Computer Management application.

To use Computer Management, select Start -> Run and type in compmgmt.msc, or navigate to Programs -> Administrative Tools -> Computer Management. From within Computer Management, you can select Storage -> Disk Management. From here, you can select the drive you added (e.g. C:\), right-click and choose to extend the partition. Windows should detect the total size of the disk, and let you extend the partition to fill up the entire disk, which is generally what you would want.

Issues

• Some linux boot CDs may not allow you to write to /mnt. You may need to mount your network location to /tmp/some_folder_name/, instead of /mnt.
  linux>> mount -t nfs 192.168.0.44:/export/path /tmp/some_folder_name
• My upgrade was done on a 36GB Vista disk that was 90% full. It compressed down to a 13GB image (sda.img.gz) file on my network server. If you don't have room to store even a compressed disk image, you can take advantage of the fact that your source disk probably wasn't 100% full. The above process will blindly copy unused disk space, but you can minimize that impact by first writing a file of all zeros to the entire unused portion of the source disk. Then, upon compressing/copying the disk, all those zeros will be compressed down to almost nothing (from "zeros" to "almost zero"!). This will save you a little more space on your network drive.
  

  linux>> mkdir /mnt/source_disk
  linux>> mount -t ntfs /dev/sda2 /mnt/source_disk
  linux>> dd if=/dev/zero of=/mnt/source_disk/zeros && rm -f /mnt/source_disk/zeros

• If the old disk will not be part of the new configuration, you may be able to avoid the intermediate storage on the network server. If you have space in your computer chassis for the new and old disks at the same time, you can copy the old image directly to the new disk, without the network copy. This will obviously be faster, and doesn't require the network storage, or even the activation of your network card (NIC)
  linux>> dd if=/dev/sda of=/dev/sdb conv=sync,noerror bs=64k
• If you are cloning from one machine to another (identical) machine, you can clone disk images right over the network. On the destination machine, boot into linux with a boot disk (USB drive or CD/DVD), and use the netcat command, piped into gzip and dd:
  linux-1>> nc -l -p 9000 | gzip -cd | dd of=/dev/sdb bs=64k
  Then, on the source machine, also booted into a linux rescue system, use dd, gzip, and netcat in the opposite direction:
  linux-2>> dd if=/dev/sdb conv=sync,noerror bs=64k | gzip -c | nc 192.168.0.10 9000
  assuming that the source and target machines have been booted into rescue systems with networking enabled, and the target has had its IP address set to 192.168.0.10.
• If you are restoring the disk image to a disk of the same size (maybe you just want a newer/faster disk), you may get a message from dd about "No space left on device", at the end of the final restore step. I got this message (when I tested the process restoring to the exact same disk), and yet the process appears to have worked successfully. Your guess is as good as mine why dd complains about this. It may have to do with the final (64k) block having some extra padding in it that runs just past the end of the disk.

Final Thoughts

So, there you have it. In a limited number of steps, you have made an exact copy of an existing Windows installation, and laid down everything on the new disk as it was on the old. You don't need to manually reinstall the OS, restore a backup, and reinstall applications individually. Since, this is the same computer, you've avoided any unnecessary license repurchase scenarios. Long-live linux!