Linux: Low-level Data Copying with dd

When you use Linux, you know there are always multiple ways to solve a problem.

Take, for instance, the need to create backups. Yes, you could opt to use one of the many backup solutions (such as Bacula, rsync, Deja Dup, Fwbackups, Backupninja, Simple Backup Suite, Kbackup, BackupPC …  the list goes on and on). You’ll find command line tools, GUI tools, and even some web-based backup solutions.

But when you need low-level data copying (such as to create a partition or full disk copies), one tool never fails to rise above the rest. That tool is the `dd`command, which stands for data duplicator.

With dd, you can create a byte-for-byte copy of a partition or a drive and even convert data between files or devices. The `dd`  command allows you to control block size, and skip and seek data.

Needless to say, dd is very powerful. In fact, this command is so powerful that you really need to exercise caution when using it. Run the command improperly and you could wipe out an entire drive of data.

One thing dd is good for is copying a Linux installation from one disk to a larger disk. This is a good option when you have a server or desktop with a drive that’s full. You can even create a bootable USB drive from an ISO, thereby avoiding having to use third-party software for the task.

Let’s take a look at some examples of how the dd command is used. Remember, this command is very powerful, so I would highly recommend you test it on a non-production machine before you take on the important task of migrating an installation from one drive to another.

What You’ll Need

To make use of dd, you’ll need the following:

• A running instance of Linux (it doesn’t matter the distribution).
• A user with sudo privileges.
• A drive that’s larger than the one being copied.

That’s it. Let’s get to work.

dd: The Basic Syntax

The basic syntax of the dd command can be in one of two forms:

• dd [OPERAND]
• dd OPTION

With that in mind, let’s take a look at some examples.

Create a Bootable USB

Because it’s less likely you’ll do damage by creating a bootable USB device, we’ll take a look at that example first. Let’s say your ISO image is AlmaLinux-9.3-x86_64-dvd.iso and the USB drive you want to use is located at /dev/sdg.

It’s very important that you know the exact path of your USB drive. If you’re not sure, you can locate it with the lsblk command.

To create this bootable USB drive, the command would look something like this:

sudo dd if=/home/jack/Downloads/AlmaLinux-9.3-x86_64-dvd.iso of=/dev/sdg bs=4M status=progress && sync

A bit of an explanation for the above:

• if= This is the input file or the ISO file you want to use for the bootable USB device.
• of= This is the output file or the location of the USB drive.
• bs=4M This defines how many bytes will be read and written to (the default is 512).
• status= This is the level of information to print to the output. In this case, progress shows periodic transfer statistics.
• sync This makes certain all data is written to the USB device before the process finishes.

Converting Files

Before we continue, there’s a really cool feature found with dd that allows you to convert text. For example, you might have a file comprised of all lowercase text and you want to convert it to all uppercase. Say the file in question is called “testing” and you want to create a new file called “testing2” that is the uppercase version of the original. To do that, the command would be:

dd if=testing of=testing2 conv=ucase

The possible options you can use with conv include:

• ascii Convert from EBCDIC to ASCII.
• ebcdic Convert from ASCII to EBCDIC.
• ibm Convert from ASCII to alternate EBCDIC.
• block Pads newline-terminated records with spaces to cbs-size.
• unblock Replaces trailing spaces in cbs-size records with newline.
• lcase Converts upper case to lowercase.
• ucase Converts lower case to uppercase.
• sparse Tries to seek rather than write all-NUL output blocks.
• swab Swaps every pair of input bytes.
• sync Pads every input block with NULs to ibs-size; when used with block or unblock, pad with spaces rather than NULs.
• excl Fails if the output file already exists.

Cloning Disks with dd

Finally, we’re going to take on the meat and potatoes of the dd command … disk cloning. Remember, you’ll need a drive that’s  bigger than the one you’re cloning. Cloning with dd creates an exact, byte-for-byte copy of a drive or partition, such that the new drive is identical to the source. This is great when you suspect a drive is failing and you want to salvage it before it’s too late.

To do this, you’ll want to connect the new drive to the system that contains the drive to be cloned. Let’s say, for example, the source drive is /dev/sda and the destination is /dev/sdb.

What you first need to do is boot the machine using either a rescue distribution or any live version of a Linux distribution. The reason you must do this is that both source and destination must be unmounted (and you can’t unmount a drive that’s in use).

Once you’ve booted the live distribution, connect the destination drive and clone the source with the command:

sudo dd if=/dev/sda of=/dev/sdb bs=4M status=progress

The process will take some time, so exercise your patience here. When the process completes, reboot the machine, making sure to remove the USB device.

To verify everything works, you should open the boot menu of the machine and select the destination drive as the boot source. If everything works out fine, you could then remove the source drive, and insert the designation drive, and all should be good to go.