Encrypted filesystems
Linux offers several ways to establish a transparently encrypted filesystem.
That means a hierarchical subtree of directories and files, such that all data written to
it gets encrypted just before being written, and all that is read
from it gets decrypted just after being read. The whole time in between (after encrypt but before decrypt)
the data sits on the disk in encrypted form. It
is never disk-resident in the clear. That's the benefit.
Accomplishing this involves inserting an additional processing step or extra layer in the chain of activities performed to drive the disk. That might mean using an modified version of the regular disk driver but usually means preserving it unaltered while cascading an additional driver together with it. Utilities (ls, cat, et. al.) that would normally talk to the regular disk driver talk to the extra one instead, which in turn passes through to the regular one. Any special processing of the written datastream (encrypting it, splitting it for striping,...) can be done in this additional "predriver." Since disk drivers get auto-selected depending on what part of the tree you're writing to, by mounting a particular subtree in association with your predriver the predriver gets its hands on the all data heading for the subtree. It can then call the regular driver to actually inscribe onto the disk, but handing it an encrypted version of what the utility handed it. This happens automatically subtree-wide.
The traditional technique is sometimes called "cryptoloop" because it uses the so-called loop device. Its centerpiece command is losetup. More recently, with the 2.6 version of the kernel, the device mapper was introduced. It too embodies this concept of driver chaining. Encryption is one of its applications, though there are others. Device mapping is more general-purpose and tends to replace cryptoloop. Its central command is dmcreate. To simplify the use of dmcreate for the specific purpose of encryption, the cryptsetup command was created. Then there are dedicated opensource projects like EncFS and Truecrypt.
cryptoloop
losetup
device mapper based
dmsetup
cryptsetup
EncFS home download sources
VeraCrypt (successor to defunct, popular truecrypt)
Often a mounted subtree corresponds to a physical device, and often it might be
a removable one (floppy, USB stick). If one of these is plugged into a Windows
system, software can be found in most cases to use them
there. Truecrypt has versions for both platforms. cryptoloop and device mapper
volumes can be read in Windows by FreeOTFE.
The cryptsetup exercise to perform
For this exercise you will set up a container as an encrypted device using cryptsetup. You will write specific material to it. Use a scratch USB flash drive. That means one with no data on it you care about. This exercise is data-destructive, but as long as you use a scratch drive you don't care. So use a scratch drive.
Flash drives nowadays are inconveniently big for this purpose. They make this exercise needlessly time-consuming. So rather than let the drive itself serve as our container, let' make a smaller container. Namely, a partition on the drive whose size we can control.
Identify the name by which your system refers to the flash drive device. Upon inserting it:
dmesg
will bring up some kernel log messages that will reflect the kernel having detected the insertion, and will give the drive a name. It might be "sdb" for example. As a second method to see the drive and learn its name:
fdisk -l
It should show the drive as "/dev/sdb", if dmesg showed "sdb". It will also show the other drive device(s) in the machine, and their capacities. Your hard drive might be "sda". Look at the reported drive sizes and make sure. If you think the flash drive is supposed to be "sdb" for example and you have a 16GB flash drive, ascertain that sdb's reported size is indeed 16GB. Do not mis-identify the flash drive's name, because you will take actions that destroy data and don't want to accidentally destroy what's on the main hard drive, or any other than the scratch USB.
Now let's make a partition of reasonable size for this tutorial. How about 10MB:
parted /dev/sdb mklabel msdos
parted /dev/sdb mkpart primary 10 20
A storage drive like your USB is a data container. A partition is also a data container. A file is also a data container. The data contained on a drive can be any data, and could include a partition. That's what we have here now. The partition contains data, but it is randomly whatever was sitting there where we have defined the partition, already, before we defined it. Look, for example, at the first 512 bytes in your partition:
dd if=/dev/sdb1 bs=512 count=1 2> /dev/null | xxd -g1
You should be looking at what is fondly called garbage, just some random stuff. Let's put some recognizable data of our choosing there, filling up the whole 10MB partition with "roses are red violets are blue".
while true; do echo -n "roses are red violets are blue "; done | dd
of=/dev/sdb1
then sample the partition's first 512 bytes again to see if it worked:
dd if=/dev/sdb1 bs=512 count=1 2> /dev/null | xxd -g1
WITHOUT ENCRYPTION
Now that we have a data container to work with, we want to compare putting data in it unencrypted versus encrypted, before and after. I choose to put files into my data container, i.e. my partition, as opposed to just putting non-file data into it. To do that you first need a filesystem, because you can't write files onto a disk or other device (directly). That's right. You can only write them into filesystems (note the "file" in "filesystem!). And while a filesystem might reside on a disk, a case in which the file would transitively land on that disk, that is incidental (ram disks place/write filesystems into ram, which then along with its contained files are not on any disk). So far, you have a partition but want to write files so need a filesystem which you don't have (yet). So you make a filesystem by using the mkfs command.
mkfs -t vfat /dev/sdb1
It took over and wrote stuff of its own at the beginning of the partition:
dd if=/dev/sdb1 bs=512 count=1 2> /dev/null | xxd -g1
There's something other than "roses are red" on there. Now you have a filesystem. Actually probably two, and this is your second. You are already running linux so you have the familiar filesystem with directories like /, /home, /etc, /var/log, etc etc. You want to integrate the other one which you just created on the outside device, into this first one as a subtree. Get a place within the first filesystem at which to do so:
mkdir -p /tmp/mountpoint
and integrate/attach/incorporate the second filesystem into the first at that point:
mount /dev/sdb1 /tmp/mountpoint
Now the new filesystem in within reach. You can write and read files on it. That happens implicitly, just by the act of writing or reading the /tmp/mountpoint subtree. Whenever you write to that subdirectory your data goes onto the partition in your USB drive.
echo hello > /tmp/mountpoint/hello.txt
echo goodbye > /tmp/mountpoint/goodbye.txt
Let's verify and retrieve what you just stored:
ls -l /tmp/mountpoint
cat /tmp/mountpoint/hello.txt
cat /tmp/mountpoint/goodbye.txt
You have made your point. You created a couple of files in a filesystem in a partition on your USB flash drive. You are done. So electronically de-connect the flash drive from the main filesystem (leaving it plugged in though):
umount /dev/sdb1
The flash drive, and the filesystem within it, are now detached from the main filesystem, unavailable till next remounted back into this filesystem or some other. But whether accessible or not, the file named "hello.txt" containing the word "hello" is in your physical device. Demonstrate that to yourself:
grep -E " hello|goodbye" /dev/sdb1
or try
grep -a -E --color=auto "hello|goodbye" /dev/sdb1
Yep, it looks like "hello" and "goodbye" are there within the partition, natively in the clear.
WITH ENCRYPTION:
Now I want you do pretty much the same thing all over again with one added ingredient-- encryption. As a preliminary step we will make the partition be all-encrypted. Later, after having written the hello and goodbye files into the encrypted filesystem then detached (umount'ed) it, we should be unable to find "hello" and "goodbye" within the underlying raw storage partition. They will be there, but in a different form having been encrypted before being laid down.
cryptsetup luksFormat /dev/sdb1 <--- be sure to type YES in uppercase when prompted for it; then give "passphrase" as the passphrase when prompted
Look at the beginning of the partition to see what cryptsetup's luksFormat action did:
dd if=/dev/sdb1 bs=512 count=1 2> /dev/null | xxd -g1
Note cryptsetup asked you for a passphrase. It uses your response to generate an encryption key. It will encrypt whatever it writes with that key. You'll put your partion into service, activate the encrypting software driver, when you perform cryptsetup's luksOpen action. When you take the partition out of service (via cryptsetup's luksClose action) you're later going to want to re-gain access to the data. You'll do that via cryptsetup's luksOpen action. At that time, you'll need to supply the original passphrase, in order that cryptsetup will have the wherewithal to decrypt.
Now let's perform cryptsetup's luksOpen action and see a before-and-afer:
ls /dev/mapper
cryptsetup luksOpen /dev/sdb1 mickey
ls /dev/mapper
There is a new entity /dev/mapper/mickey. You can treat it as a data container and use it syntactically anywhere you might use the name of a device or partition. This one maps whatever's written to it onto /dev/sdb1, but encrypts it all as it passes through.
From this point you will do the same things you did above on /dev/sdb1 but substitute instead /dev/mapper/mickey.
mkfs -t vfat /dev/mapper/mickey
dd if=/dev/mapper/mickey bs=512 count=1 2> /dev/null | xxd -g1
mount /dev/mapper/mickey /tmp/mountpoint
echo hello > /tmp/mountpoint/hello.txt
echo goodbye > /tmp/mountpoint/goodbye.txt
Let's verify and retrieve what you just stored:
ls -l /tmp/mountpoint
cat /tmp/mountpoint/hello.txt
cat /tmp/mountpoint/goodbye.txt
umount /dev/mapper/mickey
Look to see if "hello" and "goodbye" are within the virtual device "mickey":
grep -E " hello|goodbye" /dev/mapper/mickey
grep -a -E --color=auto "hello|goodbye" /dev/mapper/mickey
They're there. But look for them similarly within the physical device /dev/sdb1:
grep -E "hello|goodbye" /dev/sdb1
grep -a -E --color=auto "hello|goodbye" /dev/sdb1
They're not there. Not in their native clear form, but in an encrypted form they are present. Note you did not explicitly encrypt (or decrypt) anything.
Now clean up, do away with the virtual device mickey:
cryptsetup luksClose mickey
ls /dev/mapper
The data remain available in encrypted form, on/in the USB drive, and can again be retrieved in plaintext if you again luksOpen and again produce the passphrase. You could do that on the same computer, or a different one.
What to submit
When done, having fully umounted and luksClose'd, please submit your partition to the instructor in your assignments directory on the class server. Of course, you can't transfer the partition itself. So make a copy of it in a file and transfer that, like this:
dd if=/dev/sdb1 of=~/myencryptedpartition-<your last name>
Now transfer ~/myencryptedpartition-<your last name> to your assignments directory on the class server.
(note to moi: generalize to alternatively encompass whole-device or file, not only partition as target container)