Linux

Philips webcam viewer program

This little utilty is used to display images grabbed from a philips webcam under linux. It was designed to be fast, using MIT X Shared memory extension and Xlib. It also dumps ppm files.

Keybindings are: 'd' for dump, 'g' to switch black and white or color mode, 'q' to quit.

You will need several libraries and includes to compile, for example:

Compile with
gcc -O3 -o webcam webcam-xlib-shm.c -I/usr/src/linux/drivers/usb/media/pwc -L/usr/X11R6/lib -lX11 -lXext

warning: include dir in new pwc (free version 10) is /usr/src/linux/drivers/usb/media/pwc but was not the same in version 9.

Download webcam-xlib-shm.c   (01/2005)

You have two philips cameras and you want to display them at the same time ? (this does not work with some versions of linux and pwc, kernel bugs...)

Download double-webcam-xlib-shm.c   (11/2004)

0 comment


Laptop and power tweaks for Linux

To improve time that you can spend using the battery before it is discharged, it seems quite obvious to improve the operating system so that it does not consumes power for useless purposes. You can find most important tweaks that wiki page if you haven't done anything yet. On this page, you will find my more precise advices.

There are two things that consumes power on a computer that we can modify: interruptions and devices use. Interrupts are triggered both by the kernel and the system. Most of them are due to timers, so when a lot of process are launched it's more likely that you have lots of timers because programs generally have sleep loops. It's a good thing to avoid those programs if you don't need them, like cron or media players.

On this web page I draw a list of what I did to save power on my system, which is a EeePC 900, but it can be applied to most systems that have constraints on power or flash drives. With all the tweaks, I extented my battery life from 2 to 3 hours roughly, but it mostly depends on what application you use. When programming or typing text, battery can last a lot more longer, if you don't compile too often. Heat is reduced, system longevity is extented.

Embedded devices unpowering

Unused devices should obviously be powered down, but sometimes people don't even know what's inside computers, so they can't do it.

First of all, you can look into the BIOS in the section 'integrated peripherals' what it offers to activate or deactivate. Probably there will be USB controllers, with possibly a webcam and a card reader using it, the sound chipset, the embedded loudspeakers, and network interfaces. If you don't need one of those, I suggest you deactivate it here.

Then, where you are running the system, there are devices that can be put in sleep mode or completely deactivated:

Kernel tweaking

Virtual memory is an important cause of processor wake-ups. When you write some information into memory, it is stored into the system's memory for a while, the cache, and then copied on drives. If that time is too low, drives will be used very often. It is better to write a lot of information at a time but less often, so that the CPU can sleep longer.

An other parameter of virual memory is the ratio of dirty memory achieved before copying it back to the drives. When the ratio is low, it means that when a few megabytes have been modified, the write is triggered. When it's high, the write back happens less often. Once again, power saved. But you should be warned: if the system goes down unexpectedly, you will lose all data that has not been stored during this waiting cache time.

sysctl controls the values for time and ratios of write back for the virtual memory susbsystem. Put the following lines in /etc/sysctl.conf, and run # sysctl -p to take it into account. It will be loaded at system startup.

vm.dirty_writeback_centisecs = 3000
vm.dirty_background_ratio = 10
vm.dirty_ratio = 45

The difference between dirty_background_ratio and dirty_ratio is that the first is triggered as a background task, and the second blocks the systems while the dirty memory hasn't been written back. More explanations here.

System tweaking

A minimalistic system, like Arch Linux, is the better way to control all services installed and launched by the operating system. Of course, try to not launch services you don't need. You can also configure the ACPI rules to launch power eating services when you plug the AC adapter to your laptop. This is what I do with cron. cron is waking up the CPU every minute at least, and running some tasks for system maintenance, like updating man pages of find database.

Filesystem options can reduce the number of writes on the drives, which can be a good thing to do with flash memories. Moreover, it reduces the time spent working for the CPU and the northbridge.

The first thing that can be done is disable access times saving. When you access a file or a directory, they usually save the date when it was accessed. If you don't need that feature, remove it and you will experience faster file access and reduced power consumption. It is done by adding the options noatimes and nodiratimes to the options of the filesystems in /etc/fstab. For example:
/dev/sda1 / ext2 defaults,noatime,nodiratime 0 1

Using journalling filesystem is also not recommended, since it will write very often on the same block of the drive. It is bad for flash memories.

An other improvement can be added by the use of memory filesystems. Indeed, some directories are not specially useful to keep on the drive, and still, they may suffer from lots of writes and access. I converted three directories of my system to tmpfs, which uses the system memory to save the files. It is faster, less power greedy, but files are lost on reboot. Anyway, I don't really need them, here is my fstab:

tmpfs /var/log tmpfs size=2M 0 0
tmpfs /tmp tmpfs defaults,noatime 0 0
tmpfs /var/cache/pacman/pkg tmpfs defaults,noatime 0 0

Temporary files of softwares: you may not know it but some softwares you use everyday keep track of what you did, to provide history or better user experience. It is the case of less and vim. You can disable the less history file by setting the environment variable: LESSHISTFILE="-" in /etc/profile or something similar. For vim, you can disable the viminfo file, the backup file~ and the swap files by setting:

set viminfo=""
set noswapfile
set nobackup
in your ~/.vimrc or /etc/vimrc if you don't need them.

Run powertop

The best tool you have to try to hunt power leaks is powertop. It acts as the command top does for CPU but for interrupts. You can see what application and kernel thread wakes up the CPU. It also suggests some system tweaks if you don't have done them already. On the powertop website there are some tips and tricks for some known applications too.

XTerm for examples wakes up every second by default, due to session management. It can be fixed by putting xterm*sessionMgt: false in your ~/.Xdefaults.

You will see how much using a media player wakes up the CPU. And they consume even more power since they use the sound or video chipset.

Using a mouse is also very interrupt consumming. You maybe should use a tiling window manager such as awesome, which are optimized for keyboard use.

0 comment


Installing loop-aes, LVM2, and debian on a VIA C7

I have been looking for a low power consuming computer which could replace my old big home server, and I found mini-itx boards based on the VIA C7 processor. This little thing, especially in its eden version (like the 1.2GHz fanless), requires very low power to run (like 6W max, 0.1W average), and has an hardware cryptographic engine built-in, the VIA Padlock. It can compute SHA-1, SHA-256, AES128, AES192, AES256, has a hardware random number generator, and instructions for the exptmod for RSA for example. The board I use in this tutorial/howto is a Jetway J7F4K1G2E (don't buy it, it's a swindle, it needs a fan, and if you enable ACPI it freezes). And I made this box for it. You may also be interested in the AMD Geode LX, which also supports hardware acceleration of AES128 and a random number generator, but it has less processing power. The C7 has been improved to support 64-bit instruction set and multi-core operation, packages in the Via Nano CPU series.

With all this good stuff, I tried to create a very secure system, with hard drive encryption and others. With experience in no-space-left-partitions I already had, I also wanted it to run LVM2 (logical volume manager), which allows to resize partitions at runtime.

Why loop-aes? I could have installed directly debian with LVM on dm-crypt, because the installer supports it, but I've seen that loop-aes was the only secure hard drive encryption method here, so I decided to do it with loop-aes. This actually was a mistake because this document dates from 2004, and now (at the end of 2008), dm-crypt is one of the few most secure disk encryption software using XTS (since Linux-2.6.24). So I should reconsider moving from loop-aes to dm-crypt-LUKS, although loop-aes remains quite secure. If I understand right, if the attacker has only one physical access to the drive he won't be able to get any plaintext information. See links for more information on this, and all kinds of documentation on the subject. With LUKS, it seems that the encryption keys are stored locally on the hard disk drive, which is not required in loop-AES.

So here we go, installing debian on reiserfs file systems on LVM2 partitions on loop-aes AES256 encryption loop with GPG multi-access keys (this is the best protection) on a SATA hard drive, with benefit of the VIA Padlock engine of course, and in the most secure way. It wouldn't be funny if only a part of the hard drive was encrypted, so I did not even made a partition, everything is encrypted, so it has to be mounted from elsewhere... like a bootable USB key. Actually, this is more than a funny thing: by separing the key from the drive, a bad person stoling the drive wouldn't have the keys. This makes 3 things to have to actually access the files: the drive, the encryption keys, and a passphrase protecting the keys. Or just one: an access to the system running...

I chose debian because I'm familiar with it, but this can probably be applied to any Linux distribution. This can also probably be done by network boot with PXE, but I don't know how that works. See antlinux Naranja web page (in links).

Here are the steps needed set up and running this system.

1. Install debian to get the base system installation files

You might say that there is a tool called debootstrap which can install the debian's base system from the Internet, but I prefered copying files. Debootstrap has lots of dependencies, and I find this is not nice for an install from an initrd or initramfs (the install USB key).

So, install your debian on a system, in one partition or whatever, install rsync, openssh-server, lvm and reiserfs tools on it, or whatever file system you chose. You can also uninstall the default kernel, and grub, and maybe try to boot the kernel you will configure in step 2. Then rsync -a this_system:/ backup_dir to get files. Remove /etc/mtab and /etc/fstab. Create a tar.gz file from the files (cd backup_dir && tar zcvf ../base_install.tar.gz *).

2. Compile a Linux kernel optimized for the VIA C7 including loop-aes

We will use a kernel supporting the C7 hardware random generator and the loop-aes for the setup, this will make it really faster. As we have to configure and build a kernel, we will make the final one, with all VIA options, and those for the boot initrd.

Download the latest kernel, I think it works quite fine since version 2.6.22.

Download the latest loop-aes software. You can find it on the loop-aes website.

2012 update: loop-aes v3.6e works fine with the 3.2 stable kernel branch.

Patch your kernel:

	cd linux-2.6.23.12
	rm include/linux/loop.h drivers/block/loop.c
	patch -Np1 < ../loop-AES-v3.2b/kernel-2.6.23.diff

Enter the kernel's configuration. Select C7 for the processor, enable the hardware random number generator, loop-aes device with padlock hardware handling, padlock engine in cryptographic engines. For the initrd, you will need the ramdisk support, initrd, usb disk support, minix, vfat, tmpfs procfs and host (reiserfs) filesystems. Here is the real list:

initrd specific:
	CONFIG_BLK_DEV_RAM=y CONFIG_BLK_DEV_RAM_SIZE=4096
initramfs or initrd specific:
	CONFIG_BLK_DEV_INITRD=y
USB disk specific:
	CONFIG_SCSI=y CONFIG_BLK_DEV_SD=y CONFIG_USB=y CONFIG_USB_STORAGE=y
	CONFIG_USB_EHCI_HCD=y CONFIG_USB_UHCI_HCD=y
File systems for boot disk:
	CONFIG_MINIX_FS=y CONFIG_PROC_FS=y CONFIG_TMPFS=y CONFIG_CRAMFS=n
	CONFIG_FAT_FS=y CONFIG_MSDOS_FS=y CONFIG_VFAT_FS=y CONFIG_NLS_CODEPAGE_437=y
	CONFIG_NLS_CODEPAGE_850=y CONFIG_NLS_ISO8859_1=y CONFIG_NLS_ISO8859_15=y
	CONFIG_FAT_DEFAULT_CODEPAGE=850 CONFIG_FAT_DEFAULT_IOCHARSET=iso8859-15
Host file systems:
	CONFIG_BLK_DEV_DM=y CONFIG_REISERFS_FS=y
VIA C7 specific:
	CONFIG_MVIAC7=y CONFIG_HW_RANDOM_VIA=y CONFIG_CRYPTO_HW=y
	CONFIG_CRYPTO_DEV_PADLOCK=y CONFIG_CRYPTO_DEV_PADLOCK_AES=y
	CONFIG_CRYPTO_DEV_PADLOCK_SHA=y
loop-aes + padlock specific:
	CONFIG_BLK_DEV_LOOP=y CONFIG_BLK_DEV_LOOP_AES=y
	CONFIG_BLK_DEV_LOOP_PADLOCK=y
And for even less power consumption:
	CONFIG_CPU_FREQ=y CONFIG_CPU_FREQ_GOV_PERFORMANCE=y
	CONFIG_CPU_FREQ_GOV_POWERSAVE=m CONFIG_CPU_FREQ_GOV_ONDEMAND=m
	CONFIG_X86_E_POWERSAVER=y (C7 specific)
	CONFIG_TICK_ONESHOT=y CONFIG_NO_HZ=y

Initrd or initramfs? I chose initrd in the first part of tutorial because I was not familiar with initramfs, but the last part (the boot USB key) is made with initramfs. When we will boot on the real system, as the loop will be mounted in the initial ramdisk, we will never be able to unmount it. Using initrd implies that an allocated /dev/ram0 (about 8MB) will stay in RAM forever, which will be lost space. With initramfs, the allocated space is the same than needed by files, thus we remove files from the initramfs before actually booting the real system, and no memory is lost. The other point is that initramfs is easier to build and tune than an initrd, because it does not needs mount privileges, it just takes real files into an archive, so you can make lists of what you want or don't in it. So you will have in this tutorial the two ways for doing a bootable kernel with initrd or initramfs. If you choose initramfs (what I recommend), you can put CONFIG_BLK_DEV_RAM=n.

Don't forget networking support, with CONFIG_R8169 for the Realtek of the J7F4 boards. SATA and PATA support with VIA chipsets, and you may also use sensors to see voltages and temperatures of the system brought by CONFIG_I2C_VIAPRO=m CONFIG_SENSORS_EEPROM=m CONFIG_HWMON=m CONFIG_SENSORS_F71805F=m.

You can also simply download my config file here.

Save your config file, run make and make modules_install, and you're done with the kernel. Maybe the best thing would be to do this step inside the base system of the previous section on the target system, and check that the kernel you configured is working correctly, and that would install directly modules at the right place. Else, you should copy the /lib/modules/2.6.23.9 directory to base installation files or to the USB key on which you will put the base_install.tar.gz), if you want network support and if it's not built into the kernel, it can help.

3. Create an installation USB boot key

We will now create an USB boot disk which will be used to create the loop-aes, the encryption keys stored with GPG, the LVM, the filesystems and copy the base debian install.

3.1. Compile specific software

Standard tools don't handle loop-aes and mounting encrypted file systems correctly, so we will have to patch and recompile them statically, which is easier for embedding in an initrd/initramfs.

You can download my files for this usb key image, which contains patched binaries of softwares if you don't want to create everything or compile and configure your kernel here.

losetup, mount, umount. The maintainer of project Naranja told me that debian's binaries were already patched to support loop-aes -- well, I just verified this by updating my ramfs (december, 2 2008), and losetup is not working with the options we use in the boot script, you have to patch it. You can then use them directly (mount and umount) instead of patching them and compiling them by hand. See the loop-aes README for information about that.

gpg. You can either add a line to the gpg.conf file or recompile gpg to provide better resistance against dictionary attacks, by changing the default amount of iterations. This will slow down key generation and dictionary attacks.

So you can add this line to /etc/gnupg/gpg.conf: s2k-count 8388608. You can also use the command line parameter --s2k-count 8388608. This can be better, as we will try to get the lowest number of files in our initrd. You can also patch it the same way we patched util-linux, the patch is in loop-aes archive. The advantage is that you can make a static build, which does not need any shared libraries to be installed on the image. To do so, use: CFLAGS="-O2" LDFLAGS="-static -s" ./configure --prefix=/usr --enable-static-rnd=linux. Then copy gpg to /bin of the image.

See the loop-aes README for more information on softwares needed to be compiled.

3.2 Create the ram disk image (initrd)

We now create the init ram disk, which will actually contain a little OS file system.

Create the disk image with dd; 16M is probably enough for everybody, but depending on what you really want inside it can be larger. Mine was around 11M used.
Then create a file system in it, and mount it.

Creating the initrd image file
dd if=/dev/zero of=image bs=1M count=16
mke2fs -F image
mkdir mnt
mount -o loop image mnt/

Now we can put in it all the tools we will need. I recommend using busybox for basic utilities like sh, ls, mkdir, cp and so on. Some files are also needed for the system to boot and run correctly, like devices /dev/console. Create all devices you will like to use, and for all binary you put in the image, you will have to copy dependent shared libraries too. You can see them with the command ldd. To make it easier, here is my file list:

initrd's files for setup and rescue key
.:
bin   dev  initrd  mnt      proc  sbin  tmp  var
boot  etc  lib     newroot  root  sys   usr

./bin:
[         chown    env     hostname  mknod  reset    su        true      whoami
[[        chroot   false   ifconfig  more   rm       sync      umount    xargs
aespipe   cp       fgrep   kill      mount  rmdir    syslinux  uname     yes
ar        date     find    ln        mt     sed      syslogd   uniq      zcat
ash       dd       getopt  loadkeys  mv     sh       tail      uudecode
awk       df       gpg     loadkmap  patch  shred    tar       uuencode
basename  dirname  grep    login     pidof  sleep    test      vi
busybox   dmesg    gunzip  losetup   ping   sort     time      wc
cat       du       gzip    ls        ps     strings  touch     wget
chmod     echo     head    mkdir     pwd    stty     tr        which

(most are symbolic links to busybox, except mount umount gpg loadkeys shred
syslinux aespipe)

./boot:
default.kmap

./dev:
console   loop0  loop6   ram1  ram7    sda4  sdb   sdb6  sdc2  sdc8  tty3
hwrandom  loop1  loop7   ram2  random  sda5  sdb1  sdb7  sdc3  sdc9  tty4
hwrng     loop2  mapper/ ram3  sda     sda6  sdb2  sdb8  sdc4  tty   tty5
initctl   loop3  mem     ram4  sda1    sda7  sdb3  sdb9  sdc5  tty0  tty6
kmem      loop4  null    ram5  sda2    sda8  sdb4  sdc   sdc6  tty1  urandom
loop/     loop5  ram0    ram6  sda3    sda9  sdb5  sdc1  sdc7  tty2

./dev/loop:
0

./dev/mapper:
control (c 10 63)

./etc:
fstab  group  inittab  ld.so.cache  lvm/ passwd  rc  shadow

./etc/lvm:

./lib:
ld-linux.so.2    libcrypt.so.1         libm.so.6         libsepol.so.1
libblkid.so.1    libctutils.so.0       libncurses.so.5   libuuid.so.1
libc.so.6        libdevmapper.so.1.02  libpthread.so.0   lvm-0
libcfont.so.0    libdl.so.2            libreadline.so.5  lvm-200/
libcom_err.so.2  libe2p.so.2           librt.so.1        lvm-default
libconsole.so.0  libext2fs.so.2        libselinux.so.1   modules/

(lvm-200 is a copy of your distrib's one, lvm-0 and lvm-default are a symlink to
it).

./lib/modules:
2.6.23.9
(and all that follows inside it...)

./mnt:

./proc:

./root:
(you should store there documentation about the setup procedure)

./sbin:
depmod         losetup        lvreduce       modprobe    shutdown      vgmerge
fdisk          lvchange       lvremove       pvchange    swapon        vgmknodes
fsck           lvconvert      lvrename       pvcreate    vgcfgbackup   vgreduce
fsck.ext2      lvcreate       lvresize       pvdisplay   vgcfgrestore  vgremove
fsck.minix     lvdisplay      lvs            pvmove      vgchange      vgrename
fsck.msdos     lvextend       lvscan         pvremove    vgck          vgs
fsck.reiserfs  lvm            mkfs           pvresize    vgconvert     vgscan
fsck.vfat      lvmchange      mkfs.ext2      pvs         vgcreate      vgsplit
getty          lvmdiskscan    mkfs.minix     pvscan      vgdisplay
halt           lvmiopversion  mkfs.msdos     reboot      vgexport
init           lvmsadc        mkfs.reiserfs  reiserfsck  vgextend
insmod         lvmsar         mkfs.vfat      rmmod       vgimport

(lv* pv* and vg* are symlinks to lvmiopversion. Others are real files.)

./tmp:

./usr:
sbin/

./usr/sbin:
chroot
(this is a symlink to busybox again, needed to be identical to target when booting)

./var:
lock/ log/  run/

./var/lock:
lvm/

./var/lock/lvm:

./var/log:

./var/run:
utmp

Some files are not really needed, like /etc/fstab, or /var/log/ and /bin/aespipe. It's up to you, and what you want to do with it.

When you think you have copied everything fine (i.e. libs), you can test a binary with the command chroot /path/to/the/image/root /bin/command. If there are missing libraries, this will fail. You can exit the chroot with ^d.

A file we particularly need to pay attention to is the boot script /etc/rc. In it, we will mount temporary file systems, because the initrd is read-only and we cannot mount LVM and store the encryption keys on a read-only file systems.

/etc/rc boot script
mount -nt proc proc /proc
mount -nt tmpfs tmpfs /tmp
cp -a /dev /tmp/
cp -a /etc /tmp/
cp -a /var /tmp/
mount -nt tmpfs tmpfs /etc
mount -t tmpfs tmpfs /dev
mount -t tmpfs tmpfs /var
cp -a /tmp/dev /
cp -a /tmp/etc /
cp -a /tmp/var /

/bin/hostname whatever 
/bin/loadkeys /boot/default.kmap

/bin/ash

With this script, all important nodes are mounted read-write in a tmpfs (in memory). You can then generate keys and save them (but you have to use a second usb key to store them after reboot) by the time you create the loop, the LVM and file systems, which is explained in the next section.

You should also add a /boot/default.kmap to have your keyboard map well mapped, this can be useful when typing passwords... dumpkeys > boot/default.kmap

When you are done populating the init image, umount it and compress it with gzip -9 image to get the image.gz initrd.

3.3. Copy files to the USB key, and make it bootable

Now we have the image.gz initial ramdisk, and the kernel. We have to install them on the USB key. Use syslinux to install the boot block on the key; create a simple config file for it:

syslinux.cfg config file content
default vmlinuz root=/dev/ram0 initrd=image.gz init=/etc/rc
append ramdisk_size=18000

Erase its partition table, create the new file system, and copy files to the USB key.

USB key formatting and files copying
dd if=/dev/zero of=/dev/sda bs=1024k count=5 conv=notrunc
mkfs -t vfat -I /dev/sda
mount -t vfat /dev/sda /mnt
cp image.gz syslinux.cfg vmlinuz /mnt/
umount /mnt

We just have now to run syslinux to make the key bootable: syslinux /dev/sda (when not mounted). We are done with the setup USB key. Try to boot on it, selecting all USB devices in boot order in the BIOS and the "Boot other devices". Don't ask me the root password of the image, which is asked if you exit the shell, I don't even know it.

4. Setup the loop-aes, encryption keys, LVM2, and install the base system

We will now boot on our setup USB key and setup everything. The first step (if it was able to boot), is to fill the hard disk with random data. Don't use shred to do this, because it writes the same pattern all over the drive, but dd if=/dev/urandom of=/dev/sda bs=4M.

Then create your GPG key which will protect the encryption keys. We use the multi-access v3 GPG keys which is currently the best protection we can get, and permits several users to boot the system if needed. A passphrase will be asked to protect the keys, make it long and possibly random, and don't write it down, but remember it. If you forgot it, all you can do is reinstall and lost everything from your drive. Start by generating keys for each user, add them to your keyring and generate the encryption keys directly into the GPG pipe. This comes from project Naranja (see links).

GPG keys creation, importing and disk encryption key generating
# generate your GPG keys where you can write:
/bin/gpg --homedir /tmp/gpghome --gen-key

# import public keys or users which will use the loop-aes in yours:
/bin/gpg --no-default-keyring --homedir /tmp/gpghome --always-trust \
--lock-never --import KEY1.pub KEY2.pub ...

# generate the encryption keys (in the key file)
# the 0x01234567 is the key displayed for your public key in the previous command
# make sure you use /dev/random or /dev/hwrng and not /dev/urandom if you want a
# securely generated key (with good randomness).
head -c 2925 /dev/random | uuencode -m - | head -n 66 | tail -n 65 | \
gpg --no-default-keyring --homedir /tmp/gpghome --armor --encrypt \
--default-recipient-self --recipient 0x01234567 --always-trust \
--lock-never > master_disk_key-01234567.gpg

# if you imported several public keys, you have to make them able to acces the
# disk key, so you have to reencode it with their key
# start getting references to the keys:
gpg --no-default-keyring --homedir /tmp/gpghome --fin | grep pub

gpg --no-default-keyring --homedir /tmp/gpghome  --decrypt < \
master_disk_key-01234567.gpg | gpg --encrypt --armor --homedir /tmp/gpghome \
--always-trust --lock-never -r pubkey1 -r pubkey2 ... > user_disk_key.key

Using the /dev/random on a system without the hardware random number generator (via-rng for example) can take hours to get a good entropy. So make sure you have it available and it will take less than one minute.

Copy the created key files to another USB drive, or on the network if available (a secure location of course), because file systems are read only and data will be lost at shutdown.

Now, you can mount the hard drive (I don't use partitions, I encrypt the whole drive) in the loop-aes device with appropriate encryption algorithm (I use AES256), and start creating the LVM architecture over /dev/loop0. Then create file systems on created LVM volumes and mount rootfs in /newroot, and create mount points for other file systems and mount them too.

Loop mounting, LVM and file systems creation commands
# /sbin/losetup -G /tmp/gpghome -K /tmp/master_disk_key-01234567.gpg \
> -e AES256 /dev/loop0 /dev/sda

# pvcreate /dev/loop0
Physical volume "/dev/loop0" successfully created
# vgcreate evg0 /dev/loop0
Volume group "evg0" successfully created
# vgchange -a y evg0
0 logical volume(s) in volume group "evg0" now active
# lvcreate -L3500 -nrootvol evg0
volume "rootvol" created
# lvcreate -L1000 -nvarvol evg0
Logical volume "varvol" created
# lvcreate -LR100 -ntmpvol evg0
Logical volume "tmpvol" created
## see how much free space is left
# vgdisplay evg0 | grep Free
Free  PE / Size      176101 / 687.89 GB
## NOTE: the below command uses a lowercase 'l' instead of the uppercase 'L'
## lowercase 'l' == extents, uppercase 'L' == size in Mb
# lvcreate -l175600 -nhomevol evg0
Logical volume "homevol" created

## now create the filesystems
# mkfs.reiserfs /dev/evg0/rootvol
# mkfs.reiserfs /dev/evg0/varvol
# mkfs.reiserfs /dev/evg0/tmpvol
# mkfs.reiserfs /dev/evg0/homevol

## and mount them
# mount -t reiserfs /dev/evg0/rootvol /newroot
# mkdir /newroot/tmp
# mount -t reiserfs /dev/evg0/varvol /newroot/var
# mkdir /newroot/home
# mount -t reiserfs /dev/evg0/tmpvol /newroot/tmp
# mkdir /newroot/var
# mount -t reiserfs /dev/evg0/homevol /newroot/home

You can now install the base system, untaring the base_install.tar.gz created in the first step into /newroot/.

Now you should edit some files, like the host /etc/fstab if you want your system to be able to mount file systems! With this system, you access file systems through LVM, so the fstab will be a regular LVM compliant one.

Content of the host /etc/fstab
/dev/evg0/rootvol /	reiserfs defaults 0 1
/dev/evg0/varvol  /var	reiserfs defaults 0 2
/dev/evg0/tmpvol  /tmp	reiserfs defaults 0 2
/dev/evg0/homevol /home	reiserfs defaults 0 2

Umount everything, and you're done, you can halt and create the boot key.

5. Create the real USB boot key (initramfs)

If you use initrd instead of initramfs for the USB key which will really boot the system, it must be as small as possible, because the initrd cannot be umounted, since it has the loop device active. It will stay in RAM forever.

For this step, I took the setup initrd and removed all unneeded files for the only boot phase, bringing the image to around 5MB. Files need to be added: the master encryption key, your GPG home, and the run-init executable for initramfs replacing pivot_root for initrd. run-init can be found in klibc-utils, but the /lib/klibc-something.so has to be copied too. Here's what I come to (which can be probably optimized, there still must be unused files, and lighter executables such as full klibc or busybox with µClibc static can be used):

initramfs's files for boot key
.:
bin  boot  dev  etc  init  lib  mnt  newroot  proc  root  sbin  tmp  usr  var

./bin:
ash      chmod   cp   grep      ln        login  mknod  mv   rm  sleep  sync
busybox  chroot  gpg  hostname  loadkeys  mkdir  mount  pwd  sh  stty   umount

./boot:
default.kmap

./dev:
console  kmem  loop0   mem   ram0  sda1  tty   tty1
initctl  loop  mapper  null  sda   sdb   tty0

./dev/loop:
0

./dev/mapper:
control

./etc:
group  inittab  ld.so.cache  lvm  passwd  shadow

./etc/lvm:

./lib:
klibc-B9LS-Gjx2D7BYcbQig0RlgHKO9Y.so  libctutils.so.0       libselinux.so.1
ld-linux.so.2                         libdevmapper.so.1.02  libsepol.so.1
libblkid.so.1                         libdl.so.2            libuuid.so.1
libcfont.so.0                         libm.so.6             lvm-0
libconsole.so.0                       libncurses.so.5       lvm-200
libcrypt.so.1                         libpthread.so.0       lvm-default
libc.so.6                             libreadline.so.5

./lib/lvm-200:
lvchange   lvmchange    lvrename  pvdisplay  vgcfgbackup   vgdisplay  vgreduce
lvconvert  lvmdiskscan  lvresize  pvmove     vgcfgrestore  vgexport   vgremove
lvcreate   lvmsadc      lvs       pvremove   vgchange      vgextend   vgrename
lvdisplay  lvmsar       lvscan    pvresize   vgck          vgimport   vgs
lvextend   lvreduce     pvchange  pvs        vgconvert     vgmerge    vgscan
lvm        lvremove     pvcreate  pvscan     vgcreate      vgmknodes  vgsplit

./newroot:

./root:
.gpg/ master_disk_key-0x01234567.gpg

./sbin:
depmod  fsck.reiserfs  losetup  lvmiopversion  run-init  vgscan
fsck    getty          lvm      reiserfsck     vgchange

./tmp:

./usr:
sbin

./usr/sbin:
chroot

./var:
lock  run

./var/lock:
lvm

./var/lock/lvm:

./var/run:
utmp

As you see in /root/ you have to put your files, which are your gpg homedir and your master encryption key. Try to not backup this latter on a non-encrypted media.

You may try to strip binaries, operation which removes debug information and symbols from binary files, and make them smaller, which is what we seek.

The content of /etc/rc must be changed and the file is now /init which is the default for initramfs (replace your real key name).

/init boot script for the boot USB key
#!/bin/sh
mount -nt proc proc /proc
mount rootfs / -o remount,rw

/bin/hostname izakar
/bin/loadkeys /boot/default.kmap

/sbin/losetup -G /root/.gpg -K /root/master_disk_key-0x01234567.gpg \
    -e AES256 /dev/loop0 /dev/sda

if [ $? != 0 ]; then echo "Unable to mount loop device. Rebooting."
exit 1; fi

/sbin/vgscan
/sbin/vgchange -a y evg0

mount -t reiserfs -o ro /dev/mapper/evg0-rootvol /newroot

umount /proc

exec /sbin/run-init /newroot /sbin/init "$@" \
    < /newroot/dev/console > /newroot/dev/console

The rootfs from the initramfs can be mounted read/write, so tmpfs is not needed anymore for /dev /var and /etc. I did not test this -o remount,rw trick with initrd, but maybe it can work too. The run-init binary removes all files from the initramfs before remountint the hard drive's / in place. It then executes /sbin/init and the OS begins to boot.

You can find my image, the one listed above here.

Now you can create the ramfs.gz file from the directory where you have put all files with: find . | cpio -o -H newc | gzip -9 > ../ramfs.gz . Make sure you don't put a filename longer than 8 characters without the extension because it will be copied on a msdos filesystem which does not supports longer. This kind of file can be unrolled with: cd fs && gunzip -c ../ramfs.gz | cpio -iv --no-absolute-filenames .

Copy the files the same way than for the setup USB key. Change the syslinux.cfg to: default vmlinuz initrd=ramfs.gz and rerun syslinux /dev/sda, if sda is you USB key, make sure you don't run it on your hard drive!

6. Boot on your real system

Now you can boot on your boot USB key. It will ask you the gpg passphrase protecting the hard drive's encryption keys when it boots. If you fail, you have to reboot and retry, there is only one try. You can modify the /init script to get better chances. The initramfs then mounts the real root filesystem and gives hand to the host OS.

When your system is booted, you can remove the USB key and keep it safe, far from the drive. Don't make any copy of the encryption key on a non-crypted unremovable drive. I think that you should make 2 copies of the boot USB key, and keep them in safe different places.

If you try to boot on the system without the key, the hard drive will not be seen as a bootable device. Without the encryption key and the passphrase, your data will be safe.

For the funny story, after I did all that, at my first boot, I tried to copy a file from the network with ftp. At around 220MB, it crashed with a kernel panic, caused by heat. And this is when you figure out that this fanless board cannot be fanless :)

Software related to hardware

cpufreq. If you have a Jetway mini-itx board like the J7F4K1G2, you can use this script to change the power allocation (cpufreq) for the cpu.

power.sh script
#!/bin/sh

if [[ $# == 1 ]]
then	echo "writing config..."
	echo "$1" > /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor
fi	

cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_available_governors
echo -n "==> "
cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor | tr '\n' ' '
FREQ=$((`cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_cur_freq` / 1000))
echo "(running at $FREQ""MHz)"

It will display available policies for cpufreq (the one you insmoded), like performance, powersave, or ondemand. Performance always runs the CPU at 1200MHz, powersave always at 400MHz, and ondemand switches to 1200MHz on load, which decreases power consumption on idle. The power.sh script can take an argument which is the name of a policy and makes it the in-use policy.

Sensors. Install the package lm-sensors. The sensors command will display voltages and temperatures from the onboard sensors. This needs to load the modules indicated by sensors-detect (f71805f, hwmon, i2c_dev, i2c_viapro, and i2c_core). You might also install hddtemp to get the internal temperature of your hard drives disks.

OpenSSL. OpenSSL and OpenSSH can use the padlock engine to encrypt and decrypt data. Last mainstream version do, but debian's stable (etch) does not, and needs a patch (BTW, I am not able to build the debian stable package with the patch, it segfaults). Next stable (lenny) has openssh patched, but openssl is still requested to be patched to support it. This is trivial, and this time I am able to build the patched package. You should look at here for instructions and this is the bug report for lenny.

SHA-1 sum. The original coreutil sha1sum does not support Padlock Hash Engine. However, you can replace it by Michal Ludvig's implementation phe_sum. I have run a simple benchmark on a 776MB file, it takes 49s with sha1sum, and 22s with phe_sum. Unfortunately, SHA-256 support was not implemented.

7. Links and documentation

The loop-aes README, a must read!

Why use loop-aes and not cryptoloop: here. This is not really relevant now because cyptoloop has been deprecated by dm-crypt, which was designed to support advanced modes of operation, such as XTS, LRW and ESSIV, in order to avoid watermarking attacks. The wikipedia page on disk encryption theory and lots of its links are very useful to understand all issues of disk encryptions, and different methods. The Arch Linux wiki page on System Encryption with LUKS for dm-crypt will help you with dm-crypt, and there is a little comparison between different approaches.

Project Naranja which is exactly the same thing, but was not available when I did this. See this link too.

The linux documentation project Encrypted Root Filesystem HOWTO

Linux documentation on initramfs and initrd.

Help on creating initrd: Booting Debian with root filesystem on LVM

Why it is a bad idea to buy a Jetway motherboard: design is so lame...

Programming with padlock: http://www.logix.cz/michal/doc/article.xp/padlock-en

GrouNation blog: crypto, padlock, TPMs with linux.

Inside NetBSD's CGD, a talk about crypto disk drivers of diverse OSes, and the implementation of NetBSD's CGD. http://www.onlamp.com/pub/a/bsd/2005/12/21/netbsd_cgd.html

0 comment


Some numbers on hardware-accelerated crypto

I had a chance to get hands on a few CPUs that have hardware accelarators for cryptographic operations such as AES encryption and SHA hashing. Since I could not find this information for all of them on the Web, here's a list, to be updated when more is available. Some use the Padlock engine, some Intel AES-NI. Data was obtained with the cryptsetup benchmark command.

The Via Nano U2250 (1.6GHz)

Like the Via C7 processor, it features the Padlock engine for acceleration of 128-bit or 256-bit encryption operations, not 512 and not in other modes than cbc.

PBKDF2-sha1       289982 iterations per second
PBKDF2-sha256     195047 iterations per second
PBKDF2-sha512     116198 iterations per second
PBKDF2-ripemd160  193893 iterations per second
PBKDF2-whirlpool   55727 iterations per second
#  Algorithm | Key |  Encryption |  Decryption
     aes-cbc   128b   396.6 MiB/s   444.5 MiB/s
 serpent-cbc   128b    22.2 MiB/s    47.0 MiB/s
 twofish-cbc   128b    53.1 MiB/s    56.2 MiB/s
     aes-cbc   256b   380.9 MiB/s   379.1 MiB/s
 serpent-cbc   256b    26.8 MiB/s    47.0 MiB/s
 twofish-cbc   256b    56.4 MiB/s    56.2 MiB/s
     aes-xts   256b    92.6 MiB/s    92.4 MiB/s
 serpent-xts   256b    43.3 MiB/s    43.5 MiB/s
 twofish-xts   256b    50.9 MiB/s    51.0 MiB/s
     aes-xts   512b    67.5 MiB/s    67.5 MiB/s
 serpent-xts   512b    43.6 MiB/s    43.5 MiB/s
 twofish-xts   512b    51.1 MiB/s    51.0 MiB/s

The Intel Atom C2350 (1.7GHz)

Also called Avoton, this family of Atom processors is not as bad as its predecessors, but still terrible. It is clearly outperformed by the Via Nano on all accelerated operations. AES-NI supports 512-bit keys operations and xts mode, which gives it a big advantage over the Padlock. Non-accelerated operation are only 20 to 50 percent faster than the Nano above, which tells you how bad this processor is.

PBKDF2-sha1       134295 iterations per second
PBKDF2-sha256      80709 iterations per second
PBKDF2-sha512      55727 iterations per second
PBKDF2-ripemd160  120470 iterations per second
PBKDF2-whirlpool   43574 iterations per second
#  Algorithm | Key |  Encryption |  Decryption
     aes-cbc   128b   241.0 MiB/s   332.0 MiB/s
 serpent-cbc   128b    30.8 MiB/s    74.7 MiB/s
 twofish-cbc   128b    67.7 MiB/s    68.5 MiB/s
     aes-cbc   256b   182.0 MiB/s   260.0 MiB/s
 serpent-cbc   256b    30.9 MiB/s    75.0 MiB/s
 twofish-cbc   256b    67.7 MiB/s    68.2 MiB/s
     aes-xts   256b   292.0 MiB/s   292.0 MiB/s
 serpent-xts   256b    69.4 MiB/s    69.0 MiB/s
 twofish-xts   256b    63.0 MiB/s    63.0 MiB/s
     aes-xts   512b   233.0 MiB/s   234.0 MiB/s
 serpent-xts   512b    69.7 MiB/s    69.4 MiB/s
 twofish-xts   512b    63.2 MiB/s    63.2 MiB/s

1 comment


Installing encrypted root on debian stretch

This article is an update to this tutorial that is getting a bit outdated: Setting up my server: re-installing on an encripted LVM.

Debian stretch has new features that change the way to configure the initramfs and networking. It took me a while to figure them out, so this may be useful to somebody else.

The initramfs configuration has changed, especially for the dropbear part. There is now a package that enables dropbear only in the initramfs, dropbear-initramfs, and it almost does not require manual configuration. Only the ssh key has to be put in /etc/dropbear-initramfs/authorized_keys instead of directly in the initramfs files /etc/initramfs-tools/root/.ssh/authorized_keys. One reason for that is that the root homedir is not /root in the initramfs anymore, it's randomly generated when running update-initramfs. I haven't checked why, but it's probably for security reasons.

It is very important to know that since stretch, debian stopped using eth0 as default network interface name. It now uses a name generated with the MAC address, to have unique identifiers for network interfaces, as announced here. If you are installing the server and if you don't have a remote console to check the name, the best is to use dhcp the first time to be able to connect to the machine. Make sure you use that name in the kernel command line otherwise you won't be able to ping your machine or unlock the encrypted filesystem. Take also care of the /etc/network/interfaces for the regular system boot.

In case of problem, you may have access to a serial console. It's easy to configure, just add console=ttyS0 to the kernel command line in /etc/default/grub if you use grub2. You may also want to configure grub itself to interface with the serial console, add GRUB_TERMINAL=serial to the file, with possible parameters in the GRUB_SERIAL_COMMAND variable. If you don't see anything on the console it's probably because it's the other port...

In my case it was the Intel Atom Avoton-based server from online.net, the network interface is called enp0s20 and the serial port of the console is ttyS1.

1 comment