This little utilty is used to display images grabbed from a philips webcam under linux. It was designed to be fast, using MIT X Shared memory extension and Xlib. It also dumps ppm files.
Keybindings are: 'd' for dump, 'g' to switch black and white or color mode, 'q' to quit.
You will need several libraries and includes to compile, for example:
| Compile with |
| gcc -O3 -o webcam webcam-xlib-shm.c -I/usr/src/linux/drivers/usb/media/pwc -L/usr/X11R6/lib -lX11 -lXext |
warning: include dir in new pwc (free version 10) is /usr/src/linux/drivers/usb/media/pwc but was not the same in version 9.
Download webcam-xlib-shm.c (01/2005)
You have two philips cameras and you want to display them at the same time ? (this does not work with some versions of linux and pwc, kernel bugs...)
Download double-webcam-xlib-shm.c (11/2004)
To improve time that you can spend using the battery before it is discharged, it seems quite obvious to improve the operating system so that it does not consumes power for useless purposes. You can find most important tweaks that wiki page if you haven't done anything yet. On this page, you will find my more precise advices.
There are two things that consumes power on a computer that we can modify: interruptions and devices use. Interrupts are triggered both by the kernel and the system. Most of them are due to timers, so when a lot of process are launched it's more likely that you have lots of timers because programs generally have sleep loops. It's a good thing to avoid those programs if you don't need them, like cron or media players.
On this web page I draw a list of what I did to save power on my system, which is a EeePC 900, but it can be applied to most systems that have constraints on power or flash drives. With all the tweaks, I extented my battery life from 2 to 3 hours roughly, but it mostly depends on what application you use. When programming or typing text, battery can last a lot more longer, if you don't compile too often. Heat is reduced, system longevity is extented.
Unused devices should obviously be powered down, but sometimes people don't even know what's inside computers, so they can't do it.
First of all, you can look into the BIOS in the section 'integrated peripherals' what it offers to activate or deactivate. Probably there will be USB controllers, with possibly a webcam and a card reader using it, the sound chipset, the embedded loudspeakers, and network interfaces. If you don't need one of those, I suggest you deactivate it here.
Then, where you are running the system, there are devices that can be put in sleep mode or completely deactivated:
Virtual memory is an important cause of processor wake-ups. When you write some information into memory, it is stored into the system's memory for a while, the cache, and then copied on drives. If that time is too low, drives will be used very often. It is better to write a lot of information at a time but less often, so that the CPU can sleep longer.
An other parameter of virual memory is the ratio of dirty memory achieved before copying it back to the drives. When the ratio is low, it means that when a few megabytes have been modified, the write is triggered. When it's high, the write back happens less often. Once again, power saved. But you should be warned: if the system goes down unexpectedly, you will lose all data that has not been stored during this waiting cache time.
sysctl controls the values for time and ratios of write back for the virtual memory susbsystem. Put the following lines in /etc/sysctl.conf, and run # sysctl -p to take it into account. It will be loaded at system startup.
vm.dirty_writeback_centisecs = 3000 vm.dirty_background_ratio = 10 vm.dirty_ratio = 45
The difference between dirty_background_ratio and dirty_ratio is that the first is triggered as a background task, and the second blocks the systems while the dirty memory hasn't been written back. More explanations here.
A minimalistic system, like Arch Linux, is the better way to control all services installed and launched by the operating system. Of course, try to not launch services you don't need. You can also configure the ACPI rules to launch power eating services when you plug the AC adapter to your laptop. This is what I do with cron. cron is waking up the CPU every minute at least, and running some tasks for system maintenance, like updating man pages of find database.
Filesystem options can reduce the number of writes on the drives, which can be a good thing to do with flash memories. Moreover, it reduces the time spent working for the CPU and the northbridge.
The first thing that can be done is disable access times saving. When you
access a file or a directory, they usually save the date when it was accessed.
If you don't need that feature, remove it and you will experience faster file
access and reduced power consumption. It is done by adding the options
noatimes and nodiratimes to the options of the filesystems in
/etc/fstab. For example:
/dev/sda1 / ext2 defaults,noatime,nodiratime 0 1
Using journalling filesystem is also not recommended, since it will write very often on the same block of the drive. It is bad for flash memories.
An other improvement can be added by the use of memory filesystems. Indeed, some directories are not specially useful to keep on the drive, and still, they may suffer from lots of writes and access. I converted three directories of my system to tmpfs, which uses the system memory to save the files. It is faster, less power greedy, but files are lost on reboot. Anyway, I don't really need them, here is my fstab:
tmpfs /var/log tmpfs size=2M 0 0 tmpfs /tmp tmpfs defaults,noatime 0 0 tmpfs /var/cache/pacman/pkg tmpfs defaults,noatime 0 0
Temporary files of softwares: you may not know it but some softwares you use everyday keep track of what you did, to provide history or better user experience. It is the case of less and vim. You can disable the less history file by setting the environment variable: LESSHISTFILE="-" in /etc/profile or something similar. For vim, you can disable the viminfo file, the backup file~ and the swap files by setting:
set viminfo="" set noswapfile set nobackupin your ~/.vimrc or /etc/vimrc if you don't need them.
The best tool you have to try to hunt power leaks is powertop. It acts as the command top does for CPU but for interrupts. You can see what application and kernel thread wakes up the CPU. It also suggests some system tweaks if you don't have done them already. On the powertop website there are some tips and tricks for some known applications too.
XTerm for examples wakes up every second by default, due to session management. It can be fixed by putting xterm*sessionMgt: false in your ~/.Xdefaults.
You will see how much using a media player wakes up the CPU. And they consume even more power since they use the sound or video chipset.
Using a mouse is also very interrupt consumming. You maybe should use a tiling window manager such as awesome, which are optimized for keyboard use.
I have been looking for a low power consuming computer which could replace my old big home server, and I found mini-itx boards based on the VIA C7 processor. This little thing, especially in its eden version (like the 1.2GHz fanless), requires very low power to run (like 6W max, 0.1W average), and has an hardware cryptographic engine built-in, the VIA Padlock. It can compute SHA-1, SHA-256, AES128, AES192, AES256, has a hardware random number generator, and instructions for the exptmod for RSA for example. The board I use in this tutorial/howto is a Jetway J7F4K1G2E (don't buy it, it's a swindle, it needs a fan, and if you enable ACPI it freezes). And I made this box for it. You may also be interested in the AMD Geode LX, which also supports hardware acceleration of AES128 and a random number generator, but it has less processing power. The C7 has been improved to support 64-bit instruction set and multi-core operation, packages in the Via Nano CPU series.
With all this good stuff, I tried to create a very secure system, with hard drive encryption and others. With experience in no-space-left-partitions I already had, I also wanted it to run LVM2 (logical volume manager), which allows to resize partitions at runtime.
Why loop-aes? I could have installed directly debian with LVM on dm-crypt, because the installer supports it, but I've seen that loop-aes was the only secure hard drive encryption method here, so I decided to do it with loop-aes. This actually was a mistake because this document dates from 2004, and now (at the end of 2008), dm-crypt is one of the few most secure disk encryption software using XTS (since Linux-2.6.24). So I should reconsider moving from loop-aes to dm-crypt-LUKS, although loop-aes remains quite secure. If I understand right, if the attacker has only one physical access to the drive he won't be able to get any plaintext information. See links for more information on this, and all kinds of documentation on the subject. With LUKS, it seems that the encryption keys are stored locally on the hard disk drive, which is not required in loop-AES.
So here we go, installing debian on reiserfs file systems on LVM2 partitions on loop-aes AES256 encryption loop with GPG multi-access keys (this is the best protection) on a SATA hard drive, with benefit of the VIA Padlock engine of course, and in the most secure way. It wouldn't be funny if only a part of the hard drive was encrypted, so I did not even made a partition, everything is encrypted, so it has to be mounted from elsewhere... like a bootable USB key. Actually, this is more than a funny thing: by separing the key from the drive, a bad person stoling the drive wouldn't have the keys. This makes 3 things to have to actually access the files: the drive, the encryption keys, and a passphrase protecting the keys. Or just one: an access to the system running...
I chose debian because I'm familiar with it, but this can probably be applied to any Linux distribution. This can also probably be done by network boot with PXE, but I don't know how that works. See antlinux Naranja web page (in links).
Here are the steps needed set up and running this system.
You might say that there is a tool called debootstrap which can install the debian's base system from the Internet, but I prefered copying files. Debootstrap has lots of dependencies, and I find this is not nice for an install from an initrd or initramfs (the install USB key).
So, install your debian on a system, in one partition or whatever, install rsync, openssh-server, lvm and reiserfs tools on it, or whatever file system you chose. You can also uninstall the default kernel, and grub, and maybe try to boot the kernel you will configure in step 2. Then rsync -a this_system:/ backup_dir to get files. Remove /etc/mtab and /etc/fstab. Create a tar.gz file from the files (cd backup_dir && tar zcvf ../base_install.tar.gz *).
We will use a kernel supporting the C7 hardware random generator and the loop-aes for the setup, this will make it really faster. As we have to configure and build a kernel, we will make the final one, with all VIA options, and those for the boot initrd.
Download the latest kernel, I think it works quite fine since version 2.6.22.
Download the latest loop-aes software. You can find it on the loop-aes website.
2012 update: loop-aes v3.6e works fine with the 3.2 stable kernel branch.
Patch your kernel:
cd linux-2.6.23.12 rm include/linux/loop.h drivers/block/loop.c patch -Np1 < ../loop-AES-v3.2b/kernel-2.6.23.diff
Enter the kernel's configuration. Select C7 for the processor, enable the hardware random number generator, loop-aes device with padlock hardware handling, padlock engine in cryptographic engines. For the initrd, you will need the ramdisk support, initrd, usb disk support, minix, vfat, tmpfs procfs and host (reiserfs) filesystems. Here is the real list:
initrd specific: CONFIG_BLK_DEV_RAM=y CONFIG_BLK_DEV_RAM_SIZE=4096 initramfs or initrd specific: CONFIG_BLK_DEV_INITRD=y USB disk specific: CONFIG_SCSI=y CONFIG_BLK_DEV_SD=y CONFIG_USB=y CONFIG_USB_STORAGE=y CONFIG_USB_EHCI_HCD=y CONFIG_USB_UHCI_HCD=y File systems for boot disk: CONFIG_MINIX_FS=y CONFIG_PROC_FS=y CONFIG_TMPFS=y CONFIG_CRAMFS=n CONFIG_FAT_FS=y CONFIG_MSDOS_FS=y CONFIG_VFAT_FS=y CONFIG_NLS_CODEPAGE_437=y CONFIG_NLS_CODEPAGE_850=y CONFIG_NLS_ISO8859_1=y CONFIG_NLS_ISO8859_15=y CONFIG_FAT_DEFAULT_CODEPAGE=850 CONFIG_FAT_DEFAULT_IOCHARSET=iso8859-15 Host file systems: CONFIG_BLK_DEV_DM=y CONFIG_REISERFS_FS=y VIA C7 specific: CONFIG_MVIAC7=y CONFIG_HW_RANDOM_VIA=y CONFIG_CRYPTO_HW=y CONFIG_CRYPTO_DEV_PADLOCK=y CONFIG_CRYPTO_DEV_PADLOCK_AES=y CONFIG_CRYPTO_DEV_PADLOCK_SHA=y loop-aes + padlock specific: CONFIG_BLK_DEV_LOOP=y CONFIG_BLK_DEV_LOOP_AES=y CONFIG_BLK_DEV_LOOP_PADLOCK=y And for even less power consumption: CONFIG_CPU_FREQ=y CONFIG_CPU_FREQ_GOV_PERFORMANCE=y CONFIG_CPU_FREQ_GOV_POWERSAVE=m CONFIG_CPU_FREQ_GOV_ONDEMAND=m CONFIG_X86_E_POWERSAVER=y (C7 specific) CONFIG_TICK_ONESHOT=y CONFIG_NO_HZ=y
Initrd or initramfs? I chose initrd in the first part of tutorial because I was not familiar with initramfs, but the last part (the boot USB key) is made with initramfs. When we will boot on the real system, as the loop will be mounted in the initial ramdisk, we will never be able to unmount it. Using initrd implies that an allocated /dev/ram0 (about 8MB) will stay in RAM forever, which will be lost space. With initramfs, the allocated space is the same than needed by files, thus we remove files from the initramfs before actually booting the real system, and no memory is lost. The other point is that initramfs is easier to build and tune than an initrd, because it does not needs mount privileges, it just takes real files into an archive, so you can make lists of what you want or don't in it. So you will have in this tutorial the two ways for doing a bootable kernel with initrd or initramfs. If you choose initramfs (what I recommend), you can put CONFIG_BLK_DEV_RAM=n.
Don't forget networking support, with CONFIG_R8169 for the Realtek of the J7F4 boards. SATA and PATA support with VIA chipsets, and you may also use sensors to see voltages and temperatures of the system brought by CONFIG_I2C_VIAPRO=m CONFIG_SENSORS_EEPROM=m CONFIG_HWMON=m CONFIG_SENSORS_F71805F=m.
You can also simply download my config file here.
Save your config file, run make and make modules_install, and you're done with the kernel. Maybe the best thing would be to do this step inside the base system of the previous section on the target system, and check that the kernel you configured is working correctly, and that would install directly modules at the right place. Else, you should copy the /lib/modules/2.6.23.9 directory to base installation files or to the USB key on which you will put the base_install.tar.gz), if you want network support and if it's not built into the kernel, it can help.
We will now create an USB boot disk which will be used to create the loop-aes, the encryption keys stored with GPG, the LVM, the filesystems and copy the base debian install.
Standard tools don't handle loop-aes and mounting encrypted file systems correctly, so we will have to patch and recompile them statically, which is easier for embedding in an initrd/initramfs.
You can download my files for this usb key image, which contains patched binaries of softwares if you don't want to create everything or compile and configure your kernel here.
losetup, mount, umount. The maintainer of project Naranja told me that debian's binaries were already patched to support loop-aes -- well, I just verified this by updating my ramfs (december, 2 2008), and losetup is not working with the options we use in the boot script, you have to patch it. You can then use them directly (mount and umount) instead of patching them and compiling them by hand. See the loop-aes README for information about that.
gpg. You can either add a line to the gpg.conf file or recompile gpg to provide better resistance against dictionary attacks, by changing the default amount of iterations. This will slow down key generation and dictionary attacks.
So you can add this line to /etc/gnupg/gpg.conf: s2k-count 8388608. You can also use the command line parameter --s2k-count 8388608. This can be better, as we will try to get the lowest number of files in our initrd. You can also patch it the same way we patched util-linux, the patch is in loop-aes archive. The advantage is that you can make a static build, which does not need any shared libraries to be installed on the image. To do so, use: CFLAGS="-O2" LDFLAGS="-static -s" ./configure --prefix=/usr --enable-static-rnd=linux. Then copy gpg to /bin of the image.
See the loop-aes README for more information on softwares needed to be compiled.
We now create the init ram disk, which will actually contain a little OS file system.
Create the disk image with dd; 16M is probably enough for everybody, but
depending on what you really want inside it can be larger. Mine was around 11M
used.
Then create a file system in it, and mount it.
| Creating the initrd image file |
dd if=/dev/zero of=image bs=1M count=16 mke2fs -F image mkdir mnt mount -o loop image mnt/ |
Now we can put in it all the tools we will need. I recommend using busybox for basic utilities like sh, ls, mkdir, cp and so on. Some files are also needed for the system to boot and run correctly, like devices /dev/console. Create all devices you will like to use, and for all binary you put in the image, you will have to copy dependent shared libraries too. You can see them with the command ldd. To make it easier, here is my file list:
| initrd's files for setup and rescue key |
.: bin dev initrd mnt proc sbin tmp var boot etc lib newroot root sys usr ./bin: [ chown env hostname mknod reset su true whoami [[ chroot false ifconfig more rm sync umount xargs aespipe cp fgrep kill mount rmdir syslinux uname yes ar date find ln mt sed syslogd uniq zcat ash dd getopt loadkeys mv sh tail uudecode awk df gpg loadkmap patch shred tar uuencode basename dirname grep login pidof sleep test vi busybox dmesg gunzip losetup ping sort time wc cat du gzip ls ps strings touch wget chmod echo head mkdir pwd stty tr which (most are symbolic links to busybox, except mount umount gpg loadkeys shred syslinux aespipe) ./boot: default.kmap ./dev: console loop0 loop6 ram1 ram7 sda4 sdb sdb6 sdc2 sdc8 tty3 hwrandom loop1 loop7 ram2 random sda5 sdb1 sdb7 sdc3 sdc9 tty4 hwrng loop2 mapper/ ram3 sda sda6 sdb2 sdb8 sdc4 tty tty5 initctl loop3 mem ram4 sda1 sda7 sdb3 sdb9 sdc5 tty0 tty6 kmem loop4 null ram5 sda2 sda8 sdb4 sdc sdc6 tty1 urandom loop/ loop5 ram0 ram6 sda3 sda9 sdb5 sdc1 sdc7 tty2 ./dev/loop: 0 ./dev/mapper: control (c 10 63) ./etc: fstab group inittab ld.so.cache lvm/ passwd rc shadow ./etc/lvm: ./lib: ld-linux.so.2 libcrypt.so.1 libm.so.6 libsepol.so.1 libblkid.so.1 libctutils.so.0 libncurses.so.5 libuuid.so.1 libc.so.6 libdevmapper.so.1.02 libpthread.so.0 lvm-0 libcfont.so.0 libdl.so.2 libreadline.so.5 lvm-200/ libcom_err.so.2 libe2p.so.2 librt.so.1 lvm-default libconsole.so.0 libext2fs.so.2 libselinux.so.1 modules/ (lvm-200 is a copy of your distrib's one, lvm-0 and lvm-default are a symlink to it). ./lib/modules: 2.6.23.9 (and all that follows inside it...) ./mnt: ./proc: ./root: (you should store there documentation about the setup procedure) ./sbin: depmod losetup lvreduce modprobe shutdown vgmerge fdisk lvchange lvremove pvchange swapon vgmknodes fsck lvconvert lvrename pvcreate vgcfgbackup vgreduce fsck.ext2 lvcreate lvresize pvdisplay vgcfgrestore vgremove fsck.minix lvdisplay lvs pvmove vgchange vgrename fsck.msdos lvextend lvscan pvremove vgck vgs fsck.reiserfs lvm mkfs pvresize vgconvert vgscan fsck.vfat lvmchange mkfs.ext2 pvs vgcreate vgsplit getty lvmdiskscan mkfs.minix pvscan vgdisplay halt lvmiopversion mkfs.msdos reboot vgexport init lvmsadc mkfs.reiserfs reiserfsck vgextend insmod lvmsar mkfs.vfat rmmod vgimport (lv* pv* and vg* are symlinks to lvmiopversion. Others are real files.) ./tmp: ./usr: sbin/ ./usr/sbin: chroot (this is a symlink to busybox again, needed to be identical to target when booting) ./var: lock/ log/ run/ ./var/lock: lvm/ ./var/lock/lvm: ./var/log: ./var/run: utmp |
Some files are not really needed, like /etc/fstab, or /var/log/ and /bin/aespipe. It's up to you, and what you want to do with it.
When you think you have copied everything fine (i.e. libs), you can test a binary with the command chroot /path/to/the/image/root /bin/command. If there are missing libraries, this will fail. You can exit the chroot with ^d.
A file we particularly need to pay attention to is the boot script /etc/rc. In it, we will mount temporary file systems, because the initrd is read-only and we cannot mount LVM and store the encryption keys on a read-only file systems.
| /etc/rc boot script |
mount -nt proc proc /proc mount -nt tmpfs tmpfs /tmp cp -a /dev /tmp/ cp -a /etc /tmp/ cp -a /var /tmp/ mount -nt tmpfs tmpfs /etc mount -t tmpfs tmpfs /dev mount -t tmpfs tmpfs /var cp -a /tmp/dev / cp -a /tmp/etc / cp -a /tmp/var / /bin/hostname whatever /bin/loadkeys /boot/default.kmap /bin/ash |
With this script, all important nodes are mounted read-write in a tmpfs (in memory). You can then generate keys and save them (but you have to use a second usb key to store them after reboot) by the time you create the loop, the LVM and file systems, which is explained in the next section.
You should also add a /boot/default.kmap to have your keyboard map well mapped, this can be useful when typing passwords... dumpkeys > boot/default.kmap
When you are done populating the init image, umount it and compress it with gzip -9 image to get the image.gz initrd.
Now we have the image.gz initial ramdisk, and the kernel. We have to install them on the USB key. Use syslinux to install the boot block on the key; create a simple config file for it:
| syslinux.cfg config file content |
default vmlinuz root=/dev/ram0 initrd=image.gz init=/etc/rc append ramdisk_size=18000 |
Erase its partition table, create the new file system, and copy files to the USB key.
| USB key formatting and files copying |
dd if=/dev/zero of=/dev/sda bs=1024k count=5 conv=notrunc mkfs -t vfat -I /dev/sda mount -t vfat /dev/sda /mnt cp image.gz syslinux.cfg vmlinuz /mnt/ umount /mnt |
We just have now to run syslinux to make the key bootable: syslinux /dev/sda (when not mounted). We are done with the setup USB key. Try to boot on it, selecting all USB devices in boot order in the BIOS and the "Boot other devices". Don't ask me the root password of the image, which is asked if you exit the shell, I don't even know it.
We will now boot on our setup USB key and setup everything. The first step (if it was able to boot), is to fill the hard disk with random data. Don't use shred to do this, because it writes the same pattern all over the drive, but dd if=/dev/urandom of=/dev/sda bs=4M.
Then create your GPG key which will protect the encryption keys. We use the multi-access v3 GPG keys which is currently the best protection we can get, and permits several users to boot the system if needed. A passphrase will be asked to protect the keys, make it long and possibly random, and don't write it down, but remember it. If you forgot it, all you can do is reinstall and lost everything from your drive. Start by generating keys for each user, add them to your keyring and generate the encryption keys directly into the GPG pipe. This comes from project Naranja (see links).
| GPG keys creation, importing and disk encryption key generating |
# generate your GPG keys where you can write: /bin/gpg --homedir /tmp/gpghome --gen-key # import public keys or users which will use the loop-aes in yours: /bin/gpg --no-default-keyring --homedir /tmp/gpghome --always-trust \ --lock-never --import KEY1.pub KEY2.pub ... # generate the encryption keys (in the key file) # the 0x01234567 is the key displayed for your public key in the previous command # make sure you use /dev/random or /dev/hwrng and not /dev/urandom if you want a # securely generated key (with good randomness). head -c 2925 /dev/random | uuencode -m - | head -n 66 | tail -n 65 | \ gpg --no-default-keyring --homedir /tmp/gpghome --armor --encrypt \ --default-recipient-self --recipient 0x01234567 --always-trust \ --lock-never > master_disk_key-01234567.gpg # if you imported several public keys, you have to make them able to acces the # disk key, so you have to reencode it with their key # start getting references to the keys: gpg --no-default-keyring --homedir /tmp/gpghome --fin | grep pub gpg --no-default-keyring --homedir /tmp/gpghome --decrypt < \ master_disk_key-01234567.gpg | gpg --encrypt --armor --homedir /tmp/gpghome \ --always-trust --lock-never -r pubkey1 -r pubkey2 ... > user_disk_key.key |
Using the /dev/random on a system without the hardware random number generator (via-rng for example) can take hours to get a good entropy. So make sure you have it available and it will take less than one minute.
Copy the created key files to another USB drive, or on the network if available (a secure location of course), because file systems are read only and data will be lost at shutdown.
Now, you can mount the hard drive (I don't use partitions, I encrypt the whole drive) in the loop-aes device with appropriate encryption algorithm (I use AES256), and start creating the LVM architecture over /dev/loop0. Then create file systems on created LVM volumes and mount rootfs in /newroot, and create mount points for other file systems and mount them too.
| Loop mounting, LVM and file systems creation commands |
# /sbin/losetup -G /tmp/gpghome -K /tmp/master_disk_key-01234567.gpg \ > -e AES256 /dev/loop0 /dev/sda # pvcreate /dev/loop0 Physical volume "/dev/loop0" successfully created # vgcreate evg0 /dev/loop0 Volume group "evg0" successfully created # vgchange -a y evg0 0 logical volume(s) in volume group "evg0" now active # lvcreate -L3500 -nrootvol evg0 volume "rootvol" created # lvcreate -L1000 -nvarvol evg0 Logical volume "varvol" created # lvcreate -LR100 -ntmpvol evg0 Logical volume "tmpvol" created ## see how much free space is left # vgdisplay evg0 | grep Free Free PE / Size 176101 / 687.89 GB ## NOTE: the below command uses a lowercase 'l' instead of the uppercase 'L' ## lowercase 'l' == extents, uppercase 'L' == size in Mb # lvcreate -l175600 -nhomevol evg0 Logical volume "homevol" created ## now create the filesystems # mkfs.reiserfs /dev/evg0/rootvol # mkfs.reiserfs /dev/evg0/varvol # mkfs.reiserfs /dev/evg0/tmpvol # mkfs.reiserfs /dev/evg0/homevol ## and mount them # mount -t reiserfs /dev/evg0/rootvol /newroot # mkdir /newroot/tmp # mount -t reiserfs /dev/evg0/varvol /newroot/var # mkdir /newroot/home # mount -t reiserfs /dev/evg0/tmpvol /newroot/tmp # mkdir /newroot/var # mount -t reiserfs /dev/evg0/homevol /newroot/home |
You can now install the base system, untaring the base_install.tar.gz created in the first step into /newroot/.
Now you should edit some files, like the host /etc/fstab if you want your system to be able to mount file systems! With this system, you access file systems through LVM, so the fstab will be a regular LVM compliant one.
| Content of the host /etc/fstab |
/dev/evg0/rootvol / reiserfs defaults 0 1 /dev/evg0/varvol /var reiserfs defaults 0 2 /dev/evg0/tmpvol /tmp reiserfs defaults 0 2 /dev/evg0/homevol /home reiserfs defaults 0 2 |
Umount everything, and you're done, you can halt and create the boot key.
If you use initrd instead of initramfs for the USB key which will really boot the system, it must be as small as possible, because the initrd cannot be umounted, since it has the loop device active. It will stay in RAM forever.
For this step, I took the setup initrd and removed all unneeded files for the only boot phase, bringing the image to around 5MB. Files need to be added: the master encryption key, your GPG home, and the run-init executable for initramfs replacing pivot_root for initrd. run-init can be found in klibc-utils, but the /lib/klibc-something.so has to be copied too. Here's what I come to (which can be probably optimized, there still must be unused files, and lighter executables such as full klibc or busybox with µClibc static can be used):
| initramfs's files for boot key |
.: bin boot dev etc init lib mnt newroot proc root sbin tmp usr var ./bin: ash chmod cp grep ln login mknod mv rm sleep sync busybox chroot gpg hostname loadkeys mkdir mount pwd sh stty umount ./boot: default.kmap ./dev: console kmem loop0 mem ram0 sda1 tty tty1 initctl loop mapper null sda sdb tty0 ./dev/loop: 0 ./dev/mapper: control ./etc: group inittab ld.so.cache lvm passwd shadow ./etc/lvm: ./lib: klibc-B9LS-Gjx2D7BYcbQig0RlgHKO9Y.so libctutils.so.0 libselinux.so.1 ld-linux.so.2 libdevmapper.so.1.02 libsepol.so.1 libblkid.so.1 libdl.so.2 libuuid.so.1 libcfont.so.0 libm.so.6 lvm-0 libconsole.so.0 libncurses.so.5 lvm-200 libcrypt.so.1 libpthread.so.0 lvm-default libc.so.6 libreadline.so.5 ./lib/lvm-200: lvchange lvmchange lvrename pvdisplay vgcfgbackup vgdisplay vgreduce lvconvert lvmdiskscan lvresize pvmove vgcfgrestore vgexport vgremove lvcreate lvmsadc lvs pvremove vgchange vgextend vgrename lvdisplay lvmsar lvscan pvresize vgck vgimport vgs lvextend lvreduce pvchange pvs vgconvert vgmerge vgscan lvm lvremove pvcreate pvscan vgcreate vgmknodes vgsplit ./newroot: ./root: .gpg/ master_disk_key-0x01234567.gpg ./sbin: depmod fsck.reiserfs losetup lvmiopversion run-init vgscan fsck getty lvm reiserfsck vgchange ./tmp: ./usr: sbin ./usr/sbin: chroot ./var: lock run ./var/lock: lvm ./var/lock/lvm: ./var/run: utmp |
As you see in /root/ you have to put your files, which are your gpg homedir and your master encryption key. Try to not backup this latter on a non-encrypted media.
You may try to strip binaries, operation which removes debug information and symbols from binary files, and make them smaller, which is what we seek.
The content of /etc/rc must be changed and the file is now /init which is the default for initramfs (replace your real key name).
| /init boot script for the boot USB key |
#!/bin/sh
mount -nt proc proc /proc
mount rootfs / -o remount,rw
/bin/hostname izakar
/bin/loadkeys /boot/default.kmap
/sbin/losetup -G /root/.gpg -K /root/master_disk_key-0x01234567.gpg \
-e AES256 /dev/loop0 /dev/sda
if [ $? != 0 ]; then echo "Unable to mount loop device. Rebooting."
exit 1; fi
/sbin/vgscan
/sbin/vgchange -a y evg0
mount -t reiserfs -o ro /dev/mapper/evg0-rootvol /newroot
umount /proc
exec /sbin/run-init /newroot /sbin/init "$@" \
< /newroot/dev/console > /newroot/dev/console
|
The rootfs from the initramfs can be mounted read/write, so tmpfs is not needed anymore for /dev /var and /etc. I did not test this -o remount,rw trick with initrd, but maybe it can work too. The run-init binary removes all files from the initramfs before remountint the hard drive's / in place. It then executes /sbin/init and the OS begins to boot.
You can find my image, the one listed above here.
Now you can create the ramfs.gz file from the directory where you have put all files with: find . | cpio -o -H newc | gzip -9 > ../ramfs.gz . Make sure you don't put a filename longer than 8 characters without the extension because it will be copied on a msdos filesystem which does not supports longer. This kind of file can be unrolled with: cd fs && gunzip -c ../ramfs.gz | cpio -iv --no-absolute-filenames .
Copy the files the same way than for the setup USB key. Change the syslinux.cfg to: default vmlinuz initrd=ramfs.gz and rerun syslinux /dev/sda, if sda is you USB key, make sure you don't run it on your hard drive!
Now you can boot on your boot USB key. It will ask you the gpg passphrase protecting the hard drive's encryption keys when it boots. If you fail, you have to reboot and retry, there is only one try. You can modify the /init script to get better chances. The initramfs then mounts the real root filesystem and gives hand to the host OS.
When your system is booted, you can remove the USB key and keep it safe, far from the drive. Don't make any copy of the encryption key on a non-crypted unremovable drive. I think that you should make 2 copies of the boot USB key, and keep them in safe different places.
If you try to boot on the system without the key, the hard drive will not be seen as a bootable device. Without the encryption key and the passphrase, your data will be safe.
For the funny story, after I did all that, at my first boot, I tried to copy a file from the network with ftp. At around 220MB, it crashed with a kernel panic, caused by heat. And this is when you figure out that this fanless board cannot be fanless :)
cpufreq. If you have a Jetway mini-itx board like the J7F4K1G2, you can use this script to change the power allocation (cpufreq) for the cpu.
| power.sh script |
#!/bin/sh if [[ $# == 1 ]] then echo "writing config..." echo "$1" > /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor fi cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_available_governors echo -n "==> " cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor | tr '\n' ' ' FREQ=$((`cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_cur_freq` / 1000)) echo "(running at $FREQ""MHz)" |
It will display available policies for cpufreq (the one you insmoded), like performance, powersave, or ondemand. Performance always runs the CPU at 1200MHz, powersave always at 400MHz, and ondemand switches to 1200MHz on load, which decreases power consumption on idle. The power.sh script can take an argument which is the name of a policy and makes it the in-use policy.
Sensors. Install the package lm-sensors. The sensors command will display voltages and temperatures from the onboard sensors. This needs to load the modules indicated by sensors-detect (f71805f, hwmon, i2c_dev, i2c_viapro, and i2c_core). You might also install hddtemp to get the internal temperature of your hard drives disks.
OpenSSL. OpenSSL and OpenSSH can use the padlock engine to encrypt and decrypt data. Last mainstream version do, but debian's stable (etch) does not, and needs a patch (BTW, I am not able to build the debian stable package with the patch, it segfaults). Next stable (lenny) has openssh patched, but openssl is still requested to be patched to support it. This is trivial, and this time I am able to build the patched package. You should look at here for instructions and this is the bug report for lenny.
SHA-1 sum. The original coreutil sha1sum does not support Padlock Hash
Engine. However, you can replace it by Michal Ludvig's implementation phe_sum. I have
run a simple benchmark on a 776MB file, it takes 49s with sha1sum, and 22s with
phe_sum. Unfortunately, SHA-256 support was not implemented.
The loop-aes README, a must read!
Why use loop-aes and not cryptoloop: here. This is not really relevant now because cyptoloop has been deprecated by dm-crypt, which was designed to support advanced modes of operation, such as XTS, LRW and ESSIV, in order to avoid watermarking attacks. The wikipedia page on disk encryption theory and lots of its links are very useful to understand all issues of disk encryptions, and different methods. The Arch Linux wiki page on System Encryption with LUKS for dm-crypt will help you with dm-crypt, and there is a little comparison between different approaches.
Project Naranja which is exactly the same thing, but was not available when I did this. See this link too.
The linux documentation project Encrypted Root Filesystem HOWTO
Linux documentation on initramfs and initrd.
Help on creating initrd: Booting Debian with root filesystem on LVM
Why it is a bad idea to buy a Jetway motherboard: design is so lame...
Programming with padlock: http://www.logix.cz/michal/doc/article.xp/padlock-en
GrouNation blog: crypto, padlock, TPMs with linux.
Inside NetBSD's CGD, a talk about crypto disk drivers of diverse OSes, and the implementation of NetBSD's CGD. http://www.onlamp.com/pub/a/bsd/2005/12/21/netbsd_cgd.html
I had a chance to get hands on a few CPUs that have hardware accelarators for cryptographic operations such as AES encryption and SHA hashing. Since I could not find this information for all of them on the Web, here's a list, to be updated when more is available. Some use the Padlock engine, some Intel AES-NI. Data was obtained with the cryptsetup benchmark command.
Like the Via C7 processor, it features the Padlock engine for acceleration of 128-bit or 256-bit encryption operations, not 512 and not in other modes than cbc.
PBKDF2-sha1 289982 iterations per second
PBKDF2-sha256 195047 iterations per second
PBKDF2-sha512 116198 iterations per second
PBKDF2-ripemd160 193893 iterations per second
PBKDF2-whirlpool 55727 iterations per second
# Algorithm | Key | Encryption | Decryption
aes-cbc 128b 396.6 MiB/s 444.5 MiB/s
serpent-cbc 128b 22.2 MiB/s 47.0 MiB/s
twofish-cbc 128b 53.1 MiB/s 56.2 MiB/s
aes-cbc 256b 380.9 MiB/s 379.1 MiB/s
serpent-cbc 256b 26.8 MiB/s 47.0 MiB/s
twofish-cbc 256b 56.4 MiB/s 56.2 MiB/s
aes-xts 256b 92.6 MiB/s 92.4 MiB/s
serpent-xts 256b 43.3 MiB/s 43.5 MiB/s
twofish-xts 256b 50.9 MiB/s 51.0 MiB/s
aes-xts 512b 67.5 MiB/s 67.5 MiB/s
serpent-xts 512b 43.6 MiB/s 43.5 MiB/s
twofish-xts 512b 51.1 MiB/s 51.0 MiB/s
|
Also called Avoton, this family of Atom processors is not as bad as its predecessors, but still terrible. It is clearly outperformed by the Via Nano on all accelerated operations. AES-NI supports 512-bit keys operations and xts mode, which gives it a big advantage over the Padlock. Non-accelerated operation are only 20 to 50 percent faster than the Nano above, which tells you how bad this processor is.
PBKDF2-sha1 134295 iterations per second
PBKDF2-sha256 80709 iterations per second
PBKDF2-sha512 55727 iterations per second
PBKDF2-ripemd160 120470 iterations per second
PBKDF2-whirlpool 43574 iterations per second
# Algorithm | Key | Encryption | Decryption
aes-cbc 128b 241.0 MiB/s 332.0 MiB/s
serpent-cbc 128b 30.8 MiB/s 74.7 MiB/s
twofish-cbc 128b 67.7 MiB/s 68.5 MiB/s
aes-cbc 256b 182.0 MiB/s 260.0 MiB/s
serpent-cbc 256b 30.9 MiB/s 75.0 MiB/s
twofish-cbc 256b 67.7 MiB/s 68.2 MiB/s
aes-xts 256b 292.0 MiB/s 292.0 MiB/s
serpent-xts 256b 69.4 MiB/s 69.0 MiB/s
twofish-xts 256b 63.0 MiB/s 63.0 MiB/s
aes-xts 512b 233.0 MiB/s 234.0 MiB/s
serpent-xts 512b 69.7 MiB/s 69.4 MiB/s
twofish-xts 512b 63.2 MiB/s 63.2 MiB/s
|
This article is an update to this tutorial that is getting a bit outdated: Setting up my server: re-installing on an encripted LVM.
Debian stretch has new features that change the way to configure the initramfs and networking. It took me a while to figure them out, so this may be useful to somebody else.
The initramfs configuration has changed, especially for the dropbear part. There is now a package that enables dropbear only in the initramfs, dropbear-initramfs, and it almost does not require manual configuration. Only the ssh key has to be put in /etc/dropbear-initramfs/authorized_keys instead of directly in the initramfs files /etc/initramfs-tools/root/.ssh/authorized_keys. One reason for that is that the root homedir is not /root in the initramfs anymore, it's randomly generated when running update-initramfs. I haven't checked why, but it's probably for security reasons.
It is very important to know that since stretch, debian stopped using eth0 as default network interface name. It now uses a name generated with the MAC address, to have unique identifiers for network interfaces, as announced here. If you are installing the server and if you don't have a remote console to check the name, the best is to use dhcp the first time to be able to connect to the machine. Make sure you use that name in the kernel command line otherwise you won't be able to ping your machine or unlock the encrypted filesystem. Take also care of the /etc/network/interfaces for the regular system boot.
In case of problem, you may have access to a serial console. It's easy to configure, just add console=ttyS0 to the kernel command line in /etc/default/grub if you use grub2. You may also want to configure grub itself to interface with the serial console, add GRUB_TERMINAL=serial to the file, with possible parameters in the GRUB_SERIAL_COMMAND variable. If you don't see anything on the console it's probably because it's the other port...
In my case it was the Intel Atom Avoton-based server from online.net, the network interface is called enp0s20 and the serial port of the console is ttyS1.
Configuring postfix for SASL authentication can be quite tricky, especially with virtual aliases domains. Fortunately, the documentation is quite complete and help can be found in a few other places.
This article reviews the configuration for the sasldb plugin of libsasl, integrated in postfix built with Cyrus, as found in modern OS such as Debian 8 in a virtual domain set-up. sasldb is the simplest authentication method for postfix: it stores usernames and passwords in a database file and requires no external authentication service. Although simple and documented, this approch remains quite unclear on some points, or even wrongly and contradictorily documented and prone to error. The following sections will tackle all issues that were met, and solved in this serverfault question.
It is not needed to install or start saslauthd in order to use the simple sasldb authentication database. Support for this authentication method is built in libsasl. A simple mention of the method in the smtpd.conf file is enough.
What the hell is this file, and where should it be put? The first answer is found in the README, the file is the data exchange format between postfix smtpd and libsasl. I found at least three locations for it, with no clear indication of where it should go. Serverfault was helpful: there is an extra configuration variable in postfix that never appears in tutorials, that defines the path for the file: cyrus_sasl_config_path.
The sasldb file is created with the saslpasswd2 command. The realm argument is particularly unclear, especially the way postfix specifies it in the config file with smtpd_sasl_local_domain. The content of that variable will be the default domain users will authenticate to. For example, if set to "w.tf", a user john trying to authenticate with the john username will be actually checked as john@w.tf the sasldb. If you only manage one domain, using this option will facilitate login for users. Otherwise, leaving the variable empty will require users to provide the exact value of sasldb usernames, in our example "john@w.tf".
The -u argument of saslpasswd2 is optional. The commands saslpasswd2 -u w.tf john and saslpasswd2 john@w.tf are equivalent. With these two, programs will have to provide the entire username when authenticating, in our case postfix can be providing the domain part depending or not if smtpd_sasl_local_domain is configured.
Several causes for authentiation failures were met in the first configuration try. First, the chroot, as mentioned in some forum threads. Postfix smtpd runs by default in chroot. Consequently, the sasldb file must be placed inside the chroot or the smtpd has to be run without the chroot. Disabling the chroot is easily done in the master.cf configuration file. I would recommend using the chroot and moving the sasldb in the chroot anyway, for security reasons. For convenience, a symlink can be done in /etc/sasldb2 in order to have the basic sasl commands still working, with default chroot path: ln -sf /var/spool/postfix/etc/sasldb2 /etc/.
The most troubling issue for me was when the system was actually working but the test was not. The official documentation presents various ways of testing the AUTH PLAIN command, using perl or shell base64 string construction. These methods didn't work for me, only the following command did: gen-auth plain. On authentication failure with the other methods, these warnings were appearing in the mail error log:
warning: SASL authentication failure: Can only find author/en (no password) warning: localhost[127.0.0.1]: SASL PLAIN authentication failed: bad protocol / cancel |
Configuring myorigin, myhostname, mydestination and mydomain is a bit tricky too. From the serverfault replies, I made some tries and I think using one of the actual hostnames of the machine as myorigin and myhostname is the correct way to proceed. It may have two drawbacks however: the outbound emails will appear in headers as sent by your user@the.real.hostname.of.the.machine instead of user@the.mail.domain and this real hostname must not be in the list of virtual domains managed.
The problem with the latter is that myorigin is appended automatically to local users with no domain in the virtual alias maps, so you wouldn't be able to deliver to local users who have UNIX accounts if the content of myorigin wasn't also set as one of mydestination. And you can't have a virtual domain in mydestination. Also, if you don't have a real hostname that is not also present in the list of virtual domains, and just use localhost instead, some remote servers could reject your emails. And it can be a concern to have the most neutral hostname possible, because when you are managing at least two domains, you don't really want the emails from domain1 to appear as sent by a machine of domain2. So what I did was to use the default reverse DNS for the server, provided by the ISP/hosting service, which also may not be always possible.